Highlights

• Microsoft’s Office 365 ATP policies flag an unusually high percentage of legitimate emails as phishing detections
• Organizations that set their ATP policies to “Quarantine” spend considerable time fielding requests from end users to release legitimate emails (unless they take Microsoft's awful recommendation and allow "Self Release from Quarantine") 
• The alternative is to have ATP detections sent to the Junk folder, which we term the 'ATP Dumpster Dive Policy' 
• Most revert to the 'ATP Dumpster Dive Policy', whereby marketing emails, subscriptions and targeted phishing attacks are commingled in the Junk folder and immediately accessible to the end user
• The optimal solution is to deploy an advanced phishing solution that can understand the difference between a targeted phishing email and a newsletter

Avanan’s analysts have seen a resurgence in companies and organizations using the Junk folder as a way to help protect end users from phishing attacks. After analyzing over 500,000 malicious emails sent to real mailboxes protected by Microsoft’s Advanced Threat Protection (ATP) for our 2020 ATP Report, we think we know why. ATP customers are choosing from two bad options: Send a lot of legitimate emails to Quarantine or send targeted phishing attacks to Junk.

Our analysis found that 95% of emails detected by ATP policies are nothing more than marketing emails, newsletters, subscriptions and more. Some of these may be of legitimate business purpose. Very few are actually phishing emails.

Sending ATP detections to Quarantine introduces a great risk of blocking legitimate emails to your users. Unfortunately, by sending everything to the Junk folder, all emails, including malicious ones, are accessible to your end users. That means that it’s impossible to keep phishing attacks out of their reach.

Users are accustomed to dumpster diving in the Junk folder for legitimate messages. However, there’s a lot of trash to root through when looking for that one piece of treasure. And it increases the likelihood that a phishing email is acted upon by a harried employee.

Bottom line: It’s not enough to simply rely on your Junk folder to stop phishing attacks. You need a comprehensive solution that keeps phishing attacks out of reach of the end user, and out of their Junk folder.

ATP's Policies

Office 365 ATP works like this: phishing and malware policies can be configured with certain actions. For example, the ATP Domain Impersonation policy can be configured to quarantine emails, send them to the Junk folder, append the text to the subject and more.

While most companies are training employees to spot malicious emails, that training can only go so far. When scouring through hundreds of emails in the middle of a stressful workday, even the most well-trained employee can still click on a malicious email. So there’s a good chance that they’ll mistake trash for treasure, reinstating the bad email and potentially infecting themselves and the rest of your organization.

You have three options.

The first: Don’t quarantine anything, and let everything, phishing included, go into your end-users’ inboxes. That doesn’t really work—what’s the point of paying for ATP if you’re letting everything come into your inbox?

"Avanan consistently blocked malicious emails that were not caught by Microsoft."
Gartner Peer Insights
Cybersecurity Engineer, Midsize, Media

The second option is to quarantine everything, legitimate or not. We’ve seen this especially with organizations using ATP for the first time. They implement the quarantines without really understanding that legitimate emails are disappearing. In a few weeks, many revert back after their help desk receives scores of angry messages about important emails gone missing.

That brings us to the third option—the ATP Dumpster Dive Policy. This has forced frustrated companies to simply send everything to Junk, letting end users act as a de-facto help desk, deciding which emails are safe and which are phishing. That is not a tenable or lasting solution. Consider the Re:Ploy email chain hijack attack, in which a hacker impersonates an external partner email to infiltrate an organization. Even after Microsoft learned to catch it, the attack remained just as effective as before, because most people found it in their Junk folder and opened it anyway. Having your users be your enforcement tool can have disastrous results.  

As one CIO of a Fortune 500 company told us, “You now have monthly subscriptions, newsletters and targeted phishing attacks in your spam folder, and you have to leave it up to the end user to decide which ones are safe to open.”

Sifting Through the ATP Results

ATP policies are being triggered by clean, safe emails. A lot of them. Our numbers show us that only 2-4% of emails that violate ATP policies are actually phishing.  In one trial we did, the numbers were even lower.

And while ATP is busy identifying clean emails to investigate, they are letting targeted attacks slip right into user's inboxes. In a recent trial we conducted, of the attacks we caught where ATP was fully configured and tuned, 19% went straight to the inbox, while 81% went to Junk.

Remember, Avanan runs behind ATP so we see the attacks they miss. In other words, Microsoft missed a significant number of targeted attacks that went straight to the inbox and then littered end-users’ Junk folders with malicious emails. We often ask our clients: “Can you do better than putting phishing emails into the Junk folder as a last resort?”

How Avanan Fixes This

Avanan does things differently. It’s integrated within Office 365 as a layer behind EOP and ATP. Our AI is trained specifically on the sophisticated attacks that were designed to evade default Office 365 EOP and ATP. We do this without hindering their ability to filter out spam and phishing. Avanan also learns from the cyberattacks missed by other security vendors who also integrate with O365,  such as Mimecast or Proofpoint. That means the AI is tuned to what other providers miss.

Avanan catches the targeted attacks that EOP and ATP miss. Because of that, we can limit our false positive rates, meaning your protection is broader and more effective. Your users don’t have to fend for themselves because the Avanan solution is meant to be a unified, all-in-one protection platform.

Beyond that, Avanan has more flexibility in terms of implementing custom policies that work best for your company.

Conclusion

Sending your emails to the Junk folder puts too much control in the hands of end users, and comes with the risk of employees missing out on important business emails. By our analysis, ATP misses far too many attacks, with too many false positives. This then requires customers to throw all ATP detection into Junk, where they commingle with dangerous messages in a recipe for accidental disaster.

Relying on Junk as your filter isn’t an effective enforcement policy. You need a solution that filters emails correctly and is set up to protect you from attacks, one that solves the problem at the outset, not leaving it to undertrained front-line users.

Trust the experts. Let our specially-tuned system do the hard work for you. Keep your employees focused on their job, not rooting through the trash.