Within Microsoft ATP, users can control over how aggressive the phishing protections are. There are four options, ranging from the standard default to most aggressive.
Most aggressive means that any messages identified as potential phishing are treated as if there was a high confidence in that decision. In other words, even if there is low confidence that the message is phishing, it is treated as if Microsoft has high confidence that it was phishing.
Sounds good, no? Aggressive protection sounds like a great way to keep your inbox secure.
In reality, Office 365's ATP high confidence setting can lead to more phishing email false positives than useful protection.
In its own documentation, Microsoft says not to use the fourth option. The reasoning? An increase in false positives.
This is the trade off: If you make ATP more aggressive, you will also see more false positives.
When we did our 2020 ATP Report, which analyzed 500,000 malicious emails sent to real end-user mailboxes protected by ATP, we found that the organizations we looked at were split between the Aggressive (2) and Most Aggressive (4) policy settings. We found that the Most Aggressive setting did little to actually improve protection, but it did lead to an increase in false positives.
We also found that the higher the ATP sensitivity, the greater tendency to have longer and more permissive Allow Lists there are. That’s another issue: we found that a portion of malicious email (1.2%) reached user inboxes due to overzealous Allow Listing. Attackers know how to take advantage of his vulnerability.
Because there are so many false positives, users are going into the Junk folder, dumpster diving for treasure—but often mistaking it for trash. Even your most well-trained employees can mistake malicious emails for seemingly normal ones.
The Re:Ploy attack took advantage of this. The attack worked by having a hacker impersonate an external partner email to infiltrate an organization. It was particularly clever in how well it impersonated someone the recipient knows.
It was so effective that even after Microsoft picked up on it and started blocking it, users still went through their junk folder and clicked on it.
It’s why sending everything to Junk, which we coined as the ATP Dumpster Diving Policy, is so ineffective. And when you pair that with a high false positive rate, your users will undoubtedly click on bad emails, because they’re expecting good, legitimate ones to be there instead.
That’s one of the reasons we started our Incident Response-as-a-Service (IRaaS), which leverages our team of experts to manage end-user reports of suspicious emails and requests to release quarantined emails. We can deal with false positives with no pain and no build-up for your help desk team.
And when you use Avanan, you’re also utilizing our false positive engine. Our AI has been trained on years of historical email and false positive reports by our users. Within hours of deployment, our AI scans a year’s worth of emails to build a reputation network. This helps further reduce false positives, in a process that typically takes months for other security solutions.
And because we sit behind ATP, we can catch the attacks that ATP misses, again limiting our false positive rates.
It’s time for a solution that doesn’t mistake quality protection with false positives.
