Avanan created the API-based email security field in 2015.
Back then, our solution had only one mode. Here's a quick overview of how it worked.
While revolutionary at the time, it had two serious limitations. And it's why we now offer more options for our customers.
1. Inbox Rules and Timing
The first is inbox rules and timing. In a typical API-based system, here's the flow.
The issue here is timing. The idea is to remove the message, if malicious, before the user sees it. That's a great idea, but in practice there are issues.
First, the two systems have to establish a connection. That can take a second, but it can take even longer when hundreds of connections need to be made.
After the connection is made, the email in question needs to be downloaded. The connection needs to be terminated. Then it needs to be scanned. If the email is malicious, then another connection needs to be made in order to quarantine. Then it needs to be quarantined.
So once it's scanned and analyzed, it takes a millisecond to remove it from the inbox. But it has to get to that point first.
There are other limitations here as well, including not allowing outbound scanning or DLP. And we haven't even gotten to the agent sync problem (which we'll detail in an upcoming blog.)
Then, there's a second limitation.
Limitation 2: API Performance
Microsoft’s throttling is now well documented, and we’ll get to it soon. What is not documented by Microsoft is that the API will start experiencing delays when the tenant is loaded. You will get reply failures to API calls and even dropped events - the API will just ignore certain messages.
What Microsoft did document is a throttling mechanism to the API, designed to protect its servers from receiving too many requests. The exact implementation of how the throttling works is naturally only known to Microsoft, but they do acknowledge throttling is a challenge for anyone planning to use their API (Microsoft Graph - Don't Get Throttled!).
There are generally three thresholds you could run into.
The first limit is the Per app across all tenants. What it means is that the vendor that provided you an app has a limit across all their install-base of 2,000 calls per second. It might sound a lot but what it means is that even if another customer of your vendor is overloading the API, for example because of email loops, you might get throttled.
The second limit is the Per app per mailbox set at 10,000 per 10 minutes. To receive the information about an email (including its attachments, etc) there are roughly 4-5 API calls needed, so this number translates to about 2,000 emails per 10 minutes or 3 per second. Again, most of the time it’s fine but over time across all mailboxes, you’ll get throttled.
But perhaps the most alarming throttling limit is an undocumented one: Microsoft doesn’t tell us when or how, but rather declares they will throttle the API if the tenant is experiencing high load, or if their entire service is under load. As described in their documentation:
In other words, API is second priority to email processing. If there’s load, then API is the first to suffer. This makes sense from an implementation standpoint, but for email security to rely on a second-priority API is very risky.
The impact of throttling is devastating. Once it happens, it is pretty unpredictable when the service will resume and we have seen it take 24 hours or more. The failure reply from Microsoft will suggest the delay before retry, but it is just a recommendation and in fact, getting failed retries will only extend the penalty time. While waiting to retrieve old messages new emails continue to come. The queue, recovery time and total delay only gets longer. Across multiple requests, in a large environment, it basically means the service is no longer operational.
And here's what's important to know: These two limitations still apply to all other API competitors.
How Avanan is Different
Avanan added its patented inline protection to avoid those two limitations. While Avanan can still operate like the other API solutions, 95% of our customers choose our inline mode, because it's scalable and offers preventative security.
Here's what it looks like:
Avanan is using the Microsoft API across the system but not for the real-time email retrieval that needs to work at wire speed.
- We use API to set inline scanning - this is our patent
- We use API to apply the configuration so you never need to open the Office 365 console. However, it is a one-time operation during the initial setup
- We use APIs to pull the user lists and groups, their titles, etc, all to provide the context and social graphs for better analysis. But that’s an offline process with a very limited number of calls
- We use API for the manual quarantine as part of our “search and destroy” for post-delivery analysis. But that’s triggered by the admin and for a select set of emails, a far smaller subset than every email coming in
For real-time emails - we use SMTP. It’s scalable and it can be inline.
In simple words - if it cannot guarantee performance during load - it cannot scale for services that require a timely response.
This is the reason Avanan implemented a unique mode that:
Applies all configurations via API - to deploy in a click
Use API to scan existing data (E.g. historical email) and contextual data (configs, users, groups, etc), for full visibility
And get real-time emails via SMTP - because it allows inline and because it can scale
These limitations still apply to the other API-based email security services–no matter what they tell you otherwise.