Show the Value of Your Email Security Solutions: Don’t Just Measure Detection Rates
As organizations become more data-driven, it’s important to be able to point to metrics that demonstrate security programs are working. This is especially true for email security solutions. Typically, everyone in the organization uses email, so security programs are highly visible and people have firsthand experience with them.
Detection rates are the go-to metric for many vendors (and consequently for many security teams), but they don’t tell the whole story. Here are some other ways you can show the value of your email security program.
Native Cloud Email Metrics
Few companies rely solely on their cloud email provider’s native security; however, there are definitely things it does very well—like filtering out SPAM. True, SPAM isn’t malicious; however, a deluge of unsolicited emails fatigues users, which can lower their ability to spot risky emails.
Native email security in cloud programs, like Microsoft 365 and Google Workspace, is also good at detecting malware in attachments. We noted in our 2023 State of Email Security Report that improvements in both vendors’ attachment scanning resulted in a dramatic decline in attachment-based attacks reaching users.
Paired with these programs’ ability to scan URLs and match them against “known bad” lists, these capabilities together filter out around 40–50% of emails, removing and/or quarantining them to prevent user engagement with known threats or SPAM. Something to consider as you look at your own quarantine rates.
Learn how to determine the value your current email security solution is delivering.
Adding Artificial Intelligence and Machine Learning
While matching individual characteristics against known bad is where native email security provides significant value, machine learning (ML) and artificial intelligence (AI) often earn high marks for their ability to spot the anomalies present in riskier emails.
Whether part of legacy secure email gateways (SEGs) or harnessed as part of modern Integrated Cloud Email Solutions (ICES), AI/ML are able to analyze numerous characteristics such as domain reputation, your interaction history with the sender, and keywords and phrasing that is indicative of phishing (i.e., topics that are common in phishing scams or words used to convey false urgency).
What should be concerning to you is that roughly 1 in 500 emails (about 0.2%) has anomalous characteristics that bypass both native email security systems and SEGs. This ratio can quickly become concerning in large enterprises that see tens of millions of emails each month.
API Email Security Solutions Raise the Bar
Email security solutions that connect to cloud-native email platforms using APIs are becoming the preferred method for implementing email defense-in-depth. Layering on tactics that strategically align to your organization’s risk tolerance is a key differentiator.
API-based solutions raise the bar and provide strong value by applying automated layers (i.e., customized smart banners, link/URL protection, and/or quarantine) to align with “how” the organization wants to customize risk. Organizations should be able to vary control by role, for instance, to align with the varying risk tolerance for specific individuals, departments, or company-wide.
Those fine-grained risk controls, when layered with native security controls, enhance detection and provide measurable risk reduction across the multiple layers. GreatHorn’s analysis shows that combined layers of protection (beyond the deterministic model that only incorporates quarantine) results in a 99.8% risk reduction to an organization.
Causes for Concern
As security and risk professionals, detecting 99.8% will always be of concern, as the 0.2% represents the most sophisticated risks to users and organizations alike. However, as business leaders responsible for hitting budgets, managing teams, enabling productivity, and driving growth, showing the value of your email security solution may also take different forms.
Take false positive rates as an example. Across the vast majority of email security solutions, the deterministic approach to bad/good based on ML/AI creates higher false positive rates. This leads to overburdened security teams that spend a disproportionate amount of time to release emails from quarantine that are required for legitimate business purposes, slowing down the business.
User engagement can be another illuminating metric—for instance, tracking employee engagement with banners for signs of banner fatigue. As users become used to seeing the same banners, habituation causes them to fail to respond to the warnings provided. The Wall Street Journal recently shared research that showed by varying the banners’ appearance, “habituation occurred much less and adherence improved by more than 20 percentage points (87% compared with 64%) over a three-week period.” 1
The steep cost associated with Business Email Compromise (BEC) and Email Account Compromise (EAC), estimated by the Federal Bureau of Investigation as totaling nearly $2.4 billion in losses for just 20,000 incidents, highlights the importance of email security for every organization. However, the absence of an incident isn’t sufficient to demonstrate the effectiveness of your email security program. Develop a more balanced scorecard to show the full value of your email security, looking at each layer of your defense-in-depth, along with highlighting otherwise hidden benefits or costs.
Learn how to determine the effectiveness of your email security solutions.
Download “Measuring the Value of Email Security Solutions” guide
GreatHorn Resources
Supply Chain Cyberattacks:
2021 Trends You Need to Know
Breaking the Phishing Attack Kill Chain:
Gaining Control Over Phishing
Combating Advanced Phishing Attacks
with Modern Cloud Email Security
