MITER’s MDR stress test winners combine human intelligence and AI for stronger cybersecurity

Cyberattacks succeed by using social engineering and spear-phishing to find and exploit gaps in corporate IT environments, endpoints and identities. Often, they launch persistent threats immediately and then steal credentials to move sideways across networks unnoticed. MITER chose this series of offenses because of its first closed book “MITER ATT&CK Assessments for Security Service Provider.”

The purpose of the ATT&CK evaluation is to test the effectiveness of providers’ cybersecurity. How ready, proficient and accurate are these solutions in identifying and stopping an attempted breach without know when and how it will happen?

MITER Engineering ATT&CK evaluations are based on a knowledge base of tactics, techniques and sub-techniques to keep evaluations open and fair. MITERs ATT&CK Matrix in front of Company is the most widely used framework for evaluating business systems and software security.

Stress testing of managed services and MDR

Historically, MITER ATT&CK assessments have informed security vendors in advance — before active testing — which intrusion and breach attempts they will be tested for and why. With that advanced information, sellers are familiar with game evaluations, leading to inaccurate results.

In a closed book evaluation, vendors have no prior knowledge of the threats they will face in the test. MITER ATT&CK Evaluations for Security Service Providers is the first closed-book evaluation designed to test the technical effectiveness and real-world capabilities of vendor Managed Services or Managed Detection and Response (MDR) solutions.

>>Don’t miss our new special issue: Zero trust: the new security paradigm.<

Closed book evaluations provide the most realistic representation of how a security vendor would perform in a customer environment. “The closed-book test provides an opportunity to demonstrate how security platforms work against hostile commerce in a real-world setting, as sellers have no prior knowledge to guide their actions,” said Michael Sentonas, chief technology officer at CrowdStrike.

MITER’s assessment of MDRs is particularly relevant as chronic cybersecurity skills shortages put organizations at greater risk of breaches. According to the (ISC)² Cyber ​​Security workforce Study“An additional 3.4 million cybersecurity workers are needed to effectively protect corporate assets.” Managed detection and reaction (MDR) offers organizations an effective way to close the skills gap and improve business resilience.

The MITER Security Service Providers evaluation lasted five days, with a 24-hour reporting window. Sixteen MDR suppliers participating in the program had no prior knowledge of the adversary or its tactics, techniques and procedures (TTPs). They were each judged on 10 steps consisting of 76 events, including 10 unique ATT&CK tactics and 48 unique ATT&CK techniques.

“We selected OilRig based on their defense evasion and persistence techniques, their complexity and their relevance to various industries,” writes Ashwin Radhakrishnan of MITER Engenuity. The first round of MITER ATT&CK evaluations tested suppliers through the TTPs of Oil platform (also known as HELIX KITTEN), the hostile group with operations aligned with the Iranian government’s strategic objectives.

The attack scenario began with a spear-phishing attack against a national organization using malware related to HELIX KITTEN campaigns. Then, the simulated threat attack caused a lateral movement across networks to identify and collect critical information, with the ultimate goal of data exfiltration.

The combination of human intelligence with AI and ML produces the best results

MDR vendors with multiple product generations of platform and managed services experience, using a combination of artificial intelligence/machine learning (AI/ML) and real-time human intelligence, topped the MITER evaluation. The top four vendors that discovered the largest number of the 76 hostile techniques were CrowdStrike Falcon Complete, Microsoft, SentinelOne, and Palo Alto Networks.

These MDR providers rely on insights and intelligence from senior security analysts who use AI/ML apps and techniques designed to analyze telemetry captured from endpoints, networks, and cloud infrastructure. The result: AI-assisted threat hunting expertise that enables their solutions to identify and thwart breaches.

MITER Engenuity summarizes the test results ATT&CK® Evaluations: Managed Services — OilRig (2022) and the 10 best ways to interpret the results. This document provides an overview of the methodology and the interpretation of the results. MITER also makes the layer file image available for further analysis in its ATT&CK Navigatorshown below.

The results of the 16 vendors who participated in the MITER ATT&CK assessments for security service providers showed what factors enabled vendors to do well. Vendors who have done the best are experienced operators of their own security technologies. They offer a holistic range of capabilities from across their security portfolios. These vendors consistently produced the best security results with the highest detection coverage in the study.

CrowdStrike led all vendors in this category by reporting 75 of the 76 consulting techniques used during the MITER ATT&CK evaluation. Additionally, in line with the fact that top-performing vendors have built real-time threat intelligence into their platforms and managed services, CrowdStrike was able to internally identify the emulated nation-state adversary in less than 13 minutes.

For an MDR, AI-assisted threat intelligence is essential

Bringing together AI, ML, and human intelligence in an integrated MDR solution is the future of cybersecurity. Therefore, product lifecycles for cybersecurity platforms must be tightly integrated into MDR workflows. That way, valuable capabilities, such as native, first-party threat intelligence, become truly usable.

The evaluation showed how MDR solutions that can generate or create and then monitor threat intelligence succeed in identifying most events. CrowdStrike relies on Indicators of Compromise (IOCs) and other strategic insights integrated into their products show how threat intelligence can be scaled in an MDR solution. Identifying the nuances of MDR solutions and what enterprises should look for in a solution is why the MITER ATT&CK Security Service Provider Assessments are so valuable to organizations using these benchmarks as a guideline.