As a company of engineers, Praetorian firmly believes in data-driven decision making. In addition to abiding by this principle for our internal processes, we also strive to provide high quality and useful data for our clients so they can achieve the highest return on investment when improving their security. In our traditional penetration tests, which aligns with the “Identify” function in the NIST Cybersecurity Framework (CSF), we organize findings based on a risk rating calculation derived from five factors – access vector, attack feasibility, authentication, compromise impact, and business value. This enables engineers, architects, and developers to quickly understand and prioritize their vulnerabilities.
Project Release: Praetorian’s Metasploit Automation of MITRE ATT&CK™ TTPs →
Measure What Matters
When it came to improving our Purple Team service line, which maps to “Detect” and “Respond” in the NIST CSF, we wanted to provide a similar high quality of data and metrics to our clients. In our experience, it is hard to drive change in any organization unless those changes can be tied to measurable results. After conducting a survey of known frameworks, we settled on the ATT&CK™ framework from MITRE. To quote the website, “MITRE ATT&CK™ is a globally-accessible knowledge base of adversary tactics and techniques based on real-world observations. The ATT&CK knowledge base is used as a foundation for the development of specific threat models and methodologies in the private sector, in government, and in the cybersecurity product and service community.”
The goal of ATT&CK is to standardize the language that security experts are using when discussing attacker methodology. In doing so, the community can continue shifting away from signature-based detection and towards behavior-based methodology. In essence, ATT&CK is a matrix of attacker Tactics, Techniques, and Procedures (TTPs) organized into 11 Tactical columns with Techniques filling in the rows. By drilling into the Techniques, one can view what Procedures adversaries are using to accomplish their objectives. ATT&CK’s advantage is the open source and community-driven nature of the project – MITRE, as a federally funded non-profit, does not own the project but instead organizes development and contributions to the framework. Anyone interested in expanding the known corpus of adversary TTPs can add information to the framework (after it goes through a review process).
Praetorian isn’t the only one to have settled on ATT&CK as a foundational framework. A variety of security vendors, consultancy firms, and internal security teams have also found the value of ATT&CK. The recent conference sponsored by MITRE, ATT&CKcon, featured speakers from USAA, Endgame, Verizon, and Deloitte just to name a few. Not all companies have chosen to use ATT&CK for the same purposes, which shows the flexibility of the framework.
The atomic nature of ATT&CK is the fundamental reason Praetorian chose it as the underlying framework for conducting Purple Teams. It is atomic in the sense that each individual TTP can be viewed in isolation but also can form a new and complex structure when grouped together in a representative attack chain. Security teams can map detection capabilities and response actions to each atomic TTP that makes up an attack chain, as well as identify new patterns that arise from the grouping. Thus, when evaluating and improving a client’s detection and response, Praetorian can first evaluate detection at the atomic level (each TTP as a unique action), but also as a sequence (seeing how chains of TTPs produce different detection behaviors). When reporting and providing recommendations, mapped back to each TTP, promoting visibility of defensive gaps and facilitating trend analysis.
Solving the Cybersecurity Problem
At Praetorian, we believe in solving the cybersecurity problem. We envision a world where individuals, communities, businesses, and governments worry about cybersecurity as often as they worry about a natural disaster – not impossible, but rare with known risks and controls. Thus, while we love helping our clients, we also want to give back to the security community as a whole because we can’t solve this problem alone. Due to the aforementioned community-driven nature by which MITRE manages the framework, Praetorian has been able to submit some of the techniques we’ve used back into the matrix. We didn’t come up with all the techniques we’ve submitted, but we believe in making the information useful and available.
One technique, Kerberoasting (T1208), is used for collecting hashed credentials in Active Directory. Kerberoasting was being exploited as early as 2015, but it wasn’t contributed to the matrix until later releases. We at Praetorian were happy to have played a part in that contribution.
We’re always documenting and submitting more techniques to the framework. The matrix forms the base of our Purple Team exercises, but we don’t restrict ourselves to the matrix alone. We also test for malicious TTPs that we’ve seen in our past work, submitting these for inclusion into the matrix where appropriate.
Project Release: Praetorian’s Metasploit Automation of MITRE ATT&CK™ TTPs →
We open sourced our solution that implements MITRE ATT&CK™ TTPs as Metasploit Framework post modules. As of this release, we’ve automated a little over 100 TTPs as modules.
Our Approach to Purple Teams and ATT&CK
Praetorian’s approach to conducting a Purple Team exercise is broken up into four phases: 1) baselining, 2) technical fixes, 3) process improvement, and 4) a technical tabletop. The goal of baselining is to understand security technologies in use, as well as gaps in telemetry and analysis. When first integrating ATT&CK into your security policies, it can be helpful to begin with mapping your current security technologies to the framework. Many vendors, such as Carbon Black, provide ATT&CK TTP coverage information, but Praetorian recommends undergoing this exercise internally to better familiarize your security team with their own tech stack. First, begin by listing security tools currently in use. Then, the team can map those tools to the corresponding data sources and ATT&CK TTPs. A number of great open source solutions, such as this CSV from Casey Smith of Red Canary and Blue ATT&CK from Marcus Bakker and Ruben Bouman, can help facilitate the organization of this information. This exercise can be useful before beginning any Purple Team activities, as it can help the security team with prioritizing the categories and TTPs that they most want to test.
When Praetorian begins a Purple Team, we assist our clients with these activities if they haven’t already begun TTP coverage mapping. We’ve found that not only can this provide a focus to guide later Purple Team testing, but it can also facilitate executive decision making for acquiring new technologies. At one of Praetorian’s clients, going through this exercise helped identify a telemetry gap for Linux command line execution. Based on our recommendation, they deployed osquery across their fleet of servers and were able to greatly improve the quality of their data.
The second part of baselining is executing atomic tests that mirror the behavior of ATT&CK TTPs. Praetorian has automated testing for over one hundred of the TTPs utilizing the Metasploit Framework. Procedures for these TTPs have been sourced from numerous places: open source research, atomic tests from other organizations, and our own experience and tests. The engineer on the project will run through the TTPs we believe are either A. ones that should definitely be detected or B. provide the engineer information regarding the efficacy of various data sources. As an example, Credential Dumping is an attack that Praetorian believes every client should have the capacity to detect, so we have a number of ways of simulating that activity. On the other hand, PowerShell isn’t necessarily malicious on its own, but we want to establish that logging is properly configured and being fed into a system for malicious use analysis.
In the second phase, we shift from “find” to “fix”. Based on the deficiencies identified in the first phase, Praetorian will collaborate with the security team to develop rules, reduce noise, and improve the quality of alerts. In addition, Praetorian’s Defensive Enablement service line can assist with the task of installing and deploying industry-leading detection and response technology.
Praetorian’s focus in the third phase is improving the human. Detection and incident response (IR) is still an inherently human process, and Praetorian’s experienced IR and defensive experts can help with developing IR plans, assisting with event triaging procedures, and crafting threat hunting playbooks.
The capstone fourth phase combines the previous improvements into rigorous technical exercise. Working cooperatively with the client, Praetorian will execute chains of TTPs representing realistic scenarios that could be used against the environment. The goal of these chains is to ensure that alerts have sufficiently improved as well as to evaluate proper response procedures.
Penetration tests answer the question of “where are we vulnerable?” Praetorian’s Red Teams serve as a capstone exercise to the Corporate Security Journey. Purple Team exercises based off of MITRE ATT&CK are the critical bridge between the start and the end of the journey where your security team can understand current detection and response capabilities, map deficiencies, and execute targeted improvement actions. When spending precious resources improving your security program, ensure that you’re aligned to a framework in order to maximize your return on investment.