Your detections need validation too!

Adversary emulation and simulation are capabilities that defenders should consider as they hone their threat hunting and investigation skills. In fact, these capabilities can be extended into detection engineering to provide security operations teams with the ability to craft detection logic that can move from the indicators of compromise to adversary tactics and techniques.

However, once we’ve built this logic, how do we ensure our detections are still valid next year or even next month? After all, our environments aren’t static. And just to make this more challenging, many organizations have test environments that may not reflect their production environments, so in some cases, testing of detections need to be performed in production or else they won’t be tested.

This presentation will focus on highlighting the challenges security operations teams face as they attempt to proactively test and validate their detections as well as ideas how these obstacles can be overcome. To make meaningful progress in the field of detection engineering, testing and validation needs to occur due to evolving security controls, infrastructure and adversaries.

To highlight these efforts, we are leveraging the popular and open source Atomic Red Team library of tests to build a testing infrastructure to generate events that facilitate the development of detections. The presentation will discussing bridging the gap from building those initial rules using frameworks like Atomic Red Team to the practical application of continual testing and validation, and will include additional custom scripts, testing methodologies and ideas how this kind of testing can be performed in a production environment without causing disruptions to the current security operation team’s cadence and reporting. Additionally, we will look at moving beyond atomic tests into sequences of actions.