Phase 3 · Detection Engineering & Automation
Detection engineering that starts from the tool rather than the adversary produces alert volume, not security coverage. Starting from the adversary — from their specific documented techniques, their most likely course of action against your environment — produces detections that mean something. The adversary is not generic. Your detection program should not be either.
Most security teams build detections reactively — they deploy a SIEM, ingest what the vendor recommends, enable whatever detection rules come out of the box, and then measure success by alert count. The fundamental question — for the adversaries most likely to target our organization, what percentage of their documented techniques can we actually detect? — goes unasked. The answer, when it is eventually calculated, is almost always alarming.
Intelligence-led detection engineering inverts this. You start with adversary course of action templates from your threat landscape analysis. For each technique your primary adversaries are documented using, you ask: do we have the data to detect this? Do we have a detection built against it? Is that detection tested and confirmed to fire? Is it built against the behavior or against the artifact — and how high up MITRE's Pyramid of Pain does it operate? Those answers, aggregated across your adversary's TTP set, give you a defensible detection coverage map — not a false confidence score.
Summiting the Pyramid (MITRE Center for Threat-Informed Defense · December 2024): A formal methodology to score detection analytics against the Pyramid of Pain — evaluating the behavioral dependencies inside each rule and scoring them by how resistant they are to adversary evasion. A detection that fires when a specific hash is present scores at the bottom. A detection built against behavioral patterns that persist regardless of the adversary's tooling scores at the top. Summiting the Pyramid provides the methodology to move your detection program systematically up the pyramid — and to produce a defensible score that answers "how robust are our detections against adversary evasion?" with something more rigorous than an opinion.
The practical implication: most detection rules contain implicit dependencies on low-level artifacts — specific file names, specific registry keys, specific IP addresses — that make them trivially bypassable by an adversary who knows they are being hunted. The adversary does not need to change what they are doing. They change how they look while doing it. Behavioral detection — built against what the adversary must do, not what artifacts they happen to leave — survives that evasion. Building at the top of the pyramid is the goal. Summiting the Pyramid gives you the scoring methodology to get there systematically, at scale, with documentation that can be shown to leadership.
You have the tool. You do not have the telemetry. The log source is not configured, the agent is not deployed, the right fields are not being collected. Detection cannot be built without data. Answer: log strategy work comes first.
You have the data. You do not have the capability. Behind on tooling, relying on manual processes, missing a platform. Answer: capability investment — but specify exactly what the tool must do before procuring.
You have the tool and the data. No detection logic, playbook, or runbook. When the alert fires, analysts don't know what to do. Answer: detection engineering and documentation — not procurement.
You have everything. Still reactive. Waiting for alerts, validating annually, measuring metrics instead of acting on them. Answer: a program-level mindset shift toward proactive operations — not another tool.
Conflating these gap types is why security programs spend money in the wrong places. A data gap does not get solved by buying a new detection platform. A posture gap does not get solved by ingesting more log sources. Each gap type has a specific, different answer — and identifying which type you have before deciding on the solution is the difference between a defensible security investment and security theater.
The adversary is using automation at every phase of their operation: automated reconnaissance, automated social engineering at scale, automated evasion testing against your detection rules before execution. If your analysts are manually triaging every alert, manually enriching every indicator, manually executing every playbook — they are fighting an asymmetric battle they will eventually lose. Analyst fatigue is not a human resources problem. It is a security problem. When analysts are overwhelmed with volume, they miss the signal inside the noise. The adversary knows this and uses it deliberately.
Automation closes that gap: automated enrichment of alerts with context, automated triage of known-pattern indicators, automated playbook execution for repeatable response scenarios, automated case management that ensures nothing falls through. This frees analysts to focus on what genuinely requires judgment — the novel, the ambiguous, the high-stakes decisions that a rule cannot make. The goal is not to replace analysts. The goal is to ensure analysts are never wasted on work that a well-designed automation can handle faster and more reliably than a human can.
On SOAR and playbook automation: Playbooks mapped to ATT&CK techniques — when this technique fires, execute this response sequence — are the foundation. The first version of any playbook is always wrong in at least one place. The process of running playbooks, finding the gaps, and iterating is itself the detection engineering work. A playbook that has never been executed in a real scenario is a theory, not a capability.
Platform migrations are among the highest-risk periods in a security program's lifecycle. The detection logic that took years to build, tune, and validate needs to be translated to a new environment without losing coverage during the transition. A manual migration — reviewing each rule one by one, rebuilding playbooks by hand, reconfiguring data sources individually — takes years and leaves your detection posture degraded throughout.
A well-engineered migration framework moves detection logic systematically, maps data source schemas automatically, validates coverage before and after, and maintains operational continuity throughout the transition. The migration becomes an engineering project with measurable milestones and a coverage map at every stage — not an extended period of reduced visibility and degraded operations. When the off-the-shelf migration tools do not address the specific complexity of the environment, bespoke tooling fills that gap. The transparency and auditability of that tooling matters — when leadership asks what coverage was maintained during migration, the answer needs to be specific and verifiable.
There are capabilities that no commercial product addresses with sufficient precision for every environment. Field-level log analysis at scale across a complex, heterogeneous infrastructure. Coverage scoring against a specific adversary's documented TTP set with a defensible methodology. Automated gap identification that maps current telemetry to specific MITRE ATT&CK data sources and produces a prioritized remediation roadmap. These capabilities can be built when the commercial alternatives are insufficient — but they must be built with the same rigor applied to any security tool: transparent methodology, auditable logic, reproducible results. When a CISO asks why the coverage score is what it is, the answer needs to be explainable. A tool that produces an output nobody can defend is not a security tool — it is a liability.
The automation imperative: The adversary does not sleep. They do not take weekends. They do not get alert fatigue. A security program that depends on human analysts to do what automation can do is structurally disadvantaged against an adversary who has already automated their reconnaissance, their social engineering, their evasion testing, and their lateral movement. Automation is not about replacing analysts. It is about ensuring analysts are never wasted on work that should never have reached them.