Building Agents Backwards from Evaluation

## Executive Summary This article argues that LLM-based investigation agents for SOCs must be designed with measurable evaluation from the start, combining end-to-end metrics (burndown, disposition accuracy) with component-level competency tests (e.g., SIEM query construction, hypothesis generation). It recommends prioritizing high-impact, decidable competencies, leveraging domain experts to define ground truth, continuously adding real failure cases to evaluation suites, and validating agents in parallel with human analysts before production.