Shrinking the Stack of Needles

The article argues that security teams should shrink the “stack of needles” by aligning data collection to actual detection use cases, rather than ingesting all logs into real-time analytics. It highlights explosive telemetry growth, critiques the “log lemming effect” of collecting everything, and recommends using data pipelines and streaming detection to keep only detection-relevant data in high-performance stores while routing the rest to cheaper storage—improving speed, cost, and analyst effectiveness.