How Huntress Transformed Its Detection Engine

Huntress outlines its shift from a Kibana Security app-driven, batch-oriented detection approach to a custom streaming detection engine paired with AWS OpenSearch to overcome scaling, latency, redundancy, and cost constraints arising from massive data volumes (about 30TB/day). The new architecture splits document flow to an event stream and OpenSearch, preserves an at-least-once guarantee via queuing, eliminates rule scheduling and results caps, reduces inter-AZ transfer and infrastructure costs through AWS flexibility, and retains familiar search capabilities for SOC analysts.