What we learned by integrating with Google Cloud Platform

Red Canary details the design and launch of its Google Cloud Platform integration for real-time control-plane telemetry, addressing the challenge of consuming billions of tiny audit log messages by adopting Amazon Data Firehose for scalable, cost-effective batching and partitioning into its detection pipeline. The team evaluated and rejected alternatives like Kafka, large-scale serverless, and Kubernetes-heavy architectures to minimize operational overhead, ensure reliability, and simplify customer onboarding. Rolled out in early 2024, the integration now ingests roughly 200 million GCP telemetry records per day and has been extended elsewhere in the platform to support detecting and stopping adversary activity.