Billions and billions (of logs): scaling AI Gateway with the Cloudflare Developer Platform

Cloudflare outlines how AI Gateway evolved from short-lived log retention to scalable, persistent storage for billions of logs by moving request bodies from D1 to R2, adopting Durable Objects with SQLite, and sharding by account and gateway, alongside an Account Manager Durable Object for quotas and entitlements. The post also previews future scaling via sharded Durable Objects and an enhanced Universal Endpoint that integrates automatic retries, provider fallback, and content-aware decisioning to improve reliability and support human-in-the-loop and LLM-based evaluations.