Just landed: streaming ingestion on Cloudflare with Arroyo and Pipelines

Cloudflare announced the open beta of Pipelines, a managed, globally distributed service for high-throughput, real-time data ingestion that batches and stores events in R2 with partitioning and compression, alongside the acquisition of Arroyo to bring streaming SQL transformations and deeper integration across Workers, Queues, and the R2 Data Catalog. The post details architecture (Durable Objects, embedded SQLite, backpressure, retries), developer workflows (HTTP and Worker sources, configurable batching/auth/CORS), scalability targets (~100K rps), and a roadmap for transformations, additional sinks, Kafka sources, and forthcoming usage-based pricing with initial free quotas.