Making Super Slurper 5x faster with Workers, Durable Objects, and Queues

Cloudflare describes a major rearchitecture of its Super Slurper tool for migrating large datasets to R2, moving from a Kubernetes/Postgres design to a serverless model using Workers, Queues, SQLite-backed Durable Objects, and Hyperdrive for legacy data access. The new pipeline scales concurrency automatically, improves reliability with retries and DLQs, ensures idempotent transfers via locks and existence checks, supports multipart uploads, and shards state to handle billions of objects within storage limits. These changes deliver up to 5x faster migrations (e.g., 75k S3 objects in 3m25s) and pave the way for higher per-account concurrency and broader S3-compatible migrations.