Your site, your rules: new AI traffic options for all customers

Cloudflare announces a new bot classification and management framework for AI-related automated traffic—defining Search, Agent, and Training use cases—along with BotBase (a bot directory for Enterprise), content-use signals (use=immediate|reference|full) in robots.txt, updated Verified bot semantics, and defaults to block Training and Agent crawlers on ad pages starting September 15, 2026; it also proposes transitive trust via the Forwarded header to convey operator identity and content-use intent.