Cloudflare changes AI crawler access rules
ID: fa7c3564-28d5-5fc2-a13d-196aa83f8cfe
STIX ID: report--fa7c3564-28d5-5fc2-a13d-196aa83f8cfe
Feed Name: Help Net Security
Cloudflare announced enhanced AI crawler controls and a new bot taxonomy that classifies crawlers as Search, Agent, or Training, with new default behaviors effective September 15, 2026 (Training and Agent blocked by default on ad pages). The update includes BotBase for verified-bot visibility, content-use levels (Immediate, Reference, Full) exposed via an extended Content Signals format, and a proposed transitive trust model using the HTTP Forwarded header to communicate original requesters and declared content use.
Your team is not currently subscribed to this feed. You must subscribe to it in order to see this post.