Cloudflare to Block AI Training Crawlers by Default Starting September 2026
Source: https://www.engadget.com/2207360/cloudflare-will-filter-out-web-crawlers-that-serve-ai-companies
Executive Briefing
- Announces automatic blocking of mixed-use crawlers that simultaneously index sites for search and train AI models
- Defaults shift September 15, 2026 for new customers and free-tier users, blocking AI training on ad-supported pages
- Relaunches Pay Per Use feature, paying site owners when their content appears in AI chatbot answers
- Targets Google indirectly, noting Googlebot bundles search indexing with AI training without offering publishers a clear opt-out
- Partners initially with Ceramic.AI and You.com, hoping broader AI industry adoption follows
Sponsored
