Cloudflare Bot Preference Sync dashboard showing AI crawler toggles updating a robots.txt file
Web Design

Cloudflare Bot Preference Sync: Matching Robots.txt with AI Bot Rules

/

Cloudflare released Bot Preference Sync to automatically align robots.txt files with edge AI bot policies across Search, Agent, and Training crawlers.