Comment Comment by zx76 My understanding is that the current default allows AI training bots - but this actually going to change in 2 months time.https://developers.cloudflare.com/changelog/post/2026-07-01-...From Sept 15 all new sites added to CF will even block Googlebot by default on any page that serves ads as I understand it.I think it's CF trying to force Google to separate out their bot traffic into bots for training and bots for the search index.I think CF sees a big opportunity to get businesses to pay them to allow certain uses of their data but block others.They're also starting a registry of "Approved" crawlers. parent zx76 3 months ago
https://developers.cloudflare.com/changelog/post/2026-07-01-...
From Sept 15 all new sites added to CF will even block Googlebot by default on any page that serves ads as I understand it.
I think it's CF trying to force Google to separate out their bot traffic into bots for training and bots for the search index.
I think CF sees a big opportunity to get businesses to pay them to allow certain uses of their data but block others.
They're also starting a registry of "Approved" crawlers.