Comment

Comment by Jskewel

The big USA ones do, and it would be madness for them to do otherwise.

But that's meaningless because 99% of AI crawlers are "bad bots" which ignore robots.txt and use domestic IPs to circumvent blocks.

parent Jskewel 26 days ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton