Comment by cgannett

Robots.txt became 0% effective eventually. But this? With the way LLMs work? You never know.