Comment by maitland

I'd be amazed if ai companies like openai respect robots.txt or any other good faith mechanism.