Tell HN: Amazonbot aggressively scraping my website and ignoring robots.txt
At the beginning of the year I decided to set up a scraping and LLM honeypot on one of my personal websites which included a fake git repo with code containing fake HTTP endpoints. The address to this repo was hidden in a public page inside a comment.
About three weeks ago IP addresses from Amazon Searchbot attempted to make requests to the fake endpoints included inside a shell script.
My robots.txt explicitly includes Amazonbot.
I am honestly surprised that this is coming from Amazon. Is this kind of behavior legal?
About three weeks ago IP addresses from Amazon Searchbot attempted to make requests to the fake endpoints included inside a shell script.
My robots.txt explicitly includes Amazonbot.
I am honestly surprised that this is coming from Amazon. Is this kind of behavior legal?