Comment

Comment by dawidpotocki

That doesn't match my experience. For example, I recently had Meta scrape robots.txt disallowed paths. On some page they claimed they respect it but they don't. And yes, all the IPs they used to connect to me (they used a different /64 network for each connection) were owned by Meta, it was not someone pretending to be them.

Some other reports of this: https://github.com/TecharoHQ/anubis/issues/1565

I ended up just fully closing connections with no response from these assholes on any URL.

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton