Comment

Comment by pverheggen

There's been an explosion of vibe-coded scrapers that behave poorly and ignore robots.txt. Presumably OP disallowed crawling of the search endpoint for the reason stated (that crawling it would result in endless permutations of search filters.)
parent pverheggen 3 months ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton