A year or two ago I personally encountered scraping bots that were scraping every possible resultant page from a given starting point. So if it scraped a search results page it would also scrape every single distinct combination of facets on that search (including nonsensical combinations e.g. products that match the filter "products where weight<2lbs AND weight>2lbs")
We ended up having to block entire ASNs and several subnets (lots from Facebook IPs, interestingly)
We ended up having to block entire ASNs and several subnets (lots from Facebook IPs, interestingly)