* most sites prohibit scraping but beg for Google to scrape them
* many good products/websites are based on scraped content
* many good products/websites are not feasible because of scraping limitations and limited access to data, even publicly funded data (e.g. no real estate ads with noise overlay despite the EU mandating noise maps in all member states; member states have prohibitive access rules for these maps)
* "rogue" scraping causes problems for many websites; blocking it creates problems for legitimate users of various proxy/anonymization services like Tor. Captchas are not a long-term solution, good programming defeats them.
I'd welcome a simple technical solution for scraping that takes into account the interests of site owners, other publishers, the public. The sooner publishers get together and build one, the better for them.
* many good products/websites are based on scraped content
* many good products/websites are not feasible because of scraping limitations and limited access to data, even publicly funded data (e.g. no real estate ads with noise overlay despite the EU mandating noise maps in all member states; member states have prohibitive access rules for these maps)
* "rogue" scraping causes problems for many websites; blocking it creates problems for legitimate users of various proxy/anonymization services like Tor. Captchas are not a long-term solution, good programming defeats them.
I'd welcome a simple technical solution for scraping that takes into account the interests of site owners, other publishers, the public. The sooner publishers get together and build one, the better for them.