Comment Comment by lxgr > How will Internet Archive operate?Presumably increasingly less and less effectively, at least if they continue honoring robots.txt and don't implement scraping protection bypass mechanisms.https://www.theverge.com/news/757538/reddit-internet-archive... parent lxgr 1 year ago
Presumably increasingly less and less effectively, at least if they continue honoring robots.txt and don't implement scraping protection bypass mechanisms.
https://www.theverge.com/news/757538/reddit-internet-archive...