Comment by xiamx

What's wrong with scraping a bunch of pages. As long as they are following robots.txt, it's no big deal.