M michaelnielsen.org How to crawl a quarter billion webpages in 40 hours (2012) 136 points _ao789 10 years ago 23 comments