M michaelnielsen.org Crawling a quarter billion webpages in 40 hours (2012) 208 points swyx 3 years ago 63 comments