Comment by walls

A huge amount of the web is only crawlable with a googlebot user-agent and specific source IPs.