Comment by collateral0

If the google crawler actually respected robots.txt your point might be salient.