Comment by dpoloncsak

The crawlers would have still just hammered their site though, right?