Comment by sumtechguy

Most of the time it isnt mine either. But here and there there will be large swaths of some site that are just not archived. At one point if someone put up a robots text file history would just vanish. Have not checked lately if that still holds. But one company I worked for was just 'gone'. Because a webpage squatter put up a robots.txt.