Comment by jdxcode

i was going to say the biggest reason wasn't mentioned here, that github sets Disallow: /*/wiki*

however I think maybe this has changed? I don't see it in https://github.com/robots.txt now