Comment by alphan0n

:/ Common Crawl archives robots.txt and indicates that the file at wiki.diasporafoundation.org was unchanged in November and December from what it is now. Unchanged from September, in fact.

https://pastebin.com/VSHMTThJ

https://index.commoncrawl.org/