Comment Comment by ycombinete Archive.org mostly ignores robots.txthttps://blog.archive.org/2017/04/17/robots-txt-meant-for-sea... parent ycombinete 5 months ago
https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...