Comment Comment by cmeacham98 Correct. Example snippet from the nytimes.com robots.txt: User-agent: archive.org_bot Disallow: / parent cmeacham98 5 months ago
