Comment

Comment by tenplusfive

This is actually the correct way. Robots.txt only stops google from crawling your website, but it might still appear in the index. If you search for the site, then it is shown in the search results, but without any of the content of the page. If you want to prevent indexing as well, you'll have to do this.

Also explained here: https://developers.google.com/search/docs/crawling-indexing/...

parent tenplusfive 3 years ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton