This is a longstanding UX issue with unlisted URL’s. I’ve encountered this in my own products. Folks don’t know that sharing a link means it definitely will get indexed by Google.
Unfortunately this can’t be fixed via robots.txt — Google won’t proactively crawl the URL, but it does still index pages it finds from external links.
A `noindex` meta tag, however, does prevent pages from being indexed, so that feels like the appropriate solution here.
Unfortunately this can’t be fixed via robots.txt — Google won’t proactively crawl the URL, but it does still index pages it finds from external links.
A `noindex` meta tag, however, does prevent pages from being indexed, so that feels like the appropriate solution here.