> Does Mastodon have something to the effect of robots.txt
Mastodon has a "Opt-out of search engine indexing: Affects your public profile and post pages" setting, which probably inserts a "robots" meta tag. [1]
But this is just for ordinary web crawling, and the bit isn't sent along with your posts that get federated to other instances.
Mastodon has a "Opt-out of search engine indexing: Affects your public profile and post pages" setting, which probably inserts a "robots" meta tag. [1]
But this is just for ordinary web crawling, and the bit isn't sent along with your posts that get federated to other instances.
[1] https://developers.google.com/search/docs/crawling-indexing/...