Comment Comment by Kwpolska Why would AI need a new standard for excluding it? Just add a "Googlebot-AI" user agent to your list [0] and respect these rules when crawling content for use in AIs, and convince OpenAI and Bing to do the same.[0] https://developers.google.com/search/docs/crawling-indexing/... parent Kwpolska 3 years ago
[0] https://developers.google.com/search/docs/crawling-indexing/...