A few take-aways from a study we ran (~800 consumer queries, repeated over a few days):
* AI answers shift a lot. In classic search a page-1 spot can linger for weeks; in our runs, the AI result set often changed overnight.
* Google’s new “AI Mode” and ChatGPT gave the same top recommendation only ~47 % of the time on identical queries.
* ChatGPT isn’t even consistent with itself. Results differ sharply depending on whether it falls back to live retrieval or sticks to its training data.
* When it does retrieve, ChatGPT leans heavily on publications it has relationships with (NYPost and People.com for product recs) instead of sites like rtings.com
* AI answers shift a lot. In classic search a page-1 spot can linger for weeks; in our runs, the AI result set often changed overnight.
* Google’s new “AI Mode” and ChatGPT gave the same top recommendation only ~47 % of the time on identical queries.
* ChatGPT isn’t even consistent with itself. Results differ sharply depending on whether it falls back to live retrieval or sticks to its training data.
* When it does retrieve, ChatGPT leans heavily on publications it has relationships with (NYPost and People.com for product recs) instead of sites like rtings.com
Writeup: https://amplifying.ai/blog/why-ai-product-recommendations-ke...
Data: https://amplifying.ai/research/consumer-products