I watched the Google interview with Ilya yesterday and this came up. There's a large disconnect between the evals and the real-world performance, and he admitted that the evals are targeted.
There was a storm of hype the last couple weeks for Gemini 3 and everyone, correctly, rolled their eyes. Investors are demanding a return and it's not happening. They're just going to have to face reality at some point.
There was a storm of hype the last couple weeks for Gemini 3 and everyone, correctly, rolled their eyes. Investors are demanding a return and it's not happening. They're just going to have to face reality at some point.