Gemini team has a long history of releasing great benchmark charts and disappearing into the ether. There has never been a moment where I have preferred them to Claude/Codex. When I get access I eval them on my home assistant set, a small coder benchmark, etc. and they are too expensive, and too low quality.
The benefit for Google is their chat interface has privileged access to their data (maps, etc.) but their model itself is low quality and, if I'm being honest, has the promotion-oriented measurable-output bias that FAANGs tend to have. The later Llamas had the same problem.
The benefit for Google is their chat interface has privileged access to their data (maps, etc.) but their model itself is low quality and, if I'm being honest, has the promotion-oriented measurable-output bias that FAANGs tend to have. The later Llamas had the same problem.