Comment by zuzululu

i discount people who lean too heavily into benchmark as the authoritative truth when it comes to evaluation of coding capability of these models.

experience tells me that those people simply have not used models for a long period of time specifically on coding and have run their own comparisons

to someone who uses all vendors, the differences are very palpable and drives purchase decisions.

also keep in mind Gemini and other labs have repeatedly done benchmaxxing, you must have your own benchmarks to evaluate these models.