The chart that I pay attention to is Intelligence Index vs. Output Tokens per Task[0]--it penalizes models running at 'max' thinking with lower intelligence and more churn to get to the answer.
The proprietary models 'cheat' on this since they generate but don't send output thinking tokens to be counted. So to grade them we have to use cost or time.
The proprietary models 'cheat' on this since they generate but don't send output thinking tokens to be counted. So to grade them we have to use cost or time.
[0] https://artificialanalysis.ai/?intelligence-efficiency=outpu...