In our data analytics benchmark, it got the best score of all of the Chinese models and tied Opus 5.5 & Argon (41/43 questions correct). It's also 1/3rd the price of both of those models. However, it seems to use A LOT of tokens and it's the slowest model that we've tested for the exam; it's slow because of the number of tokens it uses, not the inference speed.
We also tested Space Bunny (its stealth version) a few weeks ago and the behavior changed quite a bit. Went from 30s / question to 3 minutes / question and the accuracy improved quite a bit.[1]
This benchmark is short-horizon data analytics questions, so not fully representative of software engineering tasks. YMMV.
We also tested Space Bunny (its stealth version) a few weeks ago and the behavior changed quite a bit. Went from 30s / question to 3 minutes / question and the accuracy improved quite a bit.[1]
This benchmark is short-horizon data analytics questions, so not fully representative of software engineering tasks. YMMV.
[1] https://plotly.com/blog/minimax-m3-1-flash-plotly-data-analy...