These models do pretty well in benchmarks and real world so I'm highly suspicious of this article. Further more, in the original report, the examples of bad answers are from Haiku - at least 7 out of 10. Anyone who knows anything about LLMs know that haiku shouldn't be used for anything pretty much.
There's no reproducible set either. I'm not gonna trust this report.
There's no reproducible set either. I'm not gonna trust this report.