Comment Comment by Reubend Because the website doesn't seem to show any sample size of runs, I assume they ran it once across the suite.The models are nondeterministic, and therefore it's pretty normal for different runs to give different results.I don't see this as evidence that Opus 4.6 has gotten worse. parent Reubend 6 months ago
The models are nondeterministic, and therefore it's pretty normal for different runs to give different results.
I don't see this as evidence that Opus 4.6 has gotten worse.