> The pull requests are public, and the prompts, raw model outputs, judge verdicts, bug-class labels, repeat runs and scoring scripts are committed with this article.
Okay, but where are they? This quote says that PRs are public. Does that mean that everything else is private and we can’t actually reproduce those results?
This, and the fact that the article doesn't mention reasoning levels, sounds fishy to me.
Okay, but where are they? This quote says that PRs are public. Does that mean that everything else is private and we can’t actually reproduce those results?
This, and the fact that the article doesn't mention reasoning levels, sounds fishy to me.