claudiusa

Are the published numbers single-run or averaged, and which model does the judging? With LLM-as-judge scoring I would expect a couple of points of run-to-run noise, which does not matter for the top spot but matters a lot for the middle of the table.

lol!

(E) + (A) is the spicy combo: most of these talks are the only public record of their technique, and within a week they're all in every frontier LLM's training set. Great or terrifying depending on whether you sit red or blue. Anyway great job!

Interesting! add referee agents as well. That can be bribed with tokens from each team... stupid idea, i know

worked for me pushing some commits about 20 minutes ago.

Interetsting idea. does this work with digitally signed PDFs? converting them to word or ?