Comment Comment by jonmagic I've been following jevbench twice a day for the past week and that's been a lot of fun. Latest update:Rank System Score Public / sealed accuracy Evidence1 decider-4b v2 64.13 83.5% / 34.7% Evaluator-run, offline2 Jev 1.13 63.29 86.6% / 36.7% Evaluator-run API3 JevK5 v0.2 62.04 85.3% / 33.1% Evaluator-run4 Cygnet 12B 61.76 87.9% / 33.8% Evaluator-run, offline5 Hopper 59.43 82.3% / 34.1% Evaluator-run28 Kev 4B 36.14 66.2% / 22.4% Evaluator-run41 Laya 421M 30.25 58.4% / 30.8% Evaluator-runhttps://benchmarkheaven.com/jev-models parent jonmagic 1 month ago
Rank System Score Public / sealed accuracy Evidence
1 decider-4b v2 64.13 83.5% / 34.7% Evaluator-run, offline
2 Jev 1.13 63.29 86.6% / 36.7% Evaluator-run API
3 JevK5 v0.2 62.04 85.3% / 33.1% Evaluator-run
4 Cygnet 12B 61.76 87.9% / 33.8% Evaluator-run, offline
5 Hopper 59.43 82.3% / 34.1% Evaluator-run
28 Kev 4B 36.14 66.2% / 22.4% Evaluator-run
41 Laya 421M 30.25 58.4% / 30.8% Evaluator-run
https://benchmarkheaven.com/jev-models