I build anubis-oss because my friends and I are always trying to see how local LLMs run on our different Macs with different configs. So I build a benchmarker for us with a bunch of tools like exportable benchmark reports and arena mode, then I said hey, I can add public leaderboards with the full dataset of submissions open sourced, and give that to my ML and model tuning friends to gauge performance across broad configs. Yes it cost me money, but it's a fun project and I'm trying to get it into homebrew. Just need a few more stars.
https://devpadapp.com/anubis-oss.html
https://github.com/uncSoft/anubis-oss
https://imgur.com/a/X64WsWY