Comment Comment by erikwiffin SlimeBallBench was not me, but it is cool! and at least partially what inspired me to build my own.Mine is less fun, but hopefully more useful https://erikwiffin.github.io/memory-reasoning-eval/Selling to labs is more than I'm looking for. I'm aiming for a couple hundred dollars so I don't have to finance a Fable vs Sol run out of my own pocket. It would be cool to have my benchmark be one of the ones referenced in a model card! parent erikwiffin 2 months ago
Mine is less fun, but hopefully more useful https://erikwiffin.github.io/memory-reasoning-eval/
Selling to labs is more than I'm looking for. I'm aiming for a couple hundred dollars so I don't have to finance a Fable vs Sol run out of my own pocket. It would be cool to have my benchmark be one of the ones referenced in a model card!