nikshepsvn
Joined 3 years ago
Joined 3 years ago
thimble is a model i spent the last week of my evenings trying to train in order to beat needle 2 -- a 45M param tool-calling model trained on 153+ billion tokens by a funded team (u/cactuscompute)
my model: 48M params, ~1 billion tokens, $260 total (so 150x less training data)
the model wins 3 of their 5 published benchmarks, including seal-tools, the headline suite of their eval and the hardest one to move (so not a complete victory but still incredibly impressive results if i have to say so myself)
weights, code and results from every failed experiment is public -- and tbh all the fuck ups were the best part of the whole process, learnt a lot about synth data, model arch and training along the way
code: https://github.com/nikshepsvn/thimble weights: https://huggingface.co/flashvenom/thimble
i built this using openrouter data thought this might be useful for myself and other folk
collection of projects and writings
i love looking up new models to swap to and improve speed, reduce costs etc but its always a pain
this makes it so much easier
any attention is better than no attention, so here is my proposal to the void (: