t5-notdiamond
Joined 2 years ago
Karma
6
Submissions
16
Member since
Jun 11
Recent submissions
Comment · 2 years ago
In offline training of our router, we run extensive cross-domain evaluations to determine when a smaller model can handle a request without any quality loss relative to more powerful models. In an online setting like our chat app, there's probably some more rigorous post-hoc analysis we could do on response quality—could make for a good follow-up post.
Comment · 2 years ago
Hey hn, this is Tomás, founder of Not Diamond. Today we're open-sourcing RoRF (Routing on Random Forests), a pairwise AI model router that beats all closed and open-source approaches, along with 12 pre-trained model routers:
Hugging Face: https://lnkd.in/gE4FKZWR Github: https://lnkd.in/gn2uKWaY Blog: https://lnkd.in/gutaamm4
When routing between two strong models, RoRF achieves higher accuracy than either individual model at significant cost reductions. When routing between a strong and a weak model pair, we outperform other routing approaches at a lower cost.
The architecture for RoRF is based on a Random Forest Classifier and supports both open-source embedding models like Jina which you can run locally, or closed-source embedding models like Voyage or OpenAI for lower compute requirements.
We developed RoRF earlier this year as a previous iteration of the more powerful router we released through our API. Excited to share this :)
Comment · 2 years ago
Thanks Sami :) Has been awesome to be able to share our progress with you
Comment · 2 years ago
Glad to hear that
Comment · 2 years ago
Awesome, lmk what you think
Comment · 2 years ago
Thank you—really glad we could work together on this
Comment · 2 years ago
Not yet—very much on the roadmap though
Comment · 2 years ago
Hey, thanks for checking it out! You can find some more info on how Not Diamond works here: https://notdiamond.readme.io/docs/how-not-diamond-works