testycool
Joined 1 year ago
Joined 1 year ago
This seemed relevant because it has budget caps. It also has per-user/model controls and usage attribution for teams/enterprises.
I've been exploring it for the past few days with their community edition. They also have a paid variant which uses OSS models from their infrastructure.
Woke up at 6am on a Sunday.
I think I'm becoming a thought leader.<sidenote>
Similarly, HuggingFace has a CLI + a few skills, and they are very useful.
I had a production image processing using Gemini 2.5 Flash Lite (which is getting discontinued in October), and in 20 minutes Claude Code + HF Cli recommended the best replacement small model (Qwen VL 3B something) and proceeded to fine tune it on my datataset. All this while I was in a rush to get dressed and go to the store.
It cost ~$3 I think, and results were excellent. Not perfect, but not far from perfect either.
We didn't replace Gemini in prod at the time, because we didn't have time to do all the math on how to end up with a smaller bill/mo.
</sidenote>
Initially it feels like the result will be too empty, but once the greebling is removed it most often looks better
Bifrost feels very light, though.
Will try LiteLLM if it's in Rust.
Would also appreciate a slicker aesthetic. I got used to Bifrost shadcn with 0 border radius.