karimf
Joined 9 years ago
Joined 9 years ago
The last time I checked, open source full duplex model was still so far behind GPT-Live. So, I'm super excited about Dessa and I think it's very cool.
But I don't think it'll replace the classic cascade system for now for most use cases. A couple of reasons:
- Intelligence
- Tool calling
- System prompt?
- Language support other than English
Probably far fetched for now, but I think the next big evolution is building the pareto/much cheaper alternative to GPT-Live-1.
The STT/TTS market is quite saturated, while today, there's almost no cheap/open source alternative to GPT-Live-1.
When Astra launched, I think Artifical Analysis showed that it was on par with GPT-5.6 Sol and lower than Opus or something like that? Then, they updated the scoring.
I hope that more open source models, including this model, to be "as good to use" as Astra.
I'm on a totally different path for learning inference engineering. I self-host a voice AI app that has ~2000 monthly active users on my own GPU box.
This forces me to learn about production serving, KV cache, quantization, inference engine, observability and economics, prefill optimization since I'm optimizing for TTFT instead of decode speed, and many more.
It's fun since every optimization you do directly translate to a better user experience or allow you to serve more users using the same hardware.
Then I tried GPT 5.6 Sol. It's night and day.
I think Anthropic just RL too hard on coding capabilities and never calibrated or benchmarked the writing styles.
ggerganov and the team have done a stellar job maintaining the quality while still being fast to implement new models/improvements.
> We quantize weights to ~4-bit, bringing the LM under 20 GB. We validated minimal to no degradation on agentic tasks under compression.
https://www.reddit.com/r/LocalLLaMA/comments/1vkgsum/introdu...
Is it still worth building products or companies that can be one-shotted by AI? Probably not.
One interesting consequence is that this force everyone to be more ambitious and do something bigger that's impossible before.
I hope more people are working on something that can always bring net positive to humanity even if there are hundreds of people working on the same thing, like clean energy.