CROON_tv

What I'd want to see next to accuracy is tail latency. In a real-time use, deciding when a spoken sentence is finished, a general LLM with the same prompt was slower and more hesitant for us than Jev, even though both cost about the same.

Similar result in production for me. I compared Jev with a general LLM (DeepSeek) as the decider for when to cut live speech into sentences, on the same 226 segments. With Jev the longest held segment was 17 s. With DeepSeek it was 40 s.

I use Jev in an iPhone app that dubs whatever the phone plays in real time. It decides whether a translated sentence is complete enough to speak now or should wait.

It's fast and cheap: about 0.35 s per decision, about 2 cents per hour of audio.

Thanks! Translation is done with ASR, an LLM and TTS.

The delay is about 4 seconds on average. It could be faster, but translating half a sentence gets confusing, so it waits for the sentence to end.

The LLM handles idioms.