Partially so. But the fact that we're comparing Jev to SOA LLM and pre-trained classification means it's essentially satisfied its reason for existence.
The benefits of Jev, and the reason I'm excited about them:
* Pre-trained/tuned to provide only structured output
* Small, fast, essentially trivial to locally run - this alone makes them an actual candidate for real-world planning systems. Laya is a local-first 400 million parameter model, for example.
* Their confidence scores are useful, even if those scores are not correct/true probabilities.
For me and my work, they look like nearly turn-key, tiny, fast, locally-hostable models that can manage state transitions in a deeply autonomous system.
LLM-as judge requires, well, what we know as a full LLM. A fully trained classifier model requires fully training - something you cannot / won't do for a embedded autonomous planning system (by contradiction - if this worked, we'd have used it everywhere already!). Totally unfair comparison in my mind.
The benefits of Jev, and the reason I'm excited about them:
* Pre-trained/tuned to provide only structured output
* Small, fast, essentially trivial to locally run - this alone makes them an actual candidate for real-world planning systems. Laya is a local-first 400 million parameter model, for example.
* Their confidence scores are useful, even if those scores are not correct/true probabilities.
For me and my work, they look like nearly turn-key, tiny, fast, locally-hostable models that can manage state transitions in a deeply autonomous system.
LLM-as judge requires, well, what we know as a full LLM. A fully trained classifier model requires fully training - something you cannot / won't do for a embedded autonomous planning system (by contradiction - if this worked, we'd have used it everywhere already!). Totally unfair comparison in my mind.