One thing that is exciting about Jev-style decision models is that it makes it cheap for things where they can do zero-shot of things where it's not worth your time building a curated dataset.
Bonus: after using them for a while, you get the beginnings of a training set if you change your mind.
You can do that with LLMs too - it's just a cost and latency question.
E.g. I use an LLM to label my e-mail. I do it because it's low risk enough that I don't care about it being incredibly precise - it does not automatically act on anything, it just labels - and it means I don't have to curate a dataset of what e.g. should get the "receipts" label.
An LLM zero-shots it well enough and cheaply enough to remove my incentive to train a model for my individual e-mail.
If I wanted to scale it up, Jev or similar would probably be better.
A dedicated classifier would almost certainly provide even better precision if I invested the time to curate a dataset.
But the threshold where that is worth it for me given the precision I get from a cheap LLM is pretty high, and it's also less flexible.
E.g. one of the nice things about using a generic model for this is that whenever I add a new tag, it starts using it immediately and odds are good that at most I need to add a line to a file with one-line descriptions of what the folder name means to get decent results.
So there's a space for both LLMs/Jev-likes and custom classifiers.
Use Jev-like models (or LLMs) when the effort to curate a dataset to train a dedicated classifier is too high, and switch to a dedicated classifier when/if you need to scale something to a point where the economics justifies the effort or when you already have the dataset anyway (EDIT: and if you start with an LLM or Jev, collect the data - it will provide a good starting point for your training set if/when you decide to train a dedicated classifier)
Bonus: after using them for a while, you get the beginnings of a training set if you change your mind.
You can do that with LLMs too - it's just a cost and latency question.
E.g. I use an LLM to label my e-mail. I do it because it's low risk enough that I don't care about it being incredibly precise - it does not automatically act on anything, it just labels - and it means I don't have to curate a dataset of what e.g. should get the "receipts" label.
An LLM zero-shots it well enough and cheaply enough to remove my incentive to train a model for my individual e-mail.
If I wanted to scale it up, Jev or similar would probably be better.
A dedicated classifier would almost certainly provide even better precision if I invested the time to curate a dataset.
But the threshold where that is worth it for me given the precision I get from a cheap LLM is pretty high, and it's also less flexible.
E.g. one of the nice things about using a generic model for this is that whenever I add a new tag, it starts using it immediately and odds are good that at most I need to add a line to a file with one-line descriptions of what the folder name means to get decent results.
So there's a space for both LLMs/Jev-likes and custom classifiers.
Use Jev-like models (or LLMs) when the effort to curate a dataset to train a dedicated classifier is too high, and switch to a dedicated classifier when/if you need to scale something to a point where the economics justifies the effort or when you already have the dataset anyway (EDIT: and if you start with an LLM or Jev, collect the data - it will provide a good starting point for your training set if/when you decide to train a dedicated classifier)