To me, the simple models might not cross the boundary where LLMs start to be useful versus, say, a fixed menu with choices in a helpdesk app.
It's a paradox because, to really feel human like and not make huge mistakes, we need these huge LLMs and they are expensive... and the alternative is not-so-smart traditional code.
So what I'm trying to say is that I think the small LLMs might not be that useful before they cross some arbitrary quality threshold (which they may never do.. considering more parameters => better model, in general).
It's a paradox because, to really feel human like and not make huge mistakes, we need these huge LLMs and they are expensive... and the alternative is not-so-smart traditional code.
So what I'm trying to say is that I think the small LLMs might not be that useful before they cross some arbitrary quality threshold (which they may never do.. considering more parameters => better model, in general).