If you can get a specialized agent to work in its domain at 10% parameters of a foundation model, you can feasibly run locally, which opens up e.g. offline use cases.
Personally I’d absolutely buy an LLM in a box which I could connect to my home assistant via usb.
Personally I’d absolutely buy an LLM in a box which I could connect to my home assistant via usb.