This is the most grounded and coherent take I’ve seen on the actual realizable value of LLMs.. pretty much since they came out.
> the classes of firms that can accept the use of fully autonomous LLMs are few, by my count just three:
1. those who can accept failure cheaply: firms that would otherwise hire interns, firms involved in rapid prototyping work, etc.
2. those who need done a small set of narrowly defined tasks with existing clear guardrails: repetitive physical labor in a controlled environment, call center and customer service chat work, etc.
3. those that can accept or already do by nature the costs of rigorous specification and validation: chip design, drug discovery, and other domains where failure on deployment is an existential concern.
the first two classes are price sensitive and arguably don't need the jump in reasoning quality you see going from cheap to frontier models. most of these firms will be best served by open models running on cheap hardware, perhaps even locally at the site of use. for the first and third classes, the type of fuzzy combinatorial search that has produced headline results in mathematics and security research seems more sensitive to agentic swarm width than reasoning capacity
…
This is just so on point. And for the third class (which I would extend to things like materials research as well), specification and validation are already by FAR the larger costs, so automating search and simulation is really not a massive game changer for the broader business.
> the classes of firms that can accept the use of fully autonomous LLMs are few, by my count just three:
1. those who can accept failure cheaply: firms that would otherwise hire interns, firms involved in rapid prototyping work, etc.
2. those who need done a small set of narrowly defined tasks with existing clear guardrails: repetitive physical labor in a controlled environment, call center and customer service chat work, etc.
3. those that can accept or already do by nature the costs of rigorous specification and validation: chip design, drug discovery, and other domains where failure on deployment is an existential concern.
the first two classes are price sensitive and arguably don't need the jump in reasoning quality you see going from cheap to frontier models. most of these firms will be best served by open models running on cheap hardware, perhaps even locally at the site of use. for the first and third classes, the type of fuzzy combinatorial search that has produced headline results in mathematics and security research seems more sensitive to agentic swarm width than reasoning capacity
…
This is just so on point. And for the third class (which I would extend to things like materials research as well), specification and validation are already by FAR the larger costs, so automating search and simulation is really not a massive game changer for the broader business.