rafaelmn
Joined 3 years ago
Joined 3 years ago
The only AI that makes money is enterprise. Being open weights/self-hostable is a huge win before the pricing even enters the conversation, but pricing is also adding a ceiling on what US labs get to charge. And stuff like powering other software also works better with fine-tunes, hence legal models, etc.
And this is the pattern I've seen on most (semi)successful projects I've worked on in the past - I'd say correlation between financial success and code quality is 0 (up to a point where the whole thing doesn't fall apart). Once scale (both in load and in code size/features) starts mattering you're stuck building on a foundation of shit. LLMs are very good at identifying and cleaning up said shit layers, as long as you're steering them towards a desireable outcome.
Do you even see anything underneath that rose tint ?
High level languages :
- compress low level patterns into high level abstractions which save context/token
- express higher level meaning through things like names
- express and enforce constraints through type system
LLMs are context constrained, do really well with tools that help them iterate towards a correct solution (type systems) and work on human language (although they are compressing it to be more token efficient).Why would you take something compilers are really good at (translating high level concepts into low level hardware instructions) and bake that into LLMs
If anything LLM programming language will be something like high level IR that's token optimized, but honestly the AI labs are so lazy (or bad) at fundamental engineering (look at their sandboxing solutions LOL), they are going to keep pumping training on shell/python/rust RL environments.