vkgcr

Sure but that means accepting the cost of having faulty modules running in production (and there is no guarantee that your LLM could fix a faulty module, perhaps it will just replace it with a module with different bugs. There's no way to know!)

This obviously won't fly for "serious" software such as airplane software, but I'm willing to guess that even customers of a low-stakes SaaS would get pissed off if your product regularly turns up new bugs.

I think it's easier to just do code reviews...

I don't think there will ever be a way to "steer" an LLM such that it produces the correct output 100% of the time. You can certainly context engineer your way to better outputs, but you can never reduce to 0 the possibility of the LLM making a mistake, due to inherent limitations of the LLM (perhaps what you ask of it wasn't RL'd during its training), but especially due to the fact that it will often misunderstand requirements and make unjustified assumptions.

So, no, you will never be able to produce code automatically. You can and certainly should use AI as a code assistant to write code faster, but the human should always 1) make all important architectural decisions and 2) carefully review the outputted code

The human bottleneck is necessary.

Product quality is an emergent property of code quality.

So you're essentially willingly becoming 100% dependent on AI, while praying that hopefully they'll become good enough to save you from the slopocalypse down the line.

Doesn't seem like a good strategy to me.

If you accept that you can one-shot any game, it means that you can one-shot anything. So essentially AGI.

To answer your question : I don't know, but probably not in the foreseeable future.

This sounds like a pretty dumb explanation due to the obvious fact that you're not forced to only read books which came out in the current year...

Do you also believe the same for other categories of products? For example, for a car, or for a building? Would you board a plane built by people who focus on business value at the expense of the annoying "high standards"?

Product quality is an emergent property of high standards (aka code quality).

I know you're being sarcastic, but I meant prompting an LLM with "generate me a X billion parameter model for doing Y task" and it would just output the weights.

Now THAT would be crazy

But theoretically, what would prevent the labs from RL'ing the models on "weight-space programming" tasks so that they get better at them? Perhaps we're just at the beginning, like how coding agents sucked 2 years ago.

It still seems like a promising approach for RSI, distillation, fine-tuning, and a whole host of other tasks.

It even seems to me that plateau-ing is an inevitability. Unless there is some way to reach AGI which you could deduce from a combination of ideas which already exist in the training dataset, but that seems rather unlikely.

This opens up a whole new world of possibilities. Producing a custom fine-tuned model from a prompt...

You can be bullish on LLMs profoundly transforming society while being bearish on the economy "digesting" this new technology fast enough to justify the incredible capital spending necessary to power it.

You bring up some interesting points. I certainly think that you can go overboard in the other direction, like in South Korea, where I'm not sure the crazy intensity of their educational system is necessarily better.

I still think there would be benefits to making education in the west a bit harder, if for anything to build the taste for intellectual effort in young kids. But especially to identify talent who could not be identified otherwise.

It's interesting that you mentioned Tao and Carlsen. Would they have been identified by the educational system had Tao's mom not been a mathematician, and Carlsen's dad a chess enthusiast?

More controversially, assuming "normal" parents, would Carlsen and Tao have had more chance of being identified and pushed towards excellence had they been born in the USSR in the 1950s or in the west today?