matusp

The top reply is already all you needed:

> the answer with many multi-threaded programs is often "no."

> Yeah, and if you re-run the same program with the same bug, it will fail the same way each time.

[Citation needed]

> the models are going to be much better than us at that, too.

I wonder if this is true. The code produced by these models are not really getting any more elegant over time. On contrary, the models seem to be getting worse, often proposing really baroque architectures. You can use RL to optimize for correctness, optimizing the vibe seems much more difficult.

There are bugs found in programming language implementations all the time. I guess you are switching to machine code, good luck.

Why not always write machine code? Why use programming languages at all?

The agents here very likely used search. On top of that, they have boundless patience and can quickly process top K hits to find what they need. This is exactly the skill that is super useful for finding various niche sub-proofs that can help you build the final proof. A human mathematician is not going to digest 1000 papers from a different sub-field to find the needle they want, not knowing if it is actually there. AI can do it in few hours.

The Iraqi invasion if famous for how little nation building was planned really. The US troops went in, overthrew Saddam, and then they had not idea what to do next. Nobody was really thinking about how to rebuild after Saddam is gone.

The most salient motivation was (perceived) security. The US government was absolutely paranoid after 9/11 and they really did not want Saddam to continue ruling in Iraq.

My guess is that the models are overtuned on coding troubleshooting. You can regularly see them overthinking anything when you ask them to code something these days, mulling over countless eventualities. This is an okay idea for software engineering, but it leads to extremely diverging behavior in many other cases.

The problem is that empirically regime changes very rarely lead to stable liberal democracies. Yeah, a magical societal transformation according to exactly our standards sounds great to us. But that is not how this is going to end.

How much money are Europeans paying to US companies for AI services and how much are Americans paying to EU companies? I don't think it's a good idea to let billions of dollars go to other countries because you couldn't be bothered to build a competitive product.

This is a bottom feeder mentality. Europe has enough bright people and resources to truly compete in the AI race. There is something wrong when the only selling point is that it's local.

Mushroom pickers generally know only a handful of easy to distinguish types and only pick those. I would say that 99.9% of pickers will never get poisoned.

> I disagree with this. Decoders were absolutely dominant in 2020 for chat. GPT2 was considered too dangerous to release, and I remember scrambling to get on the GPT3 waitlist. It worked.

He's not talking about decoders, he's talking about auto-regression. Before ChatGPT, the dominant paradigm was fine-tuning BERT-like models.

> Before ChatGPT there really wasn’t much of a concept of pre-training and post-training.

Again, people spend years just post-training BERTs in various ways.

There is no theory why the gap shouldn't be closed. At the same time, people are trying to make them work for years by now and it's never good enough. But they are competing with incredibly optimized architectures.