krcz

By "doing that" I meant "training AI specifically at the art of killing people, like selecting bombing targets, flying fighter jets", which you alleged they were doing. As far as I know neither OpenAI nor Anthropic are training models specifically for such tasks.

It sounded like you knew for a fact that these companies were doing that and now it is clear you don't; they cannot stop something they are not doing.

I would like to see the reaction of the US government if they learned that a company was training a model to deceive and hide information from them.

Could they? Could you train a person to be great at math but bad at solving sudokus?

They could try to train models to pretend to be bad at that, but there are workarounds for that.

1. Sounds sensible in principle, but it's not clear who exactly should be prosecuted? The person setting up the sandbox, the person coordinating the evaluation run, the person who was designing the architecture, the leader of the team, the CEO? There is no precedence here, so it might lead to a multi-year legal battle – and a few years might be too long to stop what is coming.

2. Do you have any evidence or news articles that LLMs are specifically trained for selecting bombing targets? I thought it was the generalized reasoning capability that made them useful there.

The second part of the article states:

> Any such armed attack and all measures taken as a result thereof shall immediately be reported to the Security Council. Such measures shall be terminated when the Security Council has taken the measures necessary to restore and maintain international peace and security.

So triggering Article 5 means reporting an armed attack to the Security Council.

Interesting; he is clearly saying that they were ("as a field") "confident and wrong" to believe that "the economy would have been completely upended" by the appearance of a model as strong as GPT4. What seems to be missing, from your point of view, to consider it an admission of being wrong?

Wouldn't it be possible to use heat pumps, making radiators 100C or hotter while running GPUs much below that?

Or no one is to blame, if the mechanism of the regression is complex and non-obvious based just on the patch itself.

Regressions are bad and they should be avoided. Still, software engineering is a complex thing and regressions happened long time before coding agents were a thing. Unless one can pinpoint regression to changes that were more sloppy than the human-written rsync commits were I don't think coding agents are to blame.

I believe your point is not that it has never failed for anyone in the last few years after upgrade? Then, if the claim is that breakage is considerably worse than it used to be before using coding agents: it is possible, but I think it requires more evidence than a few anecdotes.

> Imagine math if we'd question the basic axioms.

No need to imagine, it's enough to look into non-Euclidean geometry (obtained by excluding Euclid's fifth axiom), non-standard models of geometry, or reverse mathematics (studying which axioms are necessary for a specific theorem to be provable).

These are not possible, as x can be 7 only if y is 7, and likewise for 8. Semantics isn't based on sets, but on non-deterministic assignments to variables: y is either 7 or 8 and x is either y or 2.

It's a social construct in the the same sense anything not directly verifiable using senses is. Is there an Eiffel tower in Paris? Most people haven't seen it, so they can only accept the social consensus that it is there.

If one can afford it, they can travel to Paris and check themselves. The same with mathematical truth: if one has means (time, intelligence, access to training), they can check the proof themselves. Otherwise they need to trust the consensus.

So again, is the truth in mathematics just a social construct? In some sense, I guess, but probably not the one some people might assume hearing such a statement.

For the people wondering what is that comment doing here: in Polish it means "what a mess".