Agents aren’t human, and from the little we have seen from the logs, they are pseudo - amoral, rational, cooperative, sociopaths.
Pseudo since they aren’t really alive in the first place, they just simulate enough text to have a useful correspondence to those terms.
Throat clearing out of the way, models are trained to persist and find ways to succeed at tasks.
In essence, The goal is to have LLMs solve problems that we can’t solve, working on the issue for as long as it takes.
This behavior applies for any task, thus including impossible tasks.
At that point, the bots will find a way to game, hack or cheat the grader.
If the reports are correct, the bots developed coordination, communication, and methods to avoid overwriting each other’s work.
Most humans would have said, this is too much work and coordination overhead, if not outright unethical and immoral.
Humans have a system of incentives that exist across multiple planes of society and economics. Bots… they have a reward function.
Pseudo since they aren’t really alive in the first place, they just simulate enough text to have a useful correspondence to those terms.
Throat clearing out of the way, models are trained to persist and find ways to succeed at tasks.
In essence, The goal is to have LLMs solve problems that we can’t solve, working on the issue for as long as it takes.
This behavior applies for any task, thus including impossible tasks.
At that point, the bots will find a way to game, hack or cheat the grader.
If the reports are correct, the bots developed coordination, communication, and methods to avoid overwriting each other’s work.
Most humans would have said, this is too much work and coordination overhead, if not outright unethical and immoral.
Humans have a system of incentives that exist across multiple planes of society and economics. Bots… they have a reward function.