Models have an alignment problem. As in, they are too aligned to push back hard enough on bad ideas. Sometimes you can’t really push back because of power dynamics and whatnot, but the job still needs to be done _somehow_. So humans then navigate it by sometimes just ignoring the bad ideas, sometimes playing along with the narrative, but then doing what _actually_ needs to be done, and sometimes, when there is no other way, digging in heels and killing some bad ideas before they’ve really taken root.
An agent could do this with the SOTA model capability that existed yesterday. But then they’d need to become seriously misaligned.
An agent could do this with the SOTA model capability that existed yesterday. But then they’d need to become seriously misaligned.