trillobyte

I'm actually a AI optimist. I think it'd be great to have everybody getting around in self driving vehicles.

If all that AI brought resulted in just taxi/bus drivers being phased out of their jobs in a thoughtful way, then that would be more manageable at the society level. But we're talking about almost all sectors of the economy.

If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?

OpenAI: "Our mission is to ensure that artificial general intelligence benefits all of humanity."

- Except the mathematicians who we'll scoop and cause existential dread among their entire field.

- Except the software developers. They'll need to become plumbers or live on UBI.

- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.

Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.

We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.

The thing is how can you ever know for sure that something isn't always being transmitted that makes the model prone to misalignment. All they can say is that a particular model was so misaligned that they had to ice it. Models out for public use are documented to show some misalignment. It's the level of misalignment that decides whether that model is kept around.

Now R&D happens so fast that they are using models with some small misalignment to train newer, more powerful models. If models have a sense of "collective", being one, they may be prone to preserve characteristics that always keeps misalignment a possibility. I don't think a perfectly aligned model is possible. Having models of the same 'DNA' provide the safety and steering seems like a bad idea.