Announcing the start of talking about planning the beginning of work on superalignment.
This is just a marketing buzzword at this point.
They admit "Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us. Other assumptions could also break down in the future, like favorable generalization properties during deployment or our models’ inability to successfully detect and undermine supervision during training.
and so our current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs."
That's kind of scary. Is the situation really that bad, or is it just the hype department at OpenAI going too far?
They admit "Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us. Other assumptions could also break down in the future, like favorable generalization properties during deployment or our models’ inability to successfully detect and undermine supervision during training. and so our current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs."
That's kind of scary. Is the situation really that bad, or is it just the hype department at OpenAI going too far?