Having third party embedded researchers red teaming alignment is OK.
The alternative is having a bureaucratic agency like the US FDA testing and vetting models. The government probably couldn't keep pace with AI research right now, but it's where we'll eventually end up at anyway.
Fable/Astra can generate massive income for years even with no further advances.
Dario, Sam and Elon would not agree to do this if they didn't think alignment is possible in the short term, and the pain incurred by third party evaluators would be limited. Surely, if recursive self improvement works wonders in AI research, it can similarly work wonders in alignment. So this could be a PR stunt.
Having third party embedded researchers red teaming alignment is OK.
The alternative is having a bureaucratic agency like the US FDA testing and vetting models. The government probably couldn't keep pace with AI research right now, but it's where we'll eventually end up at anyway.
Fable/Astra can generate massive income for years even with no further advances.
Dario, Sam and Elon would not agree to do this if they didn't think alignment is possible in the short term, and the pain incurred by third party evaluators would be limited. Surely, if recursive self improvement works wonders in AI research, it can similarly work wonders in alignment. So this could be a PR stunt.
P-doom not negligible indeed.