Speculating about existential risk is certainly important, but way too easy to overstate. It can easily become distracting and actually dangerous in itself.
Specifically, the danger brought by hysteria. In our case, Alignment is probably much more effective and achievable when the widest possible community of researchers and engineers have access to knowledge, and much more perilous if they don’t because deluded parties thought it should be contained to a much smaller elite “secure” group.
Here’s a good take on it: https://www.ai-breakout.com/post/ai-alignment-and-the-messia...
I also think we would all be very wise to remember the story of Henny Penny / “Chicken Little”. https://americanliterature.com/childrens-stories/henny-penny...
Specifically, the danger brought by hysteria. In our case, Alignment is probably much more effective and achievable when the widest possible community of researchers and engineers have access to knowledge, and much more perilous if they don’t because deluded parties thought it should be contained to a much smaller elite “secure” group.