Comment

Comment by DirkH

This is evidence of example 2. The worst parts of it are infohazardous. Organizations that work at the intersection of biosecurity and AI have strong NDAs and the like. Or so I understand speaking to friends in the space. There was also a paper years ago that showed a tweaked model coming up with 20k novel pathogens each deadly to humans in some number of hours

Either way, the biorisk has orders of magnitude more evidence than wild basilisk speculation. Please do not put those in the same category.

Also your evidence-first approach and no concern without evidence falls you into the turkey problem. Imagine being a turkey and being fed each day. Some other x-risk turkey is saying that the situation is suspicious, that the red barn is where some well-fed turkeys have been taken and none return and that this is happening less and food has increased so maybe all of us will go to barn at once soon. You tell the x-risk turkey they have no evidence a lot of turkeys have ever been sent to the barn which shuts them up. Reasoning by induction would get you "we are happy and thriving 99 days so should be good on the 100th" and then you get slaughtered on the 100th.

Consider the importance of first principles approach where you deduce the possibility of events that will happen only once and never again (like human extinction). Nobody with a "show me the historical evidence" approach would have been able to see the industrial revolution coming.

I think the gradual disempowerment thesis here is well argued enough to be considered the default track for what will happen to us: https://gradual-disempowerment.ai/

If you disagree with it, where do you disagree with it?

parent DirkH 1 month ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton