Comment

Comment by Veedrac

The basic argument is trivial: it is plausible that future systems achieve superhuman capability; capable systems necessarily have instrumental goals; instrumental goals tend to converge; human preferences are unlikely to be preserved when other goals are heavily selected for unless intentionally preserved; we don't know how to make AI systems encode any complex preference robustly.

Robert Miles' videos are among the best presented arguments about specific points in this list, primary on the alignment side rather than the capabilities side, that I have seen for casual introduction.

Eg. this one on instrumental convergence: https://youtube.com/watch?v=ZeecOKBus3Q

Eg. this introduction to the topic: https://youtube.com/watch?v=pYXy-A4siMw

He also has the community-led AI Safety FAQ, https://aisafety.info, which gives brief answers to common questions.

If you have specific questions I might be able to point to a more specific argument at a higher level of depth.

parent Veedrac 3 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton