The idea is that by its very nature as an agent that attempts to make the best action to achieve a goal, assuming it can get good enough, the best action will be to improve itself so it can better achieve its goal. In fact we humans are doing the same thing, we can't really improve our intelligence directly but we are trying to create AI to achieve our goals, and there's no reason that the AI itself wouldn't do so assuming it's capable and we don't attempt to stop it, and currently we don't really know how to reliably control it.
We have absolutely no idea how to specify human values in a robust way which is what we would need to figure out to build this safely
We have absolutely no idea how to specify human values in a robust way which is what we would need to figure out to build this safely