I did indeed say that a superintelligent AI could realise that we don't want to paperclip the world. But it need not care what we want, unless we're really, really careful about how we program it.
What is "ridiculous" about continuing a task because we're not certain that it's done yet? It's only your human moral system saying that. Just because humans usually don't value things enough to pursue them to the exclusion of all else…
What is "ridiculous" about continuing a task because we're not certain that it's done yet? It's only your human moral system saying that. Just because humans usually don't value things enough to pursue them to the exclusion of all else…