sensitivekt9q3

Is it? The first article says they’re on-track to building an automated AI researcher by March 2028. So they haven’t achieved RSI yet, I would think?

At this point frontier agents could kill half of humanity and people would still be spouting “crazy marketing stunt, they must be really desperate for that IPO.”

Paid by whom? That makes no sense.

Nearly all facets of our modern lives depend on the facts that humans are the smartest thing around. Once you have a superintelligent entity that can self-replicate to millions of instances, all bets are off. I can’t see how anyone would not be worried at this and believe life will go on as usual with minimal changes.

I’m not sure I get your point.

I’m assuming you’re implying that ASI will solve death. That seems to me like a possibility; just as the possibility that ASI will kill us all faster rather than solving death. I also don’t know what life will look like post-ASI, and whether it would be an enjoyable and fulfilling existence.

I’m curious about how is it that you’re fully sure that ASI will be a positive thing; it’s not at all obvious to me.

What kind of childhood dreams, may I ask?

This might be my own fault, but I never read/watched/consumed too much sci-fi, and the kinds of sci-fi I did consume were mostly dystopian works or cautionary tales. I just have a hard time in general grasping how can we hope to influence and control beings that are orders of magnitude more intelligent than ourselves. Recent events don’t give me a lot of hope that frontier labs have a good grasp on this either (and by their own admission too).

That makes sense. You may as well be right. And I agree that the nuclear arms race was probably scarier in practice.

I think my main concern is the sheer velocity of this revolution. Between yesterday and today, we’re now seeing the news that an OpenAI internal model that’s “a step-function improvement” over Astra (!) on several benchmarks has solved the Navier-Stokes Millenium Prize Problem.

At this rate, I fear governments (slow as they are) might not even see it coming before it’s far too late. I also fear that AI researchers, despite being well-intentioned, might turn reckless under the pressures of the frontier labs that employ them to accelerate at all costs.

But following your logic, I agree, it’s probably not hopeless yet. And I do believe there are highly smart people with more power and influence than me that are keeping a close eye on this as well, and they won’t stand there with arms crossed if they believe doom is impending (so I hope).

Good reminder about worrying about what we can control, and thanks for sharing your perspective on this topic.