alecst

I know this isn’t an argument but when I read this my first thought was: How can a real person hold this opinion?

Solving Navier-Stokes and quasi-RH is “average” now?

You’re looking at where the ball is right now and not where it’s going.

And if we do solve it we’re pretty fucked, huh.

Maybe it’s a reason to start slowing down?

Even within my own lifetime it seems like people have less free time than ever, so I’m skeptical.

I’m seeing this perspective expressed a lot lately, but (so far) only online.

Understand that there’s more than one way to prove something, that there’s an art to making something understandable, and that humans tend to take the most understandable paths (to humans.)

Models often don’t do that. But in solving, or claiming to solve, important problems, they nevertheless remove a lot of the excitement and incentive towards working towards a solution humans would naturally find. Plus, a lot of these problems really don’t have any critical applications at the moment. They’re done for the love of the game.

Does that bother you? That your tax dollars pay people who do math out of curiosity?

Relieved to know the Assistant to the President for the Office of the First Lady is a key part of the SI Task Force.

Being a doomer sucks, man. But I see a lot of issues with your way of thinking, which I was basically getting at in the post you replied to. Let me explain:

Today it's a useful tool. But in not too long--a few years, perhaps--this tool will be able to outsmart you. If you think this is definitely not going to happen, then okay--we see the progress in LLMs differently. I see machines which, in a matter of three years, went from being able to do basic proofs, to math Olympiad problems, to century-old Millennium problems. I see no reason why progress will slow down.

Any "tool" which is that much smarter than you is no tool, any more than you are the "tool" of your pet dog.

I understand wanting to feel better and be optimistic about the future. I'm unfortunately not, and that makes a lot of people feel bad. Again, I'm sorry! I don't want to be a downer. But it's important to me that people take these risks seriously and don't minimize them with expressions like "we still have paper" and "it's a useful tool."

Most feel-good pieces on AI make the same mistake, which is looking at where the ball is right now and not where it's going. If progress were to halt today, this article would be relevant and I'd upvote it.

But since the richest and least-trustworthy people on earth are on track to make an ultraintelligent thinking machine, the risks I'm worried about are: killing most jobs, concentrating most wealth, and putting ourselves at risk of literal extinction. All I could think when reading this was, "this person really did not think this all the way through." I'm sorry.

Unhinged and honestly nauseating take. Were you hurt by a mathematician at some point? I don't even know where to begin with how negative and backwards this is. I guess I could start by saying that if you replace "mathematicians" in your comment with literally any other profession you could make the same criticism about greed and clout. With the "writing their names in blood" comment, I have no words. I will weep for the world if it turns out that this is how people see math.

You can see the difference between GPT 3, 4, 5, 6 each year. Same with Gemini and Claude. Look at image and text generation. Code generation. Math ability (Navier Stokes, Quasi-Riemann.)

We're not living in the same universe. Look at capability year-over-year. The asymptotic maximum is wishful thinking. Someone needs to step in before this gets wildly out of control.

> The assumption I need to challenge is that "an ultraintelligent machine could design even better machines".

Would you bet your life, and everyone else's, to test that assumption?

Not to mention: you can scale up performance in simple ways: more data, more compute, and more weights. Those are the obvious ones, ignoring architectural changes.

I don't understand the point of this debate. Rich's point is good. Having a mini-army of highly intelligent beings at your fingertips is not an ordinary "tool."

And it minimizes the fact that these things are only getting smarter. Having an entity much smarter than you--could that entity seriously be considered your "tool"? Just your tool? Or are the people on the "just a tool" side of this debate not taking that possibility seriously?