hodgehog11

I think this is an unfortunate reality that many miss. For simple and procedural tasks, yes, there are awesome alternatives. DeepSeek 4.1 Flash 1017 is my favourite by far. But for the headlining complex applications, and for consistently reliable reviewing, the gap is genuinely so remarkably far that it is hard to imagine having any alternative in the near future. The closest is the (very expensive) Kimi K3.1 Pro, which is, in my experience, roughly where Anthropic and OpenAI were near the start of the year. DeepSeek is a year behind in this domain. All open models have still yet to cross that monumental improvement that happened around Fable / Sol mid-year.

Because rigour is typically tied to math performance, my own experience has closely matched the FrontierMath Tier 4 benchmark. That should give you an idea of the difference.

I'm not foolish enough to say that AI is not economically viable. But these two companies specifically, OpenAI and Anthropic, just seem totally screwed. They're making great revenue, but their expenditure is unbelievably irresponsible (actually, their behaviour across the board is irresponsible). Training, testing, and marketing should not be this expensive.

I'm looking forward to the new wave of companies that will likely pick up after them, and engage in much more responsible business practices. These two have got to go.

And unfortunately, anyone who uses DeepSeek for serious math applications is extremely foolish, at least right now anyway. Disappointingly, the Chinese models are so far behind on math capability that it is a genuine moat at this point.

Totally different story for coding of course.

Yes, that's about right. It is much easier for it to roughly copy other code that it has seen, than to understand the code it has been given enough to make targeted fixes.

You can definitely still get PCSX2 working on it though, and it's fantastic. PS2 and PS3 games are among my favourite uses for handheld PCs.

I'm normally not the cynical type, and I am appreciative of Karpathy's efforts to educate people on developing language models, but this is exactly how it reads.

Every once in a while, everyone needs to touch grass.

Agreed, every behavior of this particular dev supports the hypothesis that they are producing mass slop to a "good enough" level to receive massive hype, without the intention to put in the thousands of GPU-hours it would require to polish (actually, the estimated roadmap is growing rapidly in expected timeframe as the AI understands the difficulty of the task). I expect there to be more and more of these crappy surface-level implementations before people lose interest and the dev drops all of it.

Indeed! Almost none of these most math folks are likely to read. The only thing to read is the statement, which is only a handful of lines in both cases. The point of Lean is that if that statement compiles and is validated by hand to be equivalent to the natural language statement, then it is true. That was what I was trying to say here.

That's fair, but I would argue that reading the specification is very easy by comparison. A quick one hour tutorial is usually enough judging from my students' experiences.

Why do you believe that learning Lean is a better use of time when the AI is clearly better at writing and interpreting Lean than it is writing quality papers? At this point, Lean is for autoformalization, no one is really supposed to read it.

Do you care to share then? Because anyone I know with any sense right now is aware that "planning for the next forty or so years" is genuinely impossible at this point, especially in mathematics, and the right approach is to remain on your toes, ride the waves, and diversify as much as possible. Heck, even planning for the next five years seems absurd. This is a time of immense uncertainty, and to think otherwise is foolish.

Yes. Most probably do not understand the notation involved in, and the statement of, the Lindeberg-Levy Central Limit Theorem. But every scientist uses this theorem in one way or another. These ideas have a way of trickling down because to people who work thanklessly to do so.

"Plausible" text was preferred over rational text when we trained LLMs using RLHF. It's rapidly shifting the other way now with RLVR, which enforces correctness by default.