cousin_it

> And the study and the creation of those formalisms is not something automatisable.

Lol again. Have you tried it? Go ask your favorite LLM to invent a machine-checkable formalism for something that you vaguely describe to it. It'll oblige. If it's one of the top models and you give it a good budget, it'll do a good job. Very likely it'll do a better job than you.

Consciousness isn't required for problem solving. People said for decades that chess requires Genuine Human Creativity. When chess fell, people started saying that Go requires Genuine Human Creativity. When Go fell, people started saying the same about math; starting from simple problems and going up to Millennium problems. Now that a Millennium problem fell, people are still saying this. What can I say? Some people can't extrapolate from past evidence to likely future.

In my comment that started the thread, I pointed out that "coherence" in the article is considered valuable because it's a proxy for "ease of modification", but AIs can estimate "ease of modification" directly (by looking at how hard it is for them to make changes to a codebase) and learn from that in a closed loop. No round-trip through humans required, contrary to what the article says.

This is obviously pasted from an LLM, so yeah, bye.

(Sighs)

Goedel's theorems don't stop AI from finding and formalizing math results. They certainly don't stop AI from making conjectures or writing definitions. They have nothing to do with any of that.

The theorems are about inability of formal theories to prove certain sentences involving provability (the first is about "this sentence is not provable", the second is about "this theory is consistent"). Most mathematicians never touch any such sentences in their entire career. All the life's work of someone like Gauss, an all-time great, can be formalized just fine.

The article makes claims about the future, such as:

> We cannot make an AI that improves software coherence beyond the human ceiling without leaving the bubble

That's what I was replying to.

I think you maybe didn't read my comment carefully. The article says AI will have to talk to outside the datacenter to do this, I'm saying it won't.

> Have you seen it reliably do this?

Lol, no. I don't need to. I can predict such things.

For example, I predicted about 15 years ago that AI would decisively beat humans at math, because math is formalizable. Most people didn't believe it until this summer, and some don't believe it still.

To people who don't understand that AI improves over time, my prediction skill seems magical.

This is very high effort, but the central argument is wrong.

The central argument is that AIs, living inside datacenters, won't be able to evaluate the "coherence" of software "beyond the human ceiling". Okay; why do we need this "coherence"? The article argues that it's needed for ease of adaptation. But ease of adaptation (as measured in tokens required to add new features, or whatever) can be directly observed by AI as it works. AI can make conclusions about it, and carry them over from one program to another, without running into any human ceiling. So it can stay within its datacenter and make software that's faster than human-written, smaller, and more adaptable too. The whole essay dissolves. Sorry!

Yeah. I want to note quickly that the currentScript method can do a bit more than that. For example, the page head can include script.js with the code for rendering the site shared header and footer. Then in the page body you can have inline scripts using the currentScript trick to add the header and footer in place, kinda like including shared templates. And it runs during page load, before even onLoad, so the page renders with header and footer straight away.