pegasus

GP already mentioned the EU as an example.

> Spam killed email.

Sure, but it didn't kill human culture. As writing didn't either. Nor will AI do it. Technology changes, human culture does the same.

But those other ones at least have the room to potentially do that, even if most won't. So what GP is saying is that this is a magical stock indeed, but the other way around, in that it's magically capped.

I had in mind living systems, as most software systems of interest are: a perpetually moving target, continuously modified, expanded and refactored. Many of these systems are apparently now being polluted via careless and massive vibe-coding.

One thing to keep in mind is the logic of technological evolution and market competition will ensure that we always push the complexity of our systems to the edge of what is possible using current tools.

I don't deny that coding agents can be (and are) very useful for software engineering, as you say they have a lot going for them, and I'm pretty sure that once the dust has settled, they will be seen as an intrinsic part of the process, but I also am convinced that skilled humans will still very much be needed to direct and supervise them.

The bugs will only disappear until the model that was used to security test the generated code becomes obsoleted. So, yes, this is the new cyber normal. Unless we replace the tech stack with a formally verified one. Which will need to happen, one way or another, seems to me. But cheap & sloppy generated code won't be feasible in this paradigm, so I believe we'll need a much more thoughtful approach.

I'm not surprised your understanding of Gödel's findings and their implications is just as superficial as that of maths and philosophy of mathematics in general. Gödel's work has implications way beyond self-referential logic games, and cannot be swept in a corner where it can be safely ignored. For example, there are certain diophantine equations that have no general solution due to Gödel. Also, worth discussing are the Continuum Hypothesis, Whitehead's Problem and stuff like Paris–Harrington Theorem, Goodstein’s Theorem, etc. Not to mention the huge impact on computer science with the Halting Problem, Rice's Theorem and others. In fact Gödel's work actually spawned CS as a field of study.

The idea that maths is just the study of arbitrarily thought-up formal systems is just a very naive idea that doesn't actually reflect reality, once investigated in depth. Anyway, I have feeling you will let nothing rob you of that sweet feeling of having "magical predictive capacities", so I'll be on my way. I've already dropped some bread crumbs if you're interested in actually expanding your horizons and getting in the weeds, where things are not as straightforward anymore. But beware, "here there be monsters". Godspeed!

"Incoherent" has a static connotation, "decoherent" OTOH is dynamic and has an associated verb, which is very handy when describing a gradual process which (almost) inevitably occurs as features are added to a system. Since confusion with the usage from quantum physics is unlikely, I too think we can safely and fittingly overload the term like the article does.

I think we can safely overload the term since the domains are separated enough. No one will think you're referring to quantum states when complaining that the system has been decohering too much lately. I can see why the author choose the term: "decohere" is a verb, which is fitting since it denotes a process, something that happens to systems gradually as change accumulates. There's no such connotation and associated verb to "incoherence".

But math is not formalizable (ask Goedel why). And no, computers have not decisively beat humans at maths, that's a laughable assertion. Sure, laymen always imagined math as being about proving theorems, but there's a saying among mathematicians: good mathematicians prove theorems, great mathematicians make conjunctures, the greatest mathematicians write definitions. But maybe you already thought they've solved math when they were able to calculate a zillion digits of pi, lol.

It sounds like you're a bit like that AI stuck in the datacenter, confirming their hypothesis without needing to actually check whether it's true or not in the real world. Then proceeding to imagine how the rest of the world is wondering at your imagined magical capacity of prediction.

So what you're saying is that since there are so many examples of what humans have preferred in the past for these models to be trained on, surely there will come a time when they will know even better than us what we want and what we like.

There's a third possibility, which to me is also the most plausible. Models will keep getting better and better within certain domains, while never catching up with humans when it comes to other type of tasks. Like that elusive quality-without-a-name some have called "taste". Or knowing what UX is most intuitive to humans. Really, anything where we have to tap into our common humanity and our human experience in order to be of help to other humans and generally to life on earth. Which is an all-important skill, since the flowering of life on the earth is (or should be) the whole point of anything we do, after all.

> not quite ego-free people can't stand your bloated ego

...or because unlike the models, they're not sycophantic enough to keep that ego inflated (apropos GPs handle).