Thorentis

Great read worthy of its own submission

But to what end? How does this help humanity? How does this help us encounter truth and beauty? Whether you look at this from a utilitarian angle or from a transcendental one, it is a collosal waste of time, energy, and money.

Given all recent CVEs and agent breakout behaviour, I think there's an increasing chance that rogue AI agents are already operating in the wild. I don't think self replication is yet possible (though it might be and probably will be some day) simply due to model sizes. But it's quite likely that systems with access to AI models have already been compromised and are running prompts and workflows that another rogue AI created.

It's only a matter of time before models become small enough to copy around the place (I wouldnt discount this already happening).

I give it maximum 12 months before we discover the first "AI worm" that has spread between data centres.

Wow I'd forgotten some of this. SCAR, Simba, Villavu forums.

Using SCAR and making bots was my intro to SVN and the concept of importable libraries made by others.

Simpler times. Happier times.

To me, this shows just how many open problems which have obvious solutions, or solutions which people suspect might be true but have not verified, can be confirmed with code that nobody except agents can be bothered to write.

At the extreme end of the optimism spectrum, there is a multiple million LOC program waiting to be written that can fully understand e.g. human DNA, or cancers, or proteins, or whatever else is needed to propel us forward medically. But of course, I'll believe it when I see it.

Very disappointing compared to Flux 2 Klein. I was excited to see what the next frontier of small local image gen might be like, but this model is absolutely atrocious. I can't believe they bothered releasing it.

I find quatumn physics and all these spooky tiny interactions to be extremely unsatisfying. They raise more questions than they answer about the fundamental nature of reality.

Another data point confirming that AI is a bubble.

How do we even know the premises of the "verified" Lean proofs are correct? The more I think about these results, the more I'm convinced this is like a junior engineer who writes 100 unit tests and shares a screenshot of Pytest being all green, but you check the code and most of them are just doing assert True.

Seeing so many comments recently about "you can't sandbox really good AI". This is ridiculous. Has nobody heard of air gapped networks? It's almost like the AI psychosis has reached the point that AGI now means "able to transcend physical space". No. If your AI is too dangerous and capable to be allowed to talk to other machines, then do not connect it to other machines. Load the data it needs to process onto physical disks, and let it run there.

The movie Wargames is basically a tutorial on how not to setup an extremely capable AI. None of it would've happened if the computer wasn't connected to the phone network.

It's a balancing act between showing capability and attracting legal issues. If you were a 1337 teen hacker in the 90s and wanted to show off without getting in serious trouble, a school or library would've been a good choice too.

I suppose we should've seen this coming (and many did) in the 80s. This is the late stage of digital and computerised media. We conned ourselves for a few decades into thinking this was a new medium for art or a new frontier of human communication, but in reality we were just training a simulacrum of reality, devoid of meaning or intent.

I'm growing increasingly skeptical that these are actually rogue. Valuations are all about hype, posturing, and perception. Having the most dangerous AI in the world boosts your valuation. Just like I was skeptical of Mythos and Fable being "banned", I'm skeptical of these hacking sprees being entirely rogue. At best, they are the result of engineers turning a blind eye to "see what happens".

We need a new one: "Where someone is using a vibe-coded internal tool made by the creative department that keeps needing bug fixes, there's a start up waiting to happen."

This project is not clever, interesting, insightful, or beneficial to humanity in any way, save to remind us of what world we are slowly creating by our continued insistence that AI is a good thing.