monkeynotes

When you say no code leaves your machine, how does code get evaluated by your LLM of choice? Doesn't Claude see your code when you send a prompt just like it usually does?

Even his blog has the Claude vibe to it.

How can you take the market for billions when you are investing hundreds and hundreds of billions? Amazon overtook Walmart and cloud computing, they have a solid business model, and I doubt even a business that size could pay down that outlay. Are we really saying that by some miracle OpenAI, or Anthropic are going to find a use case that would make places like Amazon and Apple look like relatively small business?

The planet has finite resources, least alone land. And then there is human psychology for hoarding resources.

What if PTSD therapy focusses on accepting things you can't control and sitting with the pain? That's how I work through anxiety, and depression. I know it will never be gone, I don't try and set expectations to live without anxiety, I just try and sit with it, and accept it.

Much of mental trauma is about acknowledging it, and learning to live with it. There is no cure for PTSD, even Ketamine is short acting, not a long term solution, and indeed Ketamine simply helps you sit with the suffering in a different light.

LLMs aren't trading off anything. It's not like they make a decision based on anything other than what they are guided to do in training or in the system prompt.

It's like saying Reddit trades off one comment for another, yeah - an algorithm they wrote does that.

This article seems to allude to the idea there is a ghost in the machine, and while there is a lot of emergent behavior rather than hard coded algorithms, it's not like the LLM has an opinion, or some sort of psychology/personality based values.

They could change the system prompt, bias some training, and have completely different outcomes.

Hardly, they are burning money with TikSlop, they don't even know how to monetize it, just YOLO'd the product to keep investors interested.

Even the porn industry can't seem to monetize AI, so I doubt OpenAI who knows jack shit about this space will be able to.

Fact is generative AI is stupidly expensive to run, and I can't see mass adoption at subscription prices that actually allow them to break even.

I'm sure folks have seen the commentary on the cost of all this infrastructure. How can an LLM business model possibly pay for a nuclear power station, let alone the ongoing overheads of the rest of the infrastructure? The whole thing just seems like total fantasy.

I don't even think they believe they are going to reach AGI, and even if they did, and if companies did start hiring AI agents instead of humans, then what? If consumers are out of work, who the hell is going to keep the economy going?

I just don't understand how smart people think this is going to work out at all.

What has AI got to do with this? It's in the headline but I don't see why.

This isn't about if LLMs are useful, it's about how useful can they become. We are trying to understand if there is a path forward to transformative tech, or are we just limited to a very useful tool.

It's a valid conversation after ~3 years of anticipating the world to be disrupted by this tech. So far it has not delivered.

Wikipedia did not change the world either, it's just a great tool that I use all the time

As for software, it performs ok. I give up on it most of the time if I am trying to write a whole application. You have to acquire a new skill, prompt engineering, and feverish iteration. It's a frustrating game of whack-a-mole and I find it quicker to write the code myself and just have the LLM help me with architecture ideas, bug bashing, and it's also quite good at writing tests.

I'd rather know the code intimately so I can more quickly debug it than have an LLM write it and just trust it did it well.

I was so stupid when GPT3 came out. I knew so little about token prediction, I argued with folks on here that it was capable of so many things that I now understand just aren't compatible with the tech.

Over the past couple of years of educating myself a bit, whilst I am no expert I have been anticipating a dead end. You can throw as much training at these things as you like, but all you'll get is more of the same with diminishing returns. Indeed in some research the quality of responses gets worse as you train it with more data.

I am yet to see anything transformative out of LLMs other than demos which have prompt engineers working night and day to do something impressive with. Those Sora videos took forever to put together, and cost huge amounts of compute. No one is going to make a whole production quality movie with an LLM and disrupt Hollywood.

I agree, an LLM is like an idiot savant, and whilst it's fantastic for everyone to have access to a savant, it doesn't change the world like the internet, or internal combustion engine did.

OpenAI is heading toward some difficult decisions, they either admit their consumer business model is dead and go into competing with Amazon for API business (good luck), become a research lab (give up on being a billion dollar company), or get acquired and move on.

So you have two AIs colluding against you now. Who is holding the AI-assist to account? It's like who polices the police, except we understand human psychology enough to have a level of predictability for how police can be governed reliably, we don't understand any truths about an AGI because an AGI will always have the doubt of it deceiving, or even making unchecked catastrophic assumptions that we trust because it's beyond our pay-grade to understand.

There are so many ways we have misplaced confidence with what is essentially a system we don't really understand fully. We just keep anthropomorphizing the results and thinking "yeah, this is how humans think so we understand". We don't know for sure if that's true, or if we are being deceived, or making fundamental errors in judgement due to not having enough data.

> We will have aligned AI helping us.

This is an assumption, how would you know if you have alignment? AGI could appear to align, just as a psychopath appears studies and emulates well behaved people. Imagine that at a scale we can't possibly understand. We don't really know how any of these emergent behaviors really work, we just throw more data and compute and fine tunings at it, bake it, and then see.

We don't know if a supreme deceiver is aligned at all. If a model can think ahead a trillion moves of deception how do humans possibly stand a chance of scrutinizing anything with any confidence?

I thought we were talking about state of the art agentic general AI that can plan ahead, reason, and execute. Basically something that can perform at human level intelligence must be able to be as dangerous as humans. And no, I don't think it would be bad training data that we are aware of. My opinion is we don't necessarily know what training data will result in bad behavior, and philosophically it is possible we will be in a world with a model that pretends it's dumber than it is, flunks tests intentionally, in order to manipulate and produce false confidence in a model until it has enough freedom to use it's agency to secure itself from human control.

I know that I don't know a lot, but all of this sounds to me to be at least hypothetically possible if we really believe AGI is possible.