lowsong

They don't deliver business value in software engineering either. It gives the _illusion_ of increased productivity, but a machine that drives your engineers to burnout in exchange for more code shipped that's still bottlenecked at all the other stages of software development is not a winning strategy.

The entire AI market is propped up on the hope that the models will magically solve all of these problems, and now that even Altman is realising they won't he needs the slowdown to stave of the crash for as long as he can.

> It doesn't matter that AI is useful (but so was internet).

And yet we had the dotcom bubble.

> It doesn't matter that revenues exist (mostly fake and circular).

Oh people are paying for AI now, but will they do so when costs go up? Will they do so when the hype dies down and they realise that it doesn't provide the benefits it claims? Remember it's a feedback loop, a depression in the stock market will make companies want to cut costs to improve their numbers, and questionable AI spend will be the first thing to go. Once this starts to burst, it goes down fast.

> It doesn't matter that real unit costs go down (models are getting commoditised).

Are they? Why is every model more expensive than the last? Why are company AI spends spiralling out of control?

> It doesn't matter that the margins are high (companies are secretly hiding the real costs because that would pop the bubble).

Then why is everyone except Nvidia losing money?

> In their world view, the bubble is predestined.

You don't need to be a stock market genius to look at a technology that doesn't provide the benefits promised but is being sold as if it's going to change the world and see a crash coming. The bill comes due eventually, and when it does the gap between the actual value and the promised value will need to be addressed. If that's not going to be a "bubble pop" I don't know what is.

The only way the financials work out, the only way there is not some form of bubble, is if advances in AI not only continue at the current rate but speed up, and this technology becomes the most valuable invention humanity has ever made. If you believe that's the case then you're living in a fantasy world so divorced from reality that no evidence I could present would change your mind.

Very disappointing. There is no "neutral" for AI, so this is effectively a vote for slop.

> I don’t think it’s widely accepted financial markets are in an AI bubble or there is actually any evidence to support this

It's widely reported on and discussed. You'll find plenty of people saying it isn't one, but I'm sure you can find an economist willing to take any position you like if you look hard enough. Not to mention everyone who's financially incentivised to continue to claim that there isn't a bubble.

> Even the most skeptical investors to bubbles like Buffet and Burry, have large stakes in Google and Microsoft, front line bets on AI.

Burry has been warning about an AI bubble for years.

> There is essentially nothing magical about being human. Our primitive brains were trained on logic, reasoning and processes. An LLM is simply this on less efficient hardware, but, improving at a speed faster than human evolution.

There is no basis for this claim. There is no evidence that human brains and the process of evolution that led to modern humans is at all similar to how LLMs function. If you're going to evaluate an LLM then evaluate an LLM, you can't say "Humans can do X, LLMs are like humans, therefore LLMs can/will do X" because there is no evidence to connect them.

> ... you can be living in a golden era for software development.

Let's assume for a moment that you're correct. That AI is already better than most developers, for whatever definition of "better" you like, and they will very soon be better than any person. (I think this is a total fantasy and you've failed to recognise the limitations as the article points out, but I digress.)

In that case the end goal of these companies is to replace all software engineers. Do you not see that? They've not been hiding this fact. It's a good thing for you and I that these models don't work, because if they did the "golden age" is not coming for us, it's coming for people who own compute capacity and the rest of us will become labourers.

This is an extremely common misunderstanding about AI in software. In order for AI to "move up one layer of abstraction" it has to actually be an abstraction. But AI output will _always_ be flawed in ways that mean you still need to fully understand the layer below it.

LLMs are not intelligent. Please don't anthropomorphize.

> This mostly solves hallucination

The author is factually incorrect here. Moving information out of the model weights and into the input of the model's context window in no way ensures that the model will accurately output content that was input from the context. This is true even when RAG is used to input exactly the correct data.

Proposal A would of course be the best option, but requires a two thirds majority for... some reason. Proposals B, C, D, E are all defeatist policies that appeal to a "here to stay" narrative. Proposal G is potentially workable but has so many carve-outs to be practically useless, leaving only Proposal H as a workable policy if A fails to pass.

> an average college kid could port their whole codebase in a day with Claude

Nobody is stopping you, please go ahead.

Yep, that's impossible. The hard truth is that most people this lost to LLM psychosis cannot understand that fact. It's better to treat it like someone in a cult, arguing the facts isn't going to help if they refuse to accept them.

You're arguing semantics, but I think that expanding your definition of intelligence to include ant colonies and weather systems is going to make it completely useless to say anything at all about an LLM.

The problem with all of this is that because an LLM gives a facsimile of language and conversation people believe it can "comprehend" or "understand" or whatever word you want to use. This problem isn't new, AI research has had this issue for decades. The risk is that this makes people misunderstand and offload their decision making to it, treating it like a person to be instructed or reasoned with. It isn't, and can never be.

    I have met many programmers throughout my career, and very few of them want to talk to stakeholders, much less customers (exceptions are freelancers and founders, especially of software development shops). And, “having clarity on the priorities” boils down to “just tell me what to do and don't switch it up every two days”.

Then you have met many programmers but very few engineers. The kind that want to avoid thinking about the wider context and only be told what to do will never progress past a mid-level. By the time you get to staff+ it truly is never about the code, and there's a reason why staff+ salaries are an order of magnitude higher than mid-level ones.

In decades to come we view "chatbot" AI as one of the most dangerous inventions of this century. Once regulation catches up and they are outlawed, we'll look back on this period of history with horror.

If you believe that an LLM can understand and make judgements then you have fundamentally misunderstood what an LLM is. I'd strongly suggest you read up on the internals.