uv-depression

In my experience dealing with people who use LLMs, approximately none of them do this. The feeling of productivity LLMs create is maximized if you just blindly accept everything they produce, and apparently as a field we've broadly given up on what little quality we had before to chase that feeling (and to boost the valuation of a couple companies).

This is just textbook Gell-Mann amnesia. If the author doesn't read the paper how can he possibly know that? Even in this case when it's someone who has related knowledge, how can you know that you're being told something that's actually in the paper? LLMs confidently state nonsense all the time because there's no difference between how they produce a correct or incorrect answer. LLMs seem incredibly good at convincing people that they're now experts in everything. "Drilling down" is just giving more and more opportunities for that to happen.

Those people generally care about learning, IME. "Why would someone play a video game without getting paid?", more like "Why would someone pay money to have a bot play a videogame without getting paid?". People IRL do, of course; they tend to be antisocial and make the game worse for everyone else.

Yes! It's just repeatedly asserted that vibecoding is taking over, going to replace us all. I have yet to see evidence of it (no, marketing stunts aren't evidence).

The industry is in a desperate race to throw away what little quality it had (could you imagine if other fields of engineering had catastrophic failures at the rate software does?) in favour of "look at all the lines of code I generated today but don't actually read them please". It's always been a fad-heavy field, but this time it's really gone fully off the deep end. The whole "it's better than me" is just re-asserted endlessly. Nobody even pretends to care about evidence. I wish the boosters would go ahead and "leave me behind" already, I've been waiting years for it to happen now. The zone has never been more flooded, I think.

Yep. Nothing makes me lose interest in something faster than slop right at the top of the page...

I don't use LLMs at all. To be honest, I've been waiting years to "get left behind" now and it still hasn't happened. The main things I've noticed are that (a) like you said, LLMs have destroyed any culture of actually talking about programming and learning/sharing things; and (b) the quality of my coworkers' output has dropped proportionally to how excited they are about LLMs, and they've no longer able to answer questions about it during video calls. I wonder how much of their "productivity boost" is just those of us who still care fixing their shitty slop code... Oh, and I guess (c) that I spend more and more time trying to pull meaning out of a mountain of LLM prose that someone sent me, which is utterly soul-crushing.

> It's just from my personal experience

I guess let's just ignore the evidence, then.

> I can't speak to you other issues but these are here to stay and I think the only realistic way forward is to learn how to use them.

People only say this about things that they want. Ignoring the consequences of one's actions is not a neutral position. Societies are perfectly capable of banning things (yeah, you can probably sneak a local model through a hypothetical law, but it would end the industrial scale problems).

> I have 0 doubts some people are seeing 5x productivity boosts because of the stuff that is regularly posted on HN - stuff that used to take months taking days.

I will cede this: it indeed seems like you can throw together a shitty half-working prototype very quickly using LLMs. There's been a deluge of half-working prototypes, but years in and where are all these projects now? No study has concluded anything near a 5x improvement.

Besides, my actual opinion is that even if these actually gave a large productivity improvement, it's wouldn't be worth it. The negatives are simply too large:

- the concentration of power they are pushing for;

- the environmental effects of training;

- the disinformation they spread at scale (which in practice is their primary use-case);

- the adverse effect on a shared reality from the fact that they cannot reliably answer questions, yet are presented as doing so;

- the adverse effect on a shared reality from their sycophancy (do the rich surrounded by yes-men seem in touch with reality?);

- their anti-social nature, as they discourage engaging with other people in favour of solipsism masked by a poor simulacrum of personhood;

- their weakening of labour power because of the managerial class' belief that they will soon replace workers (whether or not they actually can is not particularly relevant);

- the large-scale plagiarism they enable;

- lastly, their weakening of already incredibly lax standards of engineering from our field.

The promise of more software does not even come close to outweighing all of that.

> 1. The people on HN tend to think they're very clever, but they're often not as clever as they think they are. That has a lot of wide-ranging implications, from classic "engineer's disease" thinking to stuff a lot dumber.

Related: the Rationalists display this very well (though it's certainly not unique to them). The jump from "I think my beliefs through carefully, therefore I am reasonably rational" (likely already false, or at best true in a limited domain) to "I am rational, therefore my beliefs are well-thought through" is actually very small in the human mind, despite the latter being the opposite of rationality.

Also, the existence of Gell-Mann amnesia should be taught to everyone!

> 3. I think there's a certain amount of defensiveness from AI enthusiasts.

I've encountered this as well. I think this is also a huge part of why you see so many people loudly exclaiming "every programmer uses LLMs now" despite that not being even close to true. If you criticize LLMs, it (socially, to them) counts as criticism of their users as well. As well, bosses are probably the most excited about LLMs because they hope to fire us all soon, so it can be pointlessly self-destructive to speak negatively about LLMs in a professional context right now.

That, plus AI boosters tend to be incredibly annoying about it, bringing it up at every possible opportunity, changing every technical discussion to be about them, suggesting them for the stupidest use-cases, loudly proclaiming nonsensical productivity boosts (perhaps you feel 5x as productive because you're forcing other people to spend their time fixing your shitty slop code?) etc. So as someone who does not use LLMs I avoid talking to them.

So we should hold LLM slop to a LOWER standard than humans? Great...

> therein lies the difference between this with an attitude of embracing new technology and those wanting to stick to the old

And as we all know, if something is new technology it must necessarily be good. Don't stop to think, just run in the rat race!

LLMs causing problems? The solution, apparently: more LLMs!

How would you know that the new paper contained the same content as the original (let alone the actual Lean code)?

People seem to be very likely to say that LLMs are good at things that they don't want to do. That's also why you see so much of "well of course they can't do x, but they're great at y" but everyone has a different x and y.

> I’m able to understand the paper after using ChatGPT to help me through it.

How could you possibly know that? If you don't have the knowledge to understand the paper without using a chatbot, you don't have the knowledge to verify that what the chatbot said is the same thing as what's in the paper.