pona-a

Communities form around shared friction. The sciences used to be sources of great community, solving grand problems with untold levels of friction. Having the right skills and knowledge to contribute could unite people from warring nations and social classes, let alone the benign evils like an anti-social culture, punishing people for talking to strangers...

In the imagined faux-utopia, there is no more adversity, and it's assumed no active effort was made to preserve its struggle. A god-like nanny will step in to meet your every need before it could even cross into your consciousness.

There'd be no point to music if Mozart could already hear the 6th symphony extracted from his brain with the first thought of inspiration. This art would in turn hardly be comparable to the real deal, even if it were an empirically flawless replica, is there is only so much personality you could extrapolate without a long execution limited by effort or resources. But a convincing replica of art might be all we need to sabotage the real deal, either through spam or demotivating the novice artists.

This future is, of course, ludicrous. It's a childish fantasy, a bastardization of sci-fi writers' nuanced utopias, flattening the social systems they wrote to give their characters motivation and meaning, in favor of fetishizing the technology, which was just a plot device to put the characters in such setting, and acting surprised when taking it at face-value will result in immediate social decay.

The author might or might not be surprised he's not the first to write on the subject, and real sci-fi literature already argues the same points in say Metamorphosis of Prime Intellect.

Perhaps that's why office jobs seem less productive than ever, since the invention of the computer... The usual meritocracy of the workers choosing their own tools is upset by top-down planning by insufficiently experienced people, having to bear no consequences for their choices.

git was picked meritocratically, Teams was not...

Religion is metaphysical, ethics are not. Simulation theory might pass as a religion, opposition to animal testing doesn't. Of course the latter does involve a moral axiom but one a lot more simple than religion's vast bundled sets of axioms in scripture...

I need to genuinely start saving HN comments, sorting them into two folders: insightful and psychopathy.

I think people can solve real-world problems better than LLMs including through code. A UX designer is gonna design a better UX than an LLM, but a worse one than a programmer with UX design skills, because the tacit knowledge of your users workflow and a good mental model of your codebase can be synthesized into a more cohesive experience than with top-down planning.

It's the problem of delegation: you can do more of something, but worse, the more people you bring in. LLMs are just a very cheap contractor with a zero liability clause and some unchecked sycophantic tendencies.

HN is one of the few sites where someone can launch into an actual villain monologue, the kind you'd read in leaked corporate emails, and receive approximately no negative reaction.

If this is the face of our industry, that might explain a lot of things...

They could A/B test feature flags like subagents without actually reading the chats.

I've been recently trying out FPGA programming in a niche HDL, and made a tiny promise to myself, that I would NEVER let an AI ingest any of my or others code doing it.

It's been exhilarating.

I've done things I couldn't imagine I could on my own, met some amazing people in the Discord, and felt immense satisfaction seeing the result and sharing it with friends. All things I thought AI permanently stripped from me.

If you keep training your brain to reach for the quick dopamine of asking Claude for a fix, you won't feel the long-lasting fulfillment from applying yourself to a hard problem. You have to quit that voice in your head, saying an LLM could do it in seconds, or it'd be too much work to study the problem yourself, and sit with some pen and paper. Your brain is so much more capable than you realize.

Now with a long laundry list of projects to try make, all I can ask for is more people giving it a go.

Idk, some queer people are ironically using it in that way, which kinda blurs the line. The word gay itself started out as an insult, but later became the preferred term people identify with, as opposed to the clinical "homosexual" which I'd identify as more uncomfortable today.

I don't know about others, but yes I'm entirely fine with people using singular they on me. My native language doesn't have an animate gender-neutral pronoun and I like how English gives me the option to actually be more gender-inclusive without an unnatural "he/she" construct.

If another cis man was suddenly uncomfortable with "they", I'd attribute that to some fragile sense of masculinity or a weird anti-woke crusade, as was the case with most such cases in my life, and find it personally annoying.

I don't want to deliberately make other people uncomfortable with my language, but it does feel exhausting to second-guess yourself, having to double-check if any innocuous phrase might be misinterpreted as an underhanded aggression, preempting a correction we know will be awkward for both sides.

I think we'd do better to consider context more than the specific language...

I meant fail in the sense that asking a lookup table of first 10,000 primes would pass any tests within this range perfectly, but when you ask it for the 1,001st prime, it would return an error. Mathematically an infinite lookup table of every prime is just a perfectly valid model of a factorization algorithm, unless you're interested in how one actually works, just as an infinite perceptron might be a valid model of any function.

> I really am still failing to understand whats the big deal about our brains emitting language and thought and a LLM doing so.

Our brains "output" language as a byproduct (or rather shadow?) of conscious experience. An LLM is a machine built to project similar shadows, but that doesn't necessarily imply there's an entire world like ours shrunk inside that box.

Both brains and LLMs are physical, and both output language, but one is an appliance built to synthesize language and the other is a living being that merely uses it as a tool. LLMs are made out of pure language in a way people are simple not. Our minds undergo physical changes with every stimulus we process, we conceptualize ourselves separately from our environment without needing to be taught an ego, and we did not emerge from an optimizer crossing wires in our brains until we can perfectly imitate some external intellect, just to list some of these differences.

The biggest issue is you assume the LLM has any "I" from pre-training. Take a pure completion model like GPT-3 and you'll see it's most appropriate to describe as a simulator, not an entity in itself. The model itself doesn't have any notion of an "I", it will gladly write dialog as any character, from Skynet to the user themselves, if you don't apply external rules to constrain it, first coming in the form of stop tokens and prompts, and having later evolved into RLHF, bashing a "personality" into it by mangling its existing probability distributions (which is probably why all the most "aligned" models write in the most unnatural language, worse than those early non-chat LLMs). Any persistent self an LLM demonstrates was explicitly imposed on it by humans, which makes them obviously categorically different from animals or disabled people.

If your simulacra actually fails on certain inputs in a way, like say a lookup table would be, then I'd say it's not a factoring algorithm.

You can generalize intelligence to a lot of things. A* or minimax are algorithms that make intelligent decisions, often much better than a human. Consciousness is a very abstract notion, or maybe even a social construct, which we usually agree centers on human experiences, and progressively (not always, see historic religious scholars likening cats and dogs to automata lacking the free will granted to humans) expanded to animals, or generally all living things with complex enough brains.

LLMs are by their nature dissimilar from any biological organism. Biological intelligence evolved in bodies to locomote away from threats and towards food/mates, over time growing to model the physical world as it relates to these goals, and eventually becoming sophisticated enough we can manipulate entirely abstract objects, like algebraic terms, just as early homonids or crows could use physical tools.

LLMs by contrast are algorithms trained to mimic human language. Without extensive post-training and prompting, they would not have a concept of a self, and even with these they can still occasionally confuse roles in a conversation. They can't learn beyond the limited memory afforded by context, which tends to simply degrade the more you utilize it.

I suppose Q-Learning does demonstrate a more similar dynamic to nature, and you could draw parallels with RL on LLMs, but I still think it's categorically different from the entirely open-ended environment that is the natural world.

You could say what makes it valuable for us to empathize with octopuses, even though their brains have a fundamentally different structure from ours, is our shared experience of pain, and you could interpret negative rewards as a kind of painful stimuli. But at that point, you've opened up a whole can of worms that you can't close until reductio ad absurdum.

As for any human-like behaviors, I think the answer should be pretty obvious. It's an algorithm trained to generate text, of course with enough expressive ability, the optimizer will seek to model emotional utterances, assigning a speaker's general mood to some internal parameter. But as I've said, it would be a trap, even if you're very much not a stochastic parrot believer like say most Less Wrongers, to infer that just because it speaks like us it thinks remotely like us (see the whole Shoggoth meme).

Nice! CSS-Tricks served me well over the years with whatever limited web-dev I did. I'm glad it's not actually shutting down. And if the author is reading this, I hope you can ignore all the nay-sayers. There are people who still want to actually know their tools instead of leasing their brain from Anthropic, and there's been a shortage of great teachers.