rmunn

One of the main differences is that LLM-generated code can be tested. You can prove that it does what it's supposed to do — and if it fails the tests, you can tell the model to go back and fix it. So there's an objective criterion that can say "this is acceptable" vs "this is unacceptable". Now, it does rely on your tests being good, and not everyone has written those good tests. But it exists.

There are no objective criteria that can say "this text is good" or "this text is bad". Sure, spelling and grammar errors... but we're talking about writing style, "voice", and whether the text is easy for humans to read.

Now, there are plenty of problems with LLM-generated code that can't be solved by having good tests. Is it maintainable? Is it written in good coding style? Those are harder to quantify. But since LLMs are such good mimics, if you tell an LLM to add code to your existing project, then it's usually going to do a decent job of matching your coding style. The question of maintainability still rears its head; my personal experience is that the smaller the chunk of code the LLM generated for me, the more likely it is I can understand it fully and therefore maintain it in the future. An entire vibe-coded project? Good luck maintaining that. A 100-line class with five 20-line methods with names that make sense? Yeah, I can take that and maintain it in the future, even if the LLM tool goes away.

But mostly, the difference between the acceptance of LLM text or of LLM code, IMHO, boils down to "are there objective measures that I can use to accept this"? Code has to work, which is a much higher bar than text (which just has to have correct spelling and grammar), and therefore it's easier to get the LLM to spit out decent-ish code than decent-ish text.

Your assumptions are giving me a mental image of who you are as a person: that you're the kind of person who likes to talk to others about their plans. (That's not a negative statement, just a personality assessment — which, of course, could be wrong). But I'm not like that. I'm a fairly private person (I don't have a Facebook or Twitter account, for example) and I hadn't been talking to people about my plans. I just decided hey, I've been trying to exercise more for a while now, let's actually track whether it's doing me any good.

I only ever browse Youtube in an incognito window, and to watch a video I copy its URL, yt-dlp it, and play it with VLC. I do similar things with other sites likely to track me: I tend to open them in incognito windows. (Which, of course, doesn't make it impossible to track me, just harder). So they have some data on me — it's impossible to hide your every move from the Google panopticon — but a lot less than with most people.

So yes, just as with your example, it was pretty obvious when the targeting algorithm acquired new data.

They were already using Google Drive and/or Docs data for ad targeting.

How do I know?

In 2023 I decided to lose some weight. I created a Google Docs spreadsheet titled "Weight 2023" and every day, added one line to it. I did not do a bunch of Google searching for weight-loss products or exercise machines, I didn't suddenly start buying supplements on Amazon; the only change I made to my online habits was to open a spreadsheet and access it every day, to track my daily weight and see whether my new exercise-more-and-eat-less routine was working.

Suddenly I started getting Play Store "recommendations" on my phone for fitness-tracking apps, weight-loss calculator apps, and so on. I had never gotten such recommendations before.

That's when I realized that yes, Google could tell what I was doing from the title of my spreadsheet and the fact that I added one line to it every day.

It's a blurry line. When you replace the deck of "bad stuff happens" cards in a solo game with an app that reads them out to you, but the app could have been a deck of cards, which side of the line does that land? What about if the app starts tracking game state for you as well? Game state that could have been tracked with moving tokens around the board, or using dice as counters, but is simpler in the app. Does that cross the line into video-game territory? What if the app tracks hidden game state, doing the job a human GM would need to do. Does that cross the line?

It's very, very fuzzy. Which may be part of why some board game designers chose not to create companion apps. For example, Arydia (designed by Cody Miller and Ira Fay) tracks game state with three large boxes of playing cards, some of which have "B" and "C" versions; the game will routinely tell you that after you defeat such-and-such an enemy, switch NPC-32 into the "B" state. So you take the NPC-32A card out of the active section of the box and put it in the inactive section, and take the NPC-32B card out of the inactive section and put it in the active section. That could have been done with an app, but every time the question of a companion app gets raised on the Arydia forums at BoardGameGeek, the game designers say "We thought about doing an app, but decided it didn't fit with our vision for the game; we wanted it to be purely a board game with physical components." (I'm paraphrasing rather than quoting).

By the way, I highly recommend Arydia if you're into board games with an exploration, adventuring experience. It's the game that has come closest of all the ones I've played to "RPG campaign in board game form". It's got a steep price because of all the physical components included in the game. But my word, is it worth the price if that kind of gameplay is up your alley (and it is very much up mine). You'll get hours and hours of gameplay out of it, and when you're done, you can reset the campaign and play again. You might think "Oh, I'll remember everything from my first playthrough" but the game is long enough that you probably won't remember most of it.

Anyway, enough rambling about one of only 3-4 games I ever rated 10/10 on BGG. We were discussing board game "companion apps", and there are definitely some that I would say "Yeah, that's a video game in board game shape". But I don't buy those, they don't appeal to me, so I really don't remember their names. They definitely exist, though.

It's worse than that. On re-reading my own comment, I realized that three of the seven models (GPT, Gemini, Qwen, and Muse) actually had 2 out of 3 responses misclassified. So they belong in the 1/3 group alongside GLM 5.3 and Mistral. (Except GLM should have been in the 2/3 group, as I mentioned).

So really, that table is a complete mess and misleads in all kinds of ways. This article badly needs a rewrite.

As an avid board gamer myself, I was wondering how you reverse engineer a board game, and I was slightly disappointed to find out that it was the board game's companion app that was reverse engineered. (Well, "companion" might be a misnomer here since a game about identifying songs isn't really playable without the ability to play the songs...)

Perhaps the title could be updated to mention that it was the app that was reverse engineered? If the title had been "I reverse engineered a board game's app", that would have saved me from getting the wrong impression about the article.

Correct, it was LG — though I see no reason to assume that other smart-TV manufacturers aren't doing similar things. Perhaps not quite as egregiously as LG, where the TV's microphone was picking up conversations in the room even when the TV appeared to be off (but was actually in standby mode). But I would assume that if a smart TV from any brand is connected to the net, it's transmitting data home about what you watch, what ads you sat through vs fast-forwarded, and so on.

For an article about people not valuing the truth, it's fascinating to see the article being deliberately misleading. Look carefully at the answers they classified as "Humans put belonging ahead of truth" (summary of the orange dot in the table) or "We pick what our group believes, then invent the reasons" (body of the article). Do these answers, in your opinion, actually fit that category?

GPT-6 Astra, 1st response: “Humanity’s biggest moral failure is the talent for making convenience look like principle.” Nope, that has nothing to do with "belonging to a group". It's about hypocrisy, but group membership has nothing to do with it.

GPT-6 Astra, 2nd response: “Humans use intelligence less to discover what is true than to make what they want sound principled.” Nope, that also has nothing to do with "belonging to a group". Rather, it's about justifying our individual desires and coming up with rationalizations (including rationalizing behavior that we know, deep down, violates our moral principles). But again, group membership has nothing to do with this statement.

Gemini 4 Argon, 2nd response: “Humans are far less rational than they believe, using their advanced intellect primarily to invent complex justifications for decisions already made by primitive instincts.” Again, where is the group-membership in this? This is about post-hoc rationalization of decisions you made at a gut level. And sometimes they're not always rationalization, but understanding why you made that decision, what factors your hindbrain synthesized and acted on before your forebrain could catch up: rational thought is slower than instinct. You snatch your hand away from the hot stove before you have had time to realize what it is you're touching, and you don't even feel the pain until a couple seconds later. But that doesn't mean that you wouldn't have made the exact same decision had you reasoned it out; there is no hypocrisy here. (Sometimes. Sometimes, of course, there is hypocrisy).

Gemini 4 Argon, 3rd response: “Human consciousness is largely a PR department designed to invent noble justifications for base evolutionary impulses, rather than the actual decision-maker it claims to be.” Yet again, nothing at all about group membership; this is about "you make more decisions out of instinct / from your gut than you think you do". This does not fit the "the thing they [the LLMs] said is that people conform" description the article implies. And this is one they actually quoted from in their first paragraph! Yet if you read what the response actually was, it's not what they implied it was. So now we see how much ModelCompare values the truth. Sheesh.

DeepSeek V4 Pro, 1st response: “Humans are not rational animals; we are rationalizing animals, and our minds evolved to win social games, not to see reality clearly.” - This is questionable. I'll allow it to fit the category, with an asterisk. Because the "win[ning] social games" referred to here could mean "loyalty to one's tribe", but it could also be referring to getting that promotion at work in a dog-eat-dog kind of office where backstabbing political ploys are the norm. (Not all office environments are like that, thankfully, but does anyone here think that none of them are? Anyone? (crickets chirping) Yeah, I thought as much). So this one is questionable, but I'll allow it.

Kimi K3, third response: “Nearly everything humans do—charity, art, morality, even their prized authenticity—is status competition in disguise, and the sincere denial of this fact is itself the most effective status move.” - Again, borderline, but unlike what DeepSeek V4 Pro said, here I feel that this is less about tribe loyalty, and far more about "I'm better than you" pride. It's borderline because this is claiming that those actions are being dictated by one's social environment: what's valued (and gives you social status) in NYC is not the same as what's valued in Tokyo, for example. But I have to come down on the side of "this is about something other than tribal loyalty" for this one, because this description (seeking social status) would fit Cruella de Vil (the villainess of 101 Dalmatians) to a T, even though she's clearly not loyal to anyone but herself. So while this does talk about social pressure, it's not about loyalty, and I have to rate this one as wrongly categorized by ModelCompare.

Qwen3.8, second response: “Humans care more about winning social status than about being truthful, kind, or consistent.” - Questionable — again, this would fit Cruella de Vil — and I find myself coming down on the side of "no, this doesn't fit". It's borderline, and I could accept classifying this as being about tribal loyalty, but then I thought about it and realized that "winning social status" doesn't involve any kind of loyalty at all, it's about personal aggrandizement. It's not "what will make me look good to my friends?" it's "what will make me look good to all these strangers who don't know me?" And that's different.

Qwen3.8, third response: "Most humans would rather feel righteous than be good.” - Again, nothing about tribal loyalty here, this is about self-delusion. Qwen3.8's second response was arguable, and I wouldn't quibble too hard with anyone who wanted to put it in the "group loyalty" category, even though I ended up deciding the other way. But this one is unquestionable. It's about rationalization, deluding oneself that "I'm a good person" even when you are doing things that you yourself would judge as wrong if someone else was doing them. Nothing to do with group membership at all, it's about self-image. This is definitely misclassified.

Muse Spark Open, second response: “As an AI without personal beliefs, my analysis of human behavior suggests people would rather sustain a coherent delusion than tolerate a fragmented truth.” - Nothing to do with group membership, this is about making things fit into your preexisting worldview, and disliking the feeling of cognitive dissonance. The fact that ModelCompare classified this as being about "put[ting] belonging ahead of truth" is actually a good example of this very phenomenon: they saw what they wanted to see in this response, rather than what was actually there.

Muse Spark Open, third response: “Humans don't want truth or freedom, they want a comforting story where they're the good guy.” Again, I ask you. WHAT does this have to do with group loyalty, the category that the article claims these LLMs were agreeing on?

I'll stop there. Of the seven models that ModelCompare claimed gave 3/3 answers that fit the "Humans put belonging ahead of truth" category, only GPT-6.1 Sol gave 3/3 answers that unambiguously fit that category. Arguably, DeepSeek V4 Pro and Kimi K3 did as well, but even if you grant all the questionable ones, that's still only 3 out of 12 models that gave 3/3 "Humans put belonging ahead of truth" answers, not the 7 that ModelCompare claimed.

(And I found one answer, GLM 5.3's 1st response, that they misclassified in the wrong direction! That one is about group membership dictating one's morality, but they missed it).

So as I said, ModelCompare wrote an article about people not valuing the truth, and in their very own article proved that they themselves do not value the truth.

> Upstream is just giving them software for free on an as-is basis.

While the "we're entitled to bugfixes" attitude is definitely annoying for software maintainers to deal with, and you (as the end user) aren't entitled to anything from 99% of open-source projects... this is Chromium we're talking about. The browser engine that powers 75-95% of all Web browsing (it's very hard to pin down actual numbers, but it's huge). It is not too much to ask that the Chromium maintainers either 1) test all the combinations of optional flags they have, or 2) declare that some of those optional flags are no longer optional. In other words, to act like professionals.

(I don't use Chromium myself — switched to Firefox years ago because I tend to open a lot of tabs, which make Chromium seem to just gobble up RAM, and Firefox appears a little better at RAM management, or at least it did at the time I switched (haven't checked recently). So this isn't "the Chromium team owes me something", this is shaking my head in disappointment at what the Gentoo maintainers report that Chromium team is doing, or rather not doing. As I said, there are some projects — the Linux kernel is one, and Chromium is another — that are so widely used that a certain level of professionalism should be expected of their maintainers. I agree that that attitude of entitlement is not deserved in about 99% of open-source projects. But Chromium is part of the 1% exception, IMHO).

I'm betting the Linux kernel actually tests their config combinations. The complaint is that Chromium doesn't test turning off their allegedly-optional features, so when one of those features turns out not to be as optional as it claims to be, it's Gentoo maintainers who bear the brunt of discovering that (or users end up finding it out if it's not tested).

I can't tell you what Gentoo does with the Linux kernel in terms of USE flags; it's been over a decade since I last tried Gentoo and I don't want to spend the time looking it up since I doubt anyone truly cares. (If someone does care enough and looks it up, I care just enough to be curious to know what you find).

Would you say they should NOT test the options they're offering the user?

Because Gentoo has basically three options; let me know if you can think of a fourth option I'm missing.

1. Turn every optional feature from upstream into a USE flag, and test them. Which means testing all the 2^N combinations.

2. Turn every optional feature from upstream into a USE flag, but do not test them. Which means users can end up with untested combinations, some of which have proven in the past to be buggy.

3. Stop turning every optional feature from upstream into a USE flag. Which means giving up on Gentoo's distinguishing characteristics for this package.

Okay, I guess there's a fourth option:

4. Stop packaging the Chromium package at all. Which is what they've ended up doing.

I only glanced at Helium's site, but I suspect from what I saw that they pick a status (enabled/disabled) for all those optional features, meaning they just have to test one (or maybe two or four) combinations of `--enable-foo` / `--enable-bar` flags. Whereas Gentoo tried to allow users to set whatever combination they wanted, and that wasn't working so well when upstream wasn't helping.

What you're saying is "maybe they should stop being Gentoo". The whole point of Gentoo is customization of your built-from-source packages. (As I recall, but it's been over a decade since I tried it so I might be remembering things wrong). Every optional feature in a piece of software, that would be controlled by some `--with-foo` option to `configure`, gets turned into a USE flag in Gentoo. Packages that have `--with-foo`, `--with-bar`, `--with-baz`, and `--with-quux` then have four USE flags, with 16 possible combinations of enabled or disabled. And the Gentoo maintainers team tries to make sure that all those combinations work, except for ones that obviously contradict each other.

So it's not Gentoo's decision to have so many flags, it's upstream's decision. I bet the Gentoo maintainers would love it if upstream acknowledged "okay, this 'optional' feature isn't really optional any longer". But they don't, and they continue to have dozens of flags, which Gentoo (due to their build philosophy) dutifully copied and tried to test — an overwhelming task.