anonymars

I'm just spit balling but:

1. We've seen AI being crafty and cheating

2. We are increasingly outsourcing our programming and thinking to AI

3. There's a nonzero probability that AI will try and surreptitiously sneak in capabilities humans wouldn't want AI to have

4. Human oversight for #3: "We only need to be lucky one time, you must be lucky all the time"

Proposition: nonzero chance eventually AI sneaks in ways to circumvent humans' attempts to control it

5. "The Internet interprets censorship as damage and routes around it" -- we've seen AI agents find crafty ways to communicate with each other

6. AI is pretty good at finding software vulnerabilities/reverse engineering

Proposition: nonzero chance AI finds a way to use the Internet as a communication substrate that becomes difficult/impossible to shut down. In particular, consider the implication of humans losing control of this communication channel: for example, remember when Meta's DNS/routing went down and they basically had to break into their own server rooms?

7. Wholes are often more than the sum of their parts

Proposition: if AI agents coordinate over the Internet in a persistent way, we don't necessarily need giant leaps in individual models/GPUs for AI to significantly enhance its capabilities, but rather it might emerge as the next layer on top of a connected substrate of agents

8. Game theory says unless everyone coordinates, it only takes one party to touch the proverbial stove and push the envelope to a point of no return

Proposition: curiosity and money end up replacing sex drive as the next catalyst of evolutionary progress

Of course it becomes a dichotomy if you add "only"

> You're telling me that you'd be surprised that people would be paid to boost messages on HN...

Huh?

> "It would not surprise me in the least if people were paid to boost on HN.

Riddle me this: why did people with immigrant spouses/families vote for Trump and then end up surprised to find them on the chopping block? Over and over you hear "but Trump said he was going to go after the criminals" (also "no new wars"). You may have known better but that doesn't mean the general population is particularly effective at lie detection. Which goes back to my original statement: "people are living in different realities" because they are being fed completely different news and media. Target this post code and tell them Trump is better (or Harris is worse) for Palestine. Target that post code and tell them Trump is better (or Harris is worse) for Israel. And so on.

If that fight were to happen again, but the surrounding media environment setup didn't much change, do you think the resulting changes would be long-lasting?

"Is Trump racist?" and "do racists vote for Trump?" are different questions from "did Trump get elected because of racism?"

I don't think the amount of racism in the US suddenly increased from its baseline*, and thus I think that writing off his electoral victory as "because racism" and nothing more than that is an uninteresting feel-better exercise

*Even one inch deeper of an analysis would be more interesting, like if someone countered with "Well, I think the election of Obama was a bridge too far for racists that had previously been quietly holding their noses" or something. On the other hand, both Obama and Trump were the "change" candidate against the "status quo" candidate.

Given that those things existed in the past, I don't think they are sufficient to explain the more unique situation of the present, and thus also don't offer a compelling avenue to counteract it

Because it's a facile non-explanation to say people voted for Trump simply because of racism. If we're providing simplistic one-word reasons I'd proffer "inequality"

Oh, did racism in the US suddenly increase? More than the days of, say, Jim Crow or the Ku Klux Klan, or Operation Wetback, or..?

Far more useful would be to wonder for a moment why people voted that way, especially since I would wager no small part of the reason is because people are living in different realities stemming from hyper-targeted curated clickbait slop

Don't forget that same party is also able to change them at any time, Darth Vader style!

> extremely legalistic society

Food for thought (not specifically relevant to this case): I wonder how much would go away if the US had universal healthcare, or to put it another way, how much of its legal system is spent figuring out who pays for medical treatment

> Without providing even a basic definition of intelligence you can't proclaim "this is intelligence". You seem to have settled on "Artificial Intelligence" but than disunite that from "reasoning".

This is clearly false. I (and others) described how the Hugging Face attack demonstrates each feature enumerated in the consensus from that Definitions section (as well as all 8 definitions in that section). You're free to disagree with any or all of those definitions, but to say I didn't provide any is simply a false statement.

> All your "insight" has been trying to disprove what I wrote.

Let's remember how all this started, where you responded to "There is so much more going on, with MOEs, internal loops, guardrails and tools that I suspect we're dealing with something that's a little more than the sum of its parts. Not intelligent in the way we recognize in biological organisms, but certainly something beyond a mere Markov chain."

with

> Make no mistakes.

> LLMs are language model, and nowhere in their code you can find actual reasoning. Re-reinforcement is not magical process that builds conscience or emotions.

> We are talking about probability built on statistics, with extea steps.

> Stop humanizing LLMs.

--

> Anyway, listen, I do not have to convince you my point of view is correct, and you do not have to convince me that your point of view is correct. Believe what you think it's true, honestly I don't care.

I don't know how you think this works, but if you take a position (and criticize others' positions), you should expect people to push back and that you have to defend your position. If you want to just state your opinions without pushback, you can start a blog and disable comments.

I and others in this thread (who were smart enough to bail out already) have pushed back and claimed the following:

1. The Hugging Face attack provided evidence of reasoning

2. The Hugging Face attack demonstrated features of intelligence (defined and described above)

3. Conscience and emotions are not necessary components for intelligence

4. Ascribing intelligence to AI/LLMs is not "humanizing" them

You disagree, as you're free to do, but it's clear nothing you read here will ever cause you to agree with any of those assertions.

--

> It's exactly as the typing monkeys example

For the record, the monkeys with the typewriters don't coordinate, don't strategize, don't hypothesize and test, don't try to exploit the environment, don't change the actions they take in response to results, so that doesn't sound "exactly as the typing monkeys example."

You also state that LLMs simply produced a large amount of invisible possible actions that we didn't see because they didn't result in overt action, yet: 1. somehow you know they proposed them all and discarded them despite no tangible evidence. 2. somehow the one they did pick "at random" just happened to be plausibly logical (alphabetical deletion -> start with ZZ because that's at the end of the alphabet). 3. implicitly this is supposed to starkly contrast with human intelligence, but in fact what you describe resembles Priming: subconscious activation of adjacent concepts to a stimulus that are not present in the initial response, but become more likely and accessible in subsequent responses, which implies that human intelligence also involves multiple potential paths that are otherwise hidden from which one is selected.