scared_together

The article starts that section with “As the Big Bang has so loudly illustrated” so I assume the author is imagining a machine that replicates the conditions of the Big Bang. Exactly how that could be constructed and used safely is left as an exercise for an AI.

That kind of event happens sometimes.

The first reason it’s not more common is that wealth isn’t so static. Landlords have to collect rents, oligarchs have to skim off the top of their corrupt enterprises etc. A guard who kills the landlord, then manages to organize rent collection for the benefit of other guards, has essentially become the new landlord.

The second reason is that a guard doesn’t just have to kill the landlord, a guard potentially has to kill all the other guards who want all the landlord’s wealth for themselves.

Since rent has to be collected anyway, and an individual guard has a low chance of surviving against all other guards, remaining a guard is a lower risk option.

> The end of software secrecy is almost upon us. The death of closed source, and the final triumph of open source.

It’s a Pyrrhic victory given the negative impacts of LLMs on open source projects.

Everybody’s closed source applications can be reverse-engineered, but due to 10,000 line “contributions” from LLMs people won’t want to create the free/open source communities that used to exist.

So what’s the point of keeping the source open? Stallman’s original problem of fixing a printer driver would be resolved today by getting an LLM to read the binaries.

The LLMs can make the binaries human-readable, but also make human readability a secondary concern. The phrases “closed source” and “open source” have lost meaning.

I think you meant to respond to bergheim not to me.

> I, too, hope that one of the two or three companies will be able to improve and displace everything. Who needs Linux anyway.

In the context of browsers the domination of one or two companies is already complete, and being able to generate new browsers with AI would hopefully allow people to escape the stranglehold of Chrome/Blink.

That said, I doubt a new,competitive browser could be created that way. The (possibly AI) developers of web apps won’t want to test on somebody’s bespoke AI-generated browser.

Open source models: trilobites

ChatGPT: Hallucigenia

Claude: Haikouichthys

????: Anomalocaris

Hand-written code: Ediacaran biota :(

> This kind of statement is very difficult to build discussions upon (not a criticism). "Vast majority", "intelligent enough" are both very difficult to quantify and there is too much subjectivity involved.

Then let me provide a stronger and more detailed version of my claim.

The article said:

> There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species.

In order for this claim to make any sense, there must be TWO “intelligent” species. One species capable of taking over another species’ future, and a second species which is even more capable than the first species but willingly surrenders control.

But there is only one species which can take such roles: humans. Every other species is not even capable of controlling another species fate AND every other species does not look out for their species-wide future.

Some people might point to whales, elephants, cats, non-Sapiens hominids, or even those ants that harvest aphids. But the whales, elephants and cats lack the capacity to control entire other species, even if they can manipulate individuals of other species. The non-Sapiens hominids are not available for comment: we’ll get to them later. And the ants do not think of their future.

So even though there are a massive number of species on earth, none have the prerequisites to even apply to the above statement.

> The absence of such relationships given here are so many species is exactly the point being made by the author

If we lived in a world where there were a vast array of species capable of controlling other species and capable of thinking of their future, all of whom fiercely defended their own autonomy, your point would be correct. But most species are not capable of either, even if would be a good idea. Therefore we cannot conclude that such a surrender of control would be a good idea or bad idea based on the statement. The statement is moot.

We might as well say the author’s observation is correct because sand doesn’t control volcanoes. In terms of the ability to control other species, other species on earth are no better than sand or volcanoes.

> there is no data to show that it would be safe, let along good, for us to surrender decision making to some more intelligent entity.

There is no evidence against either because the observation lacks any examples other than humans and AI.

Truth be told, I agree with the conclusion. But we have to be honest with each other in arguing for our survival and autonomy, even if it means admitting that we are scared and irrational, instead of trying to fake objectivity and universality. The article starts with a bad argument in favour of a conclusion I agree with.

If I were to write an article about the risks of super-intelligence, I would not even attempt to construct such a “universal” rule - it would have very little bearing on the current situation. I would simply point to the bones of the rest of the Homo genus, and say “we’re next”.

I’m not an American, but here in Canada there are a lot of extra fees and such where I live as well.

I don’t believe the problem is purely innumeracy on the consumer’s part.

If companies clearly stated “A(B + C)” as a price, even people with poor math skills would understand that the price is not “B”.

But it’s more like the price is advertised as “B” and “A(B+C)” only appears in fine print or seconds before payment.

Your comment says you can “hide a business opportunity”. I would instead call that “being dishonest”.

The statement that the set has 1 member is my criticism of the text, not what the text is trying to say.

> given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation

I would expect that in the vast majority of combinations, those n species are not intelligent enough to make decisions for another species, making the statement nearly useless.

Maybe another AI could use the proof.

> OBSERVATION: There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species.

Does the set of intelligent species only have one member? I cannot treat this observation as a general rule if there is literally one example of an intelligent species to theorize about.

Not everyone would actually send the prompt to an LLM. Some would have the opportunity to see the prompt as its own artifact, devoid of LLM-generated “hallucinations”.

> let the votes speak for that.

The comment you are replying to got a fair number of votes too… Having somebody run a Pangram check saves n people the trouble of doing the same.

Are you referring to the Pangram link? I’d consider that akin to a reference supporting the comment’s point, not a substitute for the comment’s point.

> Most people do not, in fact, want to read raw prompts.

I wonder if this is really true, and if it will remain true for long. Have you observed someone read another person’s raw prompt? Or observed someone submit both their prompts and their LLM output for review?

Personally I’d be curious about the prompts for a lot of top HN articles which are LLM-generated. It would say a lot more about the human operator’s intent and thinking process compared to the LLM output.

A coworker who spoke English as a second language once screen shared their Claude session during a code review, and it was interesting that their prompts were all in their native language. The guy wasn’t writing an article, and I didn’t understand the prompt anyway, but I still found it interesting as a glimpse into how he uses LLMs.

For this particular article, if the prompt was initially a series of bullet points that wouldn’t be so bad. If there were multiple prompts spent editing and rearranging the article that would be an unusual work by pre-LLM standards. However I’d suggest that instead of dismissing the idea, we could embrace “raw prompts” as a kind of new medium, to cross the divide between pro-LLM and anti-LLM readers.