LeCompteSftware

I agree that de facto the biggest security flaw in Linux is "okay I'm tired of getting interrupted all day assisting you, I know you're competent, I'll put you on the sudoers list."

But there are a lot of academic and research institutions that actually do have good Linux user management. I worked at a pediatric hospital, and the RHEL HPC admins did not mess around in terms of who was allowed to access which patients' data. As someone who was not an admin, it was a huge pain and it should have been. So this bug has pretty serious implications, seems like anyone at that hospital can abscond with a lot of deidentified data. [research HPC not as sensitive as the clinical stuff, which I think was all Windows Server]

FWIW I think "LLMs are semideterministic" is something of a red herring. The real difference between LLM codegen and compilers is that compilers output logically the same assembly regardless of the variable names. If you're numerically solving a differential equation the compiler does not care if the floats represent heat through a pipe or dollars through a brokerage. Compilers don't care about semantic meaning, that concern is totally separated.

But even if its putatively implementing the same algorithm, LLMs certainly do not output basically the same finance Python as they would mechanical engineering Python. The style will be a little different. Sometimes the performance/clarity tradeoffs will be different. Sometimes it'll be fairly fancy and object-oriented, other times it'll be more low-level "objects are just dicts."

It's way more than a higher abstraction layer: LLM codegen involves a nontechnical tangling of concerns that doesn't exist with even the hoitiest-toitiest proof-checking compilers. It's a complete sea change. I find it incredibly disconcerting... for the same reason, by the way, that assembly programmers found Fortran and C disconcerting, and continued to reliably find employment for a good 40 years after higher-level languages were invented :) Actually even today. The assembly programmers who got hosed by C tended to be electricians who learned on the job - it's kind of cool to read old manuals from the 70s, carefully (and correctly!) explaining to electricians that a computer program is essentially an ephemeral circuit.

But I think there are specific skills around scientific thinking (learned at a formal college) and engineering carefulness (learned via hard knocks) that aren't going anywhere.

Surely the biggest difference is that you guys are mostly testing LLMs on simpler utilities, mostly involving higher-level languages, whereas ProgramBench are all very complex C programs (and much older programs with much more comprehensive test cases).

Eg cal is totally routine. I would expect most sophomores to be able to write a perfectly good cal. In fact the only program you tested which actually has anywhere close to the complexity of SQLite or FFmpeg is is Pkl, and it looks like Opus 4.6 totally failed.

I think your results are consistent. You're just measuring different things. Your benchmarks mostly tests LLMs ability to write technically routine programs of moderate length - yes the bioinformatics package involves specialized domain knowledge, but not specialized Go engineering. ProgramBench is harder.

If I invented a machine that makes chimpanzee noises in response to input chimpanzee noise, put it in front of a chimpanzee, and watched the chimp coo and yell and screech and purr in response to the machine, I would not conclude "wow, I emulated a chimpanzee's consciousness!" I would say "huh, I made a device that's good at tricking chimpanzees."

My belief is that the Turing test (and LLMs in particular) are not categorically different. Language is a tiny part of the human brain because it's a tiny part of human cognition, despite its outsized impact socially.

"Haw haw, you couldn't tell it was written with AI!"

"Oh! That explains it!"

"Uh..."

"I didn't want to say anything rude, but the whole time I was like 'yikes, how did this idiot become a professor at Notre Dame?'"

"Actually -"

"Heh! You got me good! Of course it was written with AI. Duh. These ideas are so vacuous and shallow, there's no way a fancy professor like you-"

"Actually, I asked the AI to write an essay summarizing my arguments from social media."

"...oh."

"..."

"...hey you should log off BlueSky, it's not healthy there."

This knee-jerk cynicism is badly undermined two sentences later:

> Researchers say that climate models may need to be updated to account for the warming effect of plastic, but the new study is far from conclusive.

So it's not scientific make-work, they are looking into whether climate models are missing something. That seems important. Perhaps local effects in India are more severe than "a fraction of the impact of soil" - India produces a huge amount of new plastic while also scavenging and recycling international plastic imports, all with very poor oversight and corrupt regulation.

Darkly funny that Armstrong's Twitter bio still reads "Creating more economic freedom in the world" when he has relegated humans to "the edge" of his own organization in favor of the pseudointelligent pseudogod.

Freedom for who, exactly? Coinbase's executives, I suppose.

But there would be no basis to claim this trademark was abandoned (even before Don Ho responded to infringement). Notepad++ is famous software actively getting new features and new releases. It is well-known among technically sophisticated Windows users in the US, and until this kerfuffle Don Ho's ownership of the name was never seriously contested in OSS circles. Nobody could reasonably claim this trademark is stale or generic.

This is both unethical and completely useless at the (supposed) goal of "show[ing] the current capabilities of AI." What a completely garbage case study! And what a dishonest writeup:

  We see that frontier models are intelligent enough to manage humans

Really? The manager who asked baristas to pay for things with their personal credit card is "intelligent enough to manage humans"? The manager who asked workers to put raw eggs in a high-speed oven? The one who makes such bad decisions that the workers made a Wall of Shame about those decisions?

And this is just egregious:

  Despite the learning curve, the café is working. In the first two weeks of operation, Andon Café has brought in 44,000 SEK in sales. Mona’s inbox has been flooded with messages from customers asking questions or pitching different business proposals. In one case, a customer emailed wanting to prepay for 300 coffees to give away. Mona negotiated a deal where he paid 9,000 SEK in exchange for 300 QR codes that people could redeem for a free coffee. In another case, a startup paid her 3,000 SEK to rename a pastry after them for three months.

This only demonstrates that viciously stupid AI stunts can go viral, even in otherwise decent countries like Sweden. How stupid does Andon Labs think we are to take this as a sign of AI management success? None of this reflects normal cafe operations. It reflects the Stockholm tech scene checking out the gimmicky AI cafe.

   By running this experiment, we shift the discussion of how we want this future to look earlier in time, so we can better prepare.

Better prepare for what? Evil AI labs running experiments without any ethical oversight? Shockingly evil, by the way:

  no one’s livelihood depends on the judgment of an AI alone.

"Alone." How kind of them. By the way, it is incredibly despicable, even by the low low low standards of AI researchers, to run this sort of experiment on people looking for work. I couldn't believe the humans responsible let their stupid AI post ads on Indeed and LinkedIn. What scumbags.