kikdij

This is about wild west accounting, though, when the user has no idea what they're doing.

When launching my financial business, I spent half a year learning accounting to be sure to understand everything, then I implemented my accounting software, specialized for my activity. And as luck had it, I made it be a CLI. This last few months, I've been using Codex(lately with GPT-6.1 Sol) to do the annoying work - attaching proofs to entries, fixing the dates and amounts issues, etc. It works on on a database copy, generate a script of commands using my CLI to fix and close the previous month, that I then review before executing on my prod database, sometimes asking questions to the agent about why this or how that. … And it's been flawless so far, both on my professional accounting and my personal one. Knowing a bit about what we ask the model to do can go a long way.

The worst part of that is that you can't even configure your host firewall to block it, since the OS it runs on won't be running during the boot, obviously. It has to be blocked upstream. Which means that if you just have one host at a hosting provider, you depend on them to do it (although, you may still have the option to access your BIOS settings to disable PXE, or your host provider may provide upstream firewall configuration).

What we _can_ do, though, while the OS is running is detecting that bogus large ICMP packets are sent continuously. At least, this would be early warning that a reboot may be risky without disabling PXE.

Well, that's kind of the point: she went to prison. :) (and it was "only" private fundraising). Sadly, nobody goes to jail for marketing lies.

Marketing, press releases, PR, and most of what the general audience will ever hear. Or as judges repeat way too often: "it's not illegal to lie, in most circumstances".

For those who are not familiar with S-1: this document is incredibly important. this is one of the rare times in the life of a company where it's not allowed, by law, to lie, nor to omit the truth.

The comparison with the music industry sounds a bit forced (in order to draw parallel with the switch to live as solution, I imagine). In the case of music and piracy, the demand collapsed, thus the bottom line of industry was directly hit. In the case of current creative fall, there's no sign there might be less demand. The lost jobs are, presumably, productivity gains.

Interestingly, it _is_ possible to get the exact same output from the exact same input with LLMs. For example, with llama.cpp/llama-server, you can set the temperature to 0, define a seed for the randomizer, turn off caching so it doesn't mess with the inference, and you would get the exact same result every time. But it's largely an academic distinction, because 1/ change a letter in your prompt, and it's over… and you're certainly going to want to change your prompt at some point, and 2/ model publishers have recommendation about the temperature setting, usually between 0.7 and 1, so your reproducible results… might be terrible at solving actual problems.

That being say, I really like the idea of committing our prompts. I can see a possible future in which that would be our work: write precise specs so that you can generate the program again with an other model, in an other programming language, and get the same behavior. Which sounds a hell lot like… a RFC. :)

We may be the culprit here, by the way. Maybe the only thing missing is us taking "prompt engineering" more seriously. It would be specification writing, really, or protocol design. I think it might already be possible, from an experience I had. I made Codex write a POSIX vi implementation from POSIX ex/vi specs. It took it a week-end, but I now have a POSIX vi, with only added on top multilevel undo/redo and syntax coloration, and without the baggage and instability of nvi2, I'm an happy man. I've been using it for more than a month and still have to find a fault to its POSIX implementation (and there are _a lot_ of details in POSIX vi specs). I think models are already there, when it comes to implemented behavior predictability, it's just us who are not precise enough in our specifications.

I don't think reproducibility is relevant, here. Otherwise, even your compiler example doesn't stand. No, machine code is not always the same after compiling high-level language. First, it depends on compiler. Clang and GCC won't produce the same output. Then it depends on the version of your compiler. GCC-9 and GCC-11 won't produce the same output. It obviously depends on the architecture too (arm vs x86), but even then, it depends on details about current configuration of the system you're building it on (CPU flags, hardening, etc). And then of course, there may be things randomized on purpose at each build, like randomized memory addresses to make the binary less exploitable.

His point is moot. His argument considers that natural language would be the direct expression of the program. It's not. It's an abstraction layer, that is then converted in formal language. He could have foreseen that, given he takes explicitly the example of machine code and high-level languages. High-level languages do not replace machine code, they translate into it. Just like natural language programming translates into high-level language.

Considering that "it's safe to assume everybody does X" while perfectly knowing "there are actually many people who don't, but who cares" is honestly kind of militant conformism. You can expect it without assuming.

How many times have you heard "we can't work because GitHub is down?". This has nothing to do with AI, and yes, it's commonplace.

Not to mention the "yes, there's a bug. We opened a ticket to the maintainer. Now we wait".

Saying "surely, everybody does X" is a sure way to be wrong. You're even spectacularly failing at making your own point by acknowledging in the very same comment that there are, in fact, people not doing it. Some people are different from you, deal with it.

That being said, it's true than we can assume most of new software are generated. No need for hyperbole on that. An interesting fact is that it makes grandfather's concern moot: people who are generating their code don't care much if a dependency goes unmaintained, they can have it replaced by an other one in matter of minutes.

There's an other objection to openai-cli we could raise, though: why use a generalist developer tool, containing probably hundred of features we don't need, when we could make a bespoke specialized version, perfectly fit to us, in like 25 minutes? I can get that there are audiences where this would not even come to mind to the users, but we're talking about a developer tool, here. The value of that is questionable. And ironically, those who might from benefit it the most are those who don't want to generate their code.

It _is_ complete bullshit. They're surfing the same Sci-Fi based fantasy as OpenAI and Anthropic do to push their bottom line.

OpenAI trains a model for cybersecurity, tells it it's allowed to do pentesting, then when the model does just that to fullfill the assignment it was given, OpenAI screams that "the AI escaped the sandbox", leveraging decades of AI fantasy in fiction works to make general audience react. Why? Because it pushes their bottom line. They've been at it for months, now. They see openweight and Chinese models being very close to them in the benchmarks, and want to "solve" the problem the same way banks did: through regulatory capture. It amazes me that the press is just relying their fearmongering without stopping to wonder: "wait, why are the main producers of AI warning us about how dangerous AI is?". The reason of course is that they want regulation, because regulations are a wall not many can climb. The higher the wall, the safer your moat as a first mover big company is.

And of course, Nvidia's bottom line is the exact opposite. They need as many AI companies competing as possible, so they all buy GPUs. So they insert themselves into the same narrative with the same kind of bullshit. You've all seen those movies in which the robot goes mad once their safety chip has been removed, right? Well, we're building one, problem solved! … except that "chip" is basically just a browser trying to block the AI to access what it shouldn't. How? They don't say. It would be quite comical if they used a small model to be the judge of that.