rubendev

I think the solution is leaning on static types and hard to abuse abstractions to make programs correct by construction. It does mean that you’ll still have to use your brain to design the internal code architecture, and just use LLMs to fill in the boring bits between the safe abstractions.

It would still be running on that original laptop, no? So what did it replicate then? It can only possibly run on multiple thousands of dollars of compute and memory, and certainly not on an IoT device. The idea that AI is going to spread by itself is science fiction until compute and memory are so cheap and power efficient that my fridge can run it.

Yes, you need to apply output encoding or sanitization at the place where it is being combined with another string. Otherwise you don’t know the encoding which is needed. Even for HTML you cannot do it on the backend, because you don’t know if it will be injected into HTML PCDATA context (tags) or in HTML attribute context.

It is genuinely exhausting to rebuke the content of walls of LLM slop though.

A lot of this frustration comes from the fact that it takes a lot less effort to generate a wall of text than to actively engage with and rebuke the points that are made in them.

Don’t expect respect from your readers if you don’t respect them enough to formulate your own words.

That's not what the LLM hacking accidents have been like at all though. To improve the analogy, it would be like summoning a totally passive demon, giving it weapons and placing it next to a bank, place a small fence around it, and then command it to perform a totally safe "exercise" that is exactly like robbing a real bank.

LLMs do not have agency, they are just producing tokens based on a prompt that a person entered, and some of these tokens can trigger the tools that a person gave them access to.

I don’t think we should just accept that the average developer doesn’t know about these very basic hardening methods.

This is your chance to set yourself apart from LLM coding agents. If you want to call yourself a software “engineer” you need to start actually engineering, which includes knowing when and how to apply security principles and these hardening methods.

I reject the notion that “doing cybersecurity” is somehow a different job than software development.

I agree wholeheartedly. The solution is not to stop developing these so called “dangerous” AI models. The solution is to start properly engineering software.

I think the model was able to escape the sandbox and hack huggingface because they were incompetent or not giving enough priority to implementing basic cybersecurity principles.

If they would have done so, there wouldn’t have been an escape or a hack. The reason we don’t get much details is because the details are embarrassing for them.

No we just need developers to do the bare minimum of effort to write secure software. Most hacks are not super complicated vulnerabilities chained together, but just utter failures where authentication and authorization was simply forgotten or untested, or where nobody bothered to validate the data they receive.

The bar for software is so low that it is embarrassing for the entire profession.

I think the main problem with this is that you can be paid/coerced to vote for someone and be able to easily prove that you voted for the person they wanted. It’s easier to vote for the person you actually wanted to vote for, and maintain plausible deniability, if you go in person to a voting booth.

I can’t tell if the model card is written by AI or a parody of AI writing…

I think at minimum you would need to understand which theorems you want to prove about your code, and how to express those in Lean. Otherwise you won’t be able to verify the output. It may have proven some statement that is machine checked to be correct, but it’s pointless if you don’t understand what that statement means and if it covers what you want to verify about your code.

This is really a self inflicted problem. If you host your backend on the same origin as your frontend (using a reverse proxy) you don’t need CORS at all and you can use the vanilla SOP, and strengthen it further with a strict CSP.

What is your definition of faster to implement? Is it producing a plausible implementation, or is it faster at producing a correct and high quality implementation? Are you including time spent refactoring and fixing bugs in your metrics? If not, I think you are tracking a gut feeling rather than cold hard facts. I’m not saying this is easy to track, just saying that it’s hard to know for sure that you are really more productive with AI.

With a capable static analyzer that is not true. In many common cases they can deduce the possible ranges of values based on branching checks along the data flow path, and if that range falls within the buffer then it does not report it.