rfw300
Joined 6 years ago
Joined 6 years ago
One thing the report leaves unexplained, but is curious to me, is that the agents were able to create links on a shortening service with only GET requests? Or did they bootstrap into that by first creating a sufficiently small program on the HTML sandbox that could POST to the link shortener?
Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's post a few days ago:
> The OpenAI-Hugging Face incident (opens in a new window) was a watershed moment for cybersecurity because it gave a peek into how the capabilities of a typical threat actor will evolve in upcoming months.
I suppose if OpenAI burns someone's house down with a drone, that is a "watershed moment" for arson, too. Either way, I would hope that the people responsible would be prosecuted.
Reminded of Matt Yglesias’s excellent headline from 2010: Angela Merkel Lucky the Bar for “Worst German Leader” is Very High
In my experience, human employees are much more vulnerable to this particular weakness than frontier agents (i.e. phishing attacks).
Forcing readers to wade through an unceasing string of LLM clichés demonstrates the opposite of the point you’re trying to make—that the consumers of your work are worse off because you exercised no human judgment in creating it.
For one, nothing about this project makes "every law" a commit. It just takes the _annual_ snapshots published by the House clerk and diffs chunks of those files against each other. A project which actually traced the edits in each annual snapshot to a specific passed bill would be incredibly cool (and is probably tractable now for the first time with current AI agents). This is not that!
All this does, as far as I can tell, is parse a set of well-structured XML files into chunks and commit those chunks to Git. It's not literally nothing, but it's something that the author's own README credits multiple people doing years ago with ~100 line Python scripts.
I don't mean to be overly harsh. But this is exactly the problem with treating your software as a "factory": you release something you do not understand, in a domain you did not care to learn. And we are all the poorer for it.
Everything we do builds on top of what's already been done. When I write a new program, I'm composing a bunch of heuristics and tricks I've learned from previous programs. When a mathematician approaches an open problem, they use the tactics they've developed from their experience. When Newton derived the laws of physics, he stood on the shoulders of giants. Sure, some approaches are more or less novel, but it's a difference in degree, not kind. There's no magical firebreak to separate what AI is doing or will do, and the things the most talented humans do.