gravypod
Joined 12 years ago
Joined 12 years ago
I am not a lawyer but I don't know if the first amendment allows you to block access from others to things. Some things similar to this are covered in this ACLU blog post: https://www.aclu-il.org/news/when-else-can-government-regula....
1. We can never know if we have enough safe guards
2. Banking systems are already "insecure" by some definition of this.
3. Because of this insecurity a lot of government regulations were passed on logging transactions, being able to roll them back, etc.
We've seen these trigger I'm financial markets: https://www.investor.gov/introduction-investing/investing-ba...
If you've made large purchases before or strange purchases before you have likely hit similar consumer side things.
The network banks talk on is called swift. The way it works is, as I understand it, if you say move money from account 1 to account 2 it just happens so if I knew your account number I could drain all funds. Because of this, there are systems that block certain transactions. Could ai hack around them? Maybe, but they are undocumented, battled hardened over decades, and even if they did the bank could roll back the transactions.
This statement portrays a fundamental misunderstanding of how the infrastructure which powers these systems work. Note: I am not saying there are no risks, I am just saying the risk you are focusing on is the least likely one of all I have seen people be upset by.
Far higher risks one could outline are:
1. Network-connected PLCs for big infrastructure (drinking water, sewage, power, etc) being tampered with.
2. Extremely persistent malware tailored for every permutation of hardware + software.
3. Cyber criminals improve in technical capabilities (phishing sites, scam calling, propaganda campaigns, etc).
But, the cork is out of the bottle on this one. With even basic models you can begin a loop of training specialized models on low cost hardware which can be used to do specific hacking tasks.
I don't know what the best antidote to this is but I doubt that it will be in limiting access to OSS models to people in the USA as all of the threats listed above come from *outside actors*.
> On Sept. 17, NIST’s Center for AI Standards and Innovation (CAISI) published its own assessment of GLM-5.3’s cyber capabilities. CAISI found that GLM-5.3 is “the most cyber-capable open-weight model released to date” and that it lags the US frontier by about four months on an aggregate of CAISI’s cyber benchmarks.
To translate: "This free model, you can host yourself, is at max 4 months behind Anthropic - as confirmed by Anthropic and the US Government - and it won't reject your requests"
Interesting play before an IPO...
1. Track code coverage from all current test cases in your big 8hr runs.
2. Index the coverage so it goes testcase name -> methods touched.
3. On every commit run an agent which looks at the diffs, tries to predict which tests the code being edited is touching, and run just those tests (or a random subset if there are too many).
Once you have this you can also automatically kick off builds every 8hr that contain all the previously submitted patches. Once that is done, if a test begins to fail, it can automatically bisect the history to find change, and notify the author.
You could also get these benefits with a Bazel like build system which can cache test executions so others don't need to rerun them if the binary is not changed.
I think if I fully understand everything the agent is going to do and trivially verify the output quality, LLMs are a win. For one off prototypes, the same is true. For software which is unique, complex, and performing a task which has not been fully specified I find myself needing to drop into my editor more and more.
Recently I've been moved to a research focused team and I really need to have proof that something is happening. I've had Claude Opus 5 gaslight me by telling me it did something and when I read the code it obviously did not. This happens more and more with my tasks that are kind of complex.
I've found a lot of the claims by AI people have been 6mo - 2 years "ahead" of my experience. I think we are now in an era where harness engineering is highly valuable (making a test, looping an agent, manually annealing with new ideas) but the claims that no one writes code manually seems like it may not be fully there for all code.
In the past I have:
I know: C, Java, Python, PHP, and GolangI love using: Docker, Kubernetes, Bazel, MongoDB
${username}@${username}.com & admin@${first_name}${last_name}.me
meet.hn/city/40.7709499,-74.2264601/Orange