eskibars
Joined 11 years ago
Joined 11 years ago
After that, I suspect a lot of people already are familiar with the https://github.com/cursor/plugins/tree/main/pstack skills, but in case you aren't they can be a good starting point. The `blast-radius`, `how`, `interrogate`, and several of the `principle-` may be interesting if you haven't already looked at this.
There are some types of reviews I find models don't do a great job at: architecture reviews (things can get too complex for them to grasp), security reviews (they get cagey thinking you're a wrong-doer), UI/UX design (they're getting better, but still most models are pretty bad). There are some dedicated tools in several of these types of "models are bad at it/unwilling" areas if you find you're often finding 1 particular part of reviews difficult and pre-prompting isn't helping.
IANAL, but I've worked with a lot of lawyers in my life on software licenses on technical products and am also currently building a platform (ZeroQuarry) which uses some decompilation/disassembly tools (not for rebuilding source software, but rather to find underlying vulnerabilities).
Tools for reverse engineering have existed for a long time (Ghidra, IDA Pro, cavaj, and a bunch of other language-specific tools), which is why most commercial companies producing non-OSS executables have put in caveats that you can't "reverse engineer or decompile, decrypt, disassemble or otherwise reduce the Software to human-readable form".
Some of these tools and underlying source code/compilation mechanisms required more skill/time than others, and now with AI, just like in software development, multimedia creation and editing, etc, the barrier to entry and the skills needed to produce something have collapsed to the point that many novices can at least "get started" at virtually no cost. A lot of people on the "builder" side have reached for obfuscation to make it more difficult (a suite of tools has been available there for a long time as well), but all that does is raises the cost of decompilation, and that cost is on an exponentially decreasing cost curve: it was never really a viable defense against "advanced" actors: the only defense against them was legal guards.
To me, I think the more interesting legal conundrum we're entering isn't pure decompilation/deobfuscation/disassembly, but rather the "clean room" carve out. Several big tech companies (Compaq vs IBM, NEC vs Intel, etc) have gotten away with using 1 team to decompile a piece of code and then have that team write a spec for a clean-room team to re-implement without looking at the decompiled software. That was deemed legal over several court cases, and now it's effectively trivial as well. Or just have an AI with vision capabilities click every button in a UI, see how they work, build the spec from there, and you don't even need decompilation tools, as we're now seeing with the Adobe suite of products: https://getartcraft.com/apps and many others
I honestly can't think of a real protection for these companies/products except signed/encrypted software at the lowest level, but that only protects companies against non-interactive software (or the non-interactive components). Some people might be old enough to remember that some software used to ship with physical devices that were the physical keys used to allow encrypted software to run. Maybe some of that makes a return?
It's an approach to defensive security for software products (especially smaller companies) by acting as the security team you don't have/can't afford. It does security recon/analysis, etc via AI.
I got tired of all of the LLM labs building firewalls of "oh, nobody is allowed to do security research/find+fix vulnerabilities in their software unless they apply for special registration."
Instead of saying "oh, anything that the model determines is security research is a vulnerability threat", it does reliably enforceable analysis like "look for a TXT record in DNS the same way a SSL provider would ensure you own the property". So it can do "live" penetration tests against your infrastructure if you provide authorization
It handles things like incoming "security researcher" e-mails to cut down on the noise of nonsense vulnerability reports by acting as your security team that defends against the reports
It provides provable/signed attestation that a pentester has checked your code/live infrastructure/APIs/etc and validated them, and/or has done a check after you've remediated whatever issues that were found. It helps all 3 sides of the "company needs pentester" and "pentester" and "auditor"/"customer" to come to agreements on what's important and what's been solved
I found a bunch of vulnerabilities, which I sent to the maintainer and have now been patched, but some of the most interesting ones I found were that many of these vulnerabilities were actually crafted by (or at least reviewed by) CodeRabbit.
I think there are a lot of reasons to use AI code review tools these days, and no problem with CodeRabbit, but one of the things I've found interesting is a discussion from investors and potential customers about "why would I use a security code reviewer when I have an AI code reviewer in place already". I thought some of the examples here may be interesting for others.
It's a way to augment small/overloaded security teams. It can pentest and then generate a pentester-style PDF report for auditors and procurement, triage incoming e-mails from security researchers by then checking whether the vulnerabilities they claim actually exist and are exploitable, hook into GitHub to scan for vulnerabilities and auto-propose fixes or file GitHub issues for you.
It's free for Open Source projects, if anyone here is maintaining one
Setting the prompts and the flow with a coordinator agent directly gives a system much better capability to investigate security issues because it doesn't rely on 1-shotting things
I've reached out to a number of the Obsidian plugin maintainers for responsible disclosure to let them know about the issues and how to fix them, and what surprised me even more was that the most common response was roughly "yeah, we all know Obsidian plugins are basically unsafe when used against untrusted markdown content." I was surprised by this response as an Obsidian user with a number of plugins installed. It made me rethink how I think about plugins.
I like their new community program that attempts to identify some risks, but IMO it's just far too little. Obsidian really needs to have a sandboxed system. I've reached out to Obsidian as well to flag some of these risks and suggested a sandbox system as well, but haven't really had much progress in moving the needle, so I wanted to raise awareness here.