eskibars

Joined 11 years ago

Karma
379
Submissions
115
Member since
Dec 28
https://zeroquarry.com
View on Hacker News

Recent submissions

Comment · 13 days ago
I've been building a cybersecurity platform (called ZeroQuarry) and regularly benchmark new models against a pretty wide set of security both code review tasks and live application pen testing tasks, and this one far underperforms relative to GLM 5.3 from every test I've done for real-world cybersecurity tasks
Comment · 19 days ago
I think you need to catch a lot of it before it becomes a PR. Spend a lot of time teaching the agents how you code in your prompts, AGENTS.md files, etc. Make sure they know what your desired architecture is, how you want them to talk to you, how to identify and speak to their perceived blast radius, what kind of documentation you like, and so on. So when something an agent wrote does come to your desk for review, you can optimize your time/impact. I find it's a lot harder to get an understanding after the fact.

After that, I suspect a lot of people already are familiar with the https://github.com/cursor/plugins/tree/main/pstack skills, but in case you aren't they can be a good starting point. The `blast-radius`, `how`, `interrogate`, and several of the `principle-` may be interesting if you haven't already looked at this.

There are some types of reviews I find models don't do a great job at: architecture reviews (things can get too complex for them to grasp), security reviews (they get cagey thinking you're a wrong-doer), UI/UX design (they're getting better, but still most models are pretty bad). There are some dedicated tools in several of these types of "models are bad at it/unwilling" areas if you find you're often finding 1 particular part of reviews difficult and pre-prompting isn't helping.

Comment · 21 days ago
I don't think anything new here is actually all that novel: the only thing that's changed is there used to be more of a skills and time gap than there is now.

IANAL, but I've worked with a lot of lawyers in my life on software licenses on technical products and am also currently building a platform (ZeroQuarry) which uses some decompilation/disassembly tools (not for rebuilding source software, but rather to find underlying vulnerabilities).

Tools for reverse engineering have existed for a long time (Ghidra, IDA Pro, cavaj, and a bunch of other language-specific tools), which is why most commercial companies producing non-OSS executables have put in caveats that you can't "reverse engineer or decompile, decrypt, disassemble or otherwise reduce the Software to human-readable form".

Some of these tools and underlying source code/compilation mechanisms required more skill/time than others, and now with AI, just like in software development, multimedia creation and editing, etc, the barrier to entry and the skills needed to produce something have collapsed to the point that many novices can at least "get started" at virtually no cost. A lot of people on the "builder" side have reached for obfuscation to make it more difficult (a suite of tools has been available there for a long time as well), but all that does is raises the cost of decompilation, and that cost is on an exponentially decreasing cost curve: it was never really a viable defense against "advanced" actors: the only defense against them was legal guards.

To me, I think the more interesting legal conundrum we're entering isn't pure decompilation/deobfuscation/disassembly, but rather the "clean room" carve out. Several big tech companies (Compaq vs IBM, NEC vs Intel, etc) have gotten away with using 1 team to decompile a piece of code and then have that team write a spec for a clean-room team to re-implement without looking at the decompiled software. That was deemed legal over several court cases, and now it's effectively trivial as well. Or just have an AI with vision capabilities click every button in a UI, see how they work, build the spec from there, and you don't even need decompilation tools, as we're now seeing with the Adobe suite of products: https://getartcraft.com/apps and many others

I honestly can't think of a real protection for these companies/products except signed/encrypted software at the lowest level, but that only protects companies against non-interactive software (or the non-interactive components). Some people might be old enough to remember that some software used to ship with physical devices that were the physical keys used to allow encrypted software to run. Maybe some of that makes a return?

Comment · 1 month ago
https://zeroquarry.com/

It's an approach to defensive security for software products (especially smaller companies) by acting as the security team you don't have/can't afford. It does security recon/analysis, etc via AI.

I got tired of all of the LLM labs building firewalls of "oh, nobody is allowed to do security research/find+fix vulnerabilities in their software unless they apply for special registration."

Instead of saying "oh, anything that the model determines is security research is a vulnerability threat", it does reliably enforceable analysis like "look for a TXT record in DNS the same way a SSL provider would ensure you own the property". So it can do "live" penetration tests against your infrastructure if you provide authorization

It handles things like incoming "security researcher" e-mails to cut down on the noise of nonsense vulnerability reports by acting as your security team that defends against the reports

It provides provable/signed attestation that a pentester has checked your code/live infrastructure/APIs/etc and validated them, and/or has done a check after you've remediated whatever issues that were found. It helps all 3 sides of the "company needs pentester" and "pentester" and "auditor"/"customer" to come to agreements on what's important and what's been solved

Comment · 2 months ago
Author here. I built ZeroQuarry, and I was considering deploying an open source link shortener or using a SaaS one. I googled around and then found iShortn, so ran the iShortn scan with it.

I found a bunch of vulnerabilities, which I sent to the maintainer and have now been patched, but some of the most interesting ones I found were that many of these vulnerabilities were actually crafted by (or at least reviewed by) CodeRabbit.

I think there are a lot of reasons to use AI code review tools these days, and no problem with CodeRabbit, but one of the things I've found interesting is a discussion from investors and potential customers about "why would I use a security code reviewer when I have an AI code reviewer in place already". I thought some of the examples here may be interesting for others.

Comment · 3 months ago
Yeah, I think so. It's running on a pretty small VPS and I never implemented any caching. Should have thought about that before posting I guess. I see the CPU is currently pegged. I was able to get it to load at least 1 trip myself though, so maybe retry in a bit
Comment · 3 months ago
Also, a "just for fun" project: https://drifttrip.connelly.casa/ . I was always enthralled with the idea of taking a virtual road trip and then loading the local council's tourism promo videos at each "stop"
Comment · 3 months ago
Building http://zeroquarry.com

It's a way to augment small/overloaded security teams. It can pentest and then generate a pentester-style PDF report for auditors and procurement, triage incoming e-mails from security researchers by then checking whether the vulnerabilities they claim actually exist and are exploitable, hook into GitHub to scan for vulnerabilities and auto-propose fixes or file GitHub issues for you.

It's free for Open Source projects, if anyone here is maintaining one

Comment · 5 months ago
What we've actually seen is a couple things that make this impractical "to just share a prompt". First, that nearly every major model still hallucinates a lot of vulnerabilities. Especially with temperature=0.7 as states in the original blog here, you get very inconsistent results regardless of the prompt, but that's almost kind of moot to the bigger picture. What you really need is to override the planning phase beyond asking a model "find the vulnerabilities" and you need to add another 1+ checking phases for "validate these vulnerabilities." Without that, even with the absolute best models with the highest levels of thinking enabled, you end up with garbage.

Setting the prompts and the flow with a coordinator agent directly gives a system much better capability to investigate security issues because it doesn't rely on 1-shotting things

Comment · 5 months ago
I've been building a product (https://zeroquarry.com) that can use a variety of models for finding vulnerabilities. One of the things I've noticed is that the models will nearly always comply with some of this, but how you prompt it matters a ton. I've worked on a set of prompts and approaches which rarely get flagged
Comment · 5 months ago
I've been a long-timer Obsidian user with a number of plugins. Recently I launched ZeroQuarry (a product to scan code for security vulnerabilities) and pointed it at a number of Obsidian plugins. I was initially surprised to find out that so many of them had RCEs baked in: that if you open a malicious .md file, you could inadvertently run untrusted code.

I've reached out to a number of the Obsidian plugin maintainers for responsible disclosure to let them know about the issues and how to fix them, and what surprised me even more was that the most common response was roughly "yeah, we all know Obsidian plugins are basically unsafe when used against untrusted markdown content." I was surprised by this response as an Obsidian user with a number of plugins installed. It made me rethink how I think about plugins.

I like their new community program that attempts to identify some risks, but IMO it's just far too little. Obsidian really needs to have a sandboxed system. I've reached out to Obsidian as well to flag some of these risks and suggested a sandbox system as well, but haven't really had much progress in moving the needle, so I wanted to raise awareness here.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton