dolftax
Joined 11 years ago
Karma
1,301
Submissions
199
Member since
Jan 12
Recent submissions
Comment · 10 months ago
All three covered — TypeScript, Rust, and SQL[1].
Comment · 10 months ago
We flag performance issues today alongside security and code quality. We're working on respecting AGENTS.md, detecting code complexity (AI generated code tends toward verbose, tangled logic), and letting users/teams define custom coding guidelines.
Comment · 10 months ago
On the OpenSSF CVE Benchmark[1], Semgrep CE hits 56.97% accuracy vs our 81.21%, and nearly 3x higher recall (75.61% vs 26.83%).
On when to run it, fair point. Autofix Bot is currently meant for local use (TUI, Claude Code plugin, MCP). We're integrating this pipeline into DeepSource[2], which will have inline comments in pull requests, that fits the QA/pre-merge flow you're describing.
That said, if you're using AI agents to write code, running it at checkpoints locally keeps feedback tight.
Thanks for the feedback!
[1] https://github.com/ossf-cve-benchmark/ossf-cve-benchmark
Comment · 10 months ago
The structured vs open-ended distinction here applies to code review too. When you ask an LLM to "find issues in this code", it'll happily find something to say, even if the code is fine. And when there are actual security vulnerabilities, it often gets distracted by style nitpicks and misses the real issues.
Static analysis has the opposite problem - very structured, deterministic, but limited to predefined patterns and overwhelms you in false positives.
The sweet spot seems to be to give structure to what the LLM should look for, rather than letting it roam free on an open-ended "review this" prompt.
We built Autofix Bot[1] around this idea.
[1] https://autofix.bot (disclosure: founder)
Comment · 1 year ago
Jai here, from Autofix Bot team. We've published results of the initial benchmark run[1] comparing Gitleaks, detect-secrets and trufflehog ~3 weeks ago. In the meantime, we've put together a significantly improved dataset, and we're planning to rerun those benchmarks shortly; will include Kingfisher to the list, and share the results here.
Btw, we use Kingfisher's validation system internally for generating request/expected_response pairs for a given secret, as the last step of the pipeline. We don't run/call the validation queries ourselves, due to rate limit issues. But, we add this information in a structured format as part of the response which can be executed on the client side (or) by the user who is integrating via the API. Thanks for building it :)
[1] https://autofix.bot/benchmarks/#benchmarks-secrets-detection
Comment · 5 years ago
DeepSource | Bangalore & San Francisco | Fulltime | https://deepsource.io/
DeepSource is a fast and reliable static analysis platform for developers and engineering teams. We've various roles open across Platform Engineering, Language Engineering and Marketing - https://careers.deepsource.io/
https://x.com/dolftax