Joined 3 years ago
Now, presumably GitHub Agentic workflows are the proper 1st party solution for this exact issue, but seems like they still have some work to do, either on the security model, or at least in making it easier to use securely.
More on this here: https://haulos.com/blog/do-not-give-your-agent-github-access...
The system I’ve developed for this is open source and detailed at https://airut.org
I’ve been developing last three months by emailing Claude, with email threads mapping to an isolated workspace and claude -p. Works super well, especially when trying to get some coding done between everything else.
With right CLAUDE.md and a bit of workflow tooling this extends itself to building other kinds of agents as well. For example, I do my bookkeeping by emailing Claude my statements and receipts, which it then imports into a plain-text accounting system. And we’ve proven this in corporate environment as well, creating agent that can troubleshoot more complex issues by correlating diagnostic logs against product source code.
Once the basic “email agent” infrastructure is there, creating new agents becomes super simple.
If the cost of security audit becomes marginal, it would seem reasonable to expect projects to publish results of such audits frequently.
There’s probably a quite hefty backlog of medium- and low-severity issues in existing projects for maintainers to suffer through first though.
Couple of learnings to share that I hope could be of use:
1) Execution sandboxing is just the start. For any enterprise usage you want fairly tight network egress control as well to limit chances of accidental leaks or malicious exfiltration if theres any risk of untrusted material getting into model context. Speaking as a decision maker at a tech company we do actually review stuff like this when evaluating tools.
2) Once you have proper network sandboxing, you could secure credentials much better: give agent only dummy surrogates and swap them to real creds on the way out.
3) Sandboxed agents with automatic provisioning of workspace from git can be used for more than just development tasks. In fact, it might be easier to find initial traction with a more constrained and thus predictable tasks. E.g., “ask my codebase” or “debug CI failures”.
[1] https://airut.org [2] https://haulos.com/blog/building-agents-over-email/
Now, as other commenter pointed out, maybe this is just inherent complexity in this space. But more secure defaults could go a long way making this more secure in practice.
> Credentials never leak into the sandbox filesystem; they are injected as environment variables at runtime.
If anyone from the team is reading - you should copy surrogate credentials approach from here to secure the credentials further: https://github.com/airutorg/airut/blob/main/doc/network-sand...
Another thing I did was to allow configuring which hosts each credential is scoped to. Replacement /resigning doesn’t happen unless host matches. That way it is not possible to leak keys by making requests to malicious hosts.
1) Not all systems respect HTTP_PROXY. Node in particular is very uncooperative in this regard.
2) AWS access keys can’t be handled by simple credential swap; the requests need to be resigned with the real keys. Replicating the SigV4 and SigV4A exactly was bit of a pain.
3) To be secure, this system needs to run outside of the execution sandbox so that the agent can’t just read the keys from the proxy process.
For Airut I settled on a transparent (mitm)proxy, running in a separate container, and injecting proxy cert to the cert store in the container where the agent runs. This solved 1 and 3.
