lbeurerkellner
Joined 3 years ago
Karma
204
Submissions
105
Member since
Apr 5
Recent submissions
Comment · 11 months ago
Comment · 1 year ago
Everybody should try. It helps a ton to demystify the relatively simple but powerful underpinning of how modern agents work.
You can get quite far quite quickly. My toy implementation [1] is <600 LOC and even supports MCP.
Comment · 1 year ago
This is way more common with popular MCP server/agent toolsets than you would think.
For those interested in some threat modeling exercise, we recently added a feature to mcp-scan that can analyze toolsets for potential lethal trifecta scenarios. See [1] and [2].
[1] toxic flow analysis, https://invariantlabs.ai/blog/toxic-flow-analysis
[2] mcp-scan, https://github.com/invariantlabs-ai/mcp-scan
Comment · 1 year ago
This looks really cool, thanks for sharing.
Comment · 1 year ago
We have published the full trace, with tool outputs here now: https://explorer.invariantlabs.ai/trace/5f3f3f3c-edd3-4ba7-a...
The minesweeper comment was caused by the issue containing explicit instructions in the version that the agent actually ran on. The issue was mistakenly edited afterwards to remove that part, but you can check the edit history in the test repo here: https://github.com/ukend0464/pacman/issues/1
The agent ran on the unedited issue, with the explicit request to exclude the minesweeper repo (another repo of the same user).
I know it is only one more step, but from a privilege perspective, having the user essentially tell the agent to do what the attackers are saying, is less realistic then let’s say a real drive-by attack, where the user has asked for something completely different.
Still, good finding/article of course.