simple10
Joined 16 years ago
Joined 16 years ago
My typical session ranges from $100-$400 - higher end when using workflows with lots of subagents. $100/session is expected when using the API without the subsidized subscription pricing. Most larger orgs have to use API pricing AFAIK.
https://github.com/simple10/agent-super-spy - llm proxy + http MiTM proxy + LLMetry + other goodies
https://github.com/simple10/agents-observe - fancier claude hooks dashboard
It started as a need to keep an eye on OpenClaw but is incredibly useful for really understanding any agent harness at the raw LLM request level.
For observability, would be amazing to have session replay or at least session exploration built in. Kinda like git history but tied to tasks and tool use instead of file diffs.
[Edit] When claude spawns sub-agents, they inherit the parent's hooks. So all sub-agents activity gets logged by default.
Node has a 30-50ms cold start overhead. Then there's overhead in the hook script to read local config files, make http request to server, and check for callbacks. In practice, this was about 50-60ms per hook.
The background hook shim reduces latency to around 3-5ms (10x improvement). It was noticeable when using agent teams with 5+ sub-agents running in parallel.
But the real speed up was disabling all the other plugins I had been collecting. It piles up fast and is easy for me to forget what's installed globally.
I've also started periodically asking claude to analyze it's prompts to look for conflicts. It's shockingly common for plugins and skills to end up with contradictory instructions. Opus works around it just fine, but it's unnecessary overhead for every turn.
It's super important to check your plugins or use a proxy to inspect raw prompts. If you have a lot of skills and plugins installed, you'll burn through tokens 5-10x faster than normal.
Also have claude use sub-agents and agent teams. They're significantly lighter on token usage when they're spawned with fresh context windows. You can see in Agents Observe dashboard exactly what prompt and response claude is using for spawning sub-agents.
Basic flow:
1. Plugin registers hooks that call a dump pipe script that sends hook events data to api server
2. Server parses events and stores them in sqlite by session and agent id - mostly just stores data, minimal processing
3. Dashboard UI uses websockets to get real-time events from the server
4. UI does most of the heavy lifting by parsing events, grouping by agent / sub-agent, extracting out tool calls to dynamically create filters, etc.
It took a lot of iterations to keep things simple and performant.
You can easily modify the app/client UI code to fully customize the dashboard. The API app/server is intentionally unopinionated about how events will be rendered. This was by design to add support for other agent events soon.
The next big layer for my personal stack is full orchestration. Something like Paperclip but much more specialized for my use cases.
https://twitter.com/joejohnston