I kept seeing people all over social media talking about how they were making custom local-LLM systems to help reduce the token load injected into their context window.
On the side my recent project has me looking at Hyperdimensional Distributed Memory. I couldn't help but wonder if I could make an MCP server to improve token usage. Sure enough, using a combination of HDC + TOOL format, I was able to get opty's own self audit down by 93% in token usage.
Still experimenting with large codebases but feel pretty good about how this should drive overall token usage down.
On the side my recent project has me looking at Hyperdimensional Distributed Memory. I couldn't help but wonder if I could make an MCP server to improve token usage. Sure enough, using a combination of HDC + TOOL format, I was able to get opty's own self audit down by 93% in token usage.
Still experimenting with large codebases but feel pretty good about how this should drive overall token usage down.
Happy to hear any feedback.