angarrido

must people think it’s just GPU cost. In practice it’s coordination: model latency variance + queueing + retries under load. You don’t scale linearly, you get cascading slowdowns.

I built this because I couldn’t use Copilot or Claude at work without risking exposing internal code.

This runs fully local (Gemma 4 26B), indexes your codebase, and answers questions about it without anything leaving your machine.

Still early, but works well on large projects. Curious where this breaks for others.

Local inference is getting solved pretty quickly.

What still seems unsolved is how to safely use it on real private systems (large codebases, internal tools, etc) where you can’t risk leaking context even accidentally.

In our experience that constraint changes the problem much more than the choice of runtime or SDK.

Awesome idea, usually LLMs lack of creativity, so layouts look mostly the same, I could give a try