bholagabbar

Spent this weekend building a one-file OpenRouter client — figured I'd open-source it.

We deal with a lot of dynamic structured data, where different LLM calls have different priorities — speed, latency, cost. After a year using OpenRouter heavily in production, I hit a bunch of real-world problems — bad providers, unreliable fallbacks, weird edge cases with schemas — so I baked all of that into this. We needed strong guarantees on output and simple, reliable handling around it.

The primary motivations:

Explicit structure enforcement (Zod + JSON Schema)

Fine-tune per call — optimize for speed, latency, or price

Ignore bad providers (beyond account settings)

Fallback cleanly to JSON mode (180+ models)

Add new models declaratively, no client changes needed

It handles:

Strict structured outputs (zod-to-json-schema)

Fallback to JSON if needed

Provider sorting and fallback models

Unified one-line abstraction (makeLLMCall)

Real error handling and dynamic model swapping

We run over 50k OpenRouter requests/day through this in production. Tried to make it stupid simple to read and extend - coming up with the best API abstractions I could.

(On a personal note, as an overworked founder, it felt great to step away from growth and customer support for a bit and spend a day building clean APIs again — reminded me of the simpler days at CMU, grinding through Joshua Bloch’s API design classes. Honestly feels like this could make a good assignment problem.)

Wize is an AI-powered platform that lets you chat directly with your favorite podcasts. It dives into thousands of expert interviews, extracting context-aware, insightful responses to user queries.

Spent a few sleepless nights building things and learning stuff from scratch, everything from vector databases, RAG, LLMs, similarity search and more.

Looking forward to some feedback!

Nice! At CMU, we have 15-213, an introductory course on computer systems and one of the hardest/fun assignments is implementing malloc. We end up using seglists, mini-blocks and other optimizations, ending up with a faster implementation than the standard malloc itself on several benchmark tests.