kennethwolters

First commit was 2 weeks ago. A clean-room rewrite of only the most essential APIs of litellm.

Was annoyed that DSPy was about 250 MB big. Mostly due to litellm being bloated and numpy. In order to create dspy-lite [1], I had to create a lighter litellm and make numpy optional (only essential for embeddings).

Made the lighter litellm litelm by selecting the parts of litellm's test suite that mattered and build it back to match litellms API.

Wrote a piece arguing that AI 1.0 is feudal because GPU scarcity concentrated power in a few cloud providers, but the coming GPU glut will invert this dynamic and make open weights + commodity compute the foundation of AI 2.0. The thesis is that sovereignty requires ownership of your stack (weights, compute, data) rather than renting intelligence from centralized APIs.

LLM-driven narrative game. Main technical issue is how go do compaction. I’ve devised a memory hierarchy that compacts the story to a constant amount of tokens per layer. Arc -> Scene -> Moment -> Line. Not sure if that’s the right dimensions to decompose into. Also tinkering how to get the right amount of “divergence” for story progression option generation. A lot of unanswered questions…

for me it feels like: Everything is stateful by default/convenience. Building robust systems is in part about confining statefulness to as few parts as possible. To contain statefulness. It’s to buy you some time and capacity. Yet the toughest problems often arise in the stateful parts of the system as well as quasi-stateless parts which sometimes develop hidden statefulness (think of syncing webclient and server state). So being good at handling stateful systems is valuable. Maybe one should even embrace statefulness. However, the AWS Solution Architect will tell you otherwise.

LLMs are pretty slow. Started a blog post series sharing what I've learned in search of faster LLMs.

Wrote an essay on what LLMs reaching their technological limits could mean to "AI Engineering".

not an attempt at rebutting or so, just adding my thoughts:

It doesn't matter if the product is great because of those values if there is a lack product (bc politics and bureaucracy make it infeasible to produce). Makes you think if those values might contribute to the bad politics and bureaucracy. And if so, how.

Why don't you think that calling the app "Llama-OCR" is good enough?

can someone tell me what the difference is to the webclient? Seems like a webview to me. No local caching/store.