skarnati20
Joined 2 months ago
Joined 2 months ago
Lots of configurability and easy to define new models and tools! Let me know what you think -- it's my first time using macros in a non-trivial way.
This is a very basic Local LLM Chat + RAG implementation, but it has some features I couldn't find in the mainstream products.
I built this because I found Claude/GPT overkill for lots of programming questions I had, and I needed something that could work offline. It indexes a URL you provide and lets you ask questions to a local LLM (Ollama) with that information.
Some Features * Multiple Retrieval Methods — (BM25, Dense Vector Search, ColBERT). You can choose whichever combination of these you want. * Agentic RAG — An Agent determines when there is enough context to answer a question. You can turn it off if you want speed. * Fully Local Chat — Uses Ollama so you can run any model you want, so long as you serve it locally. * Asynchronous Chat — Allows you to ask multiple questions at once which helps with multi-tasking and getting around local models' slow completion times.
Acknowledgement: This project was an extension/combination of some projects in Mark Watson's Loving Common Lisp, or the Savvy Programmer's Secret Weapon. Definitely check it out since it gave me a lot of ideas for fun side projects.
2) The SBCL REPL is a very native place for a coding agent to live. I am not a fan of the TUIs that products like Codex or Claude Code have -- they're a little too restrictive. But the SBCL REPL is fun and interactive. It means I can do things like context management more programmatically and can inspect things in the program and in my conversation history clearly.
3) This kind of continues my macro point. But if adding support for a new model, new tool call, or new coding loop takes 20-30 lines and is already abstracted, it's very easy to have an LLM modify the harness to your purpose. There's an example where I created a custom Apprentice loop (more detail in README), but it lets you do more experimentation having the simple macros defined.
Also lots of folks today are talking about Recursive Self Intelligence and evolving intelligence, and I could see a world where in this project, you can create and define a tool call that edits the Lisp code itself to create a more effective harness! This is a unique Lisp capability due to the image-based nature of the REPL, but not something I have attempted so far.