nutellalover

Codeviz is such a cool project. Love having it in VSCode.

Big fans of Cursor ourselves. One of the goals with this library is to make it easy for maintainers of OSS projects to expose chat support functionality to their users in a very streamlined, easy-to-setup fashion.

So yes you can certainly use to index and query your own repos for yourself, but it's also a way to get more of your OSS lib users onboarded.

Happy to help!

At the beginning, we started with qualitative "vibe" checks where we could iterate quickly and the delta in quality was still so significant that we could obviously see what was performing better.

Once we stopped trusting our ability to discern differences, we actually bit the bullet and made a small eval benchmark set (~20 queries across 3 repos of different sizes) and then used that to guide algorithmic development.

Great question. For most small repos (10-20 source files) this works incredibly well out-of-the-box.

We stress-tested with repos like langchain, llamaindex, kubernetes and there the retrieval still needs work to effectively return relevant chunks. This is still an open research question.

The truth is we started there. But for any reasonably-sized, complex codebase this just isn't going to work as the context window isn't sufficient and moreover it becomes harder for the LLM to reason over arbitrary parts of the context.

For the time being, indexing and retrieving a good collection of 10-20 code chunks is more effective/performant in practice.

This is a great read Simon.