So, during my internship I worked on a few RAG setups and one thing that always slowed us down was to them. Every small change in the documents made us reprocessing and reindexing everything from the start.
Recently, I have started working on optim-rag on a goal to reduce this overhead. Basically, It lets you open your data, edit or delete chunks, add new ones, and only reprocesses what actually changed when you commit those changes.
I have been testing it on my own textual notes and research material and updating stuff has been a lot a easier for me at least.
repo → github.com/Oqura-ai/optim-rag
This project is still in its early stages, and there’s plenty I want to improve. But since it’s already at a usable point as a primary application, I decided not to wait and just put it out there. Next, I’m planning to make it DB agnostic as currently it only supports qdrant. Also might want to further improve the MCP feature, to make it accessible on other applications.
Recently, I have started working on optim-rag on a goal to reduce this overhead. Basically, It lets you open your data, edit or delete chunks, add new ones, and only reprocesses what actually changed when you commit those changes.
I have been testing it on my own textual notes and research material and updating stuff has been a lot a easier for me at least.
repo → github.com/Oqura-ai/optim-rag
This project is still in its early stages, and there’s plenty I want to improve. But since it’s already at a usable point as a primary application, I decided not to wait and just put it out there. Next, I’m planning to make it DB agnostic as currently it only supports qdrant. Also might want to further improve the MCP feature, to make it accessible on other applications.