Bibabomas

"it works fine and this is not a bottleneck. For large projects, other parts are much slower": I don't think this is true, especially for large projects. I just ran your tool on the Kubernetes repo with 1536-dim embeddings. The isolated similar-pair search took ~130s and peaked at ~4.8 GB RSS, and the total runtime was ~250s with the same memory peak.

"In larger sets, all vectors are not loaded into memory at once": this is also not correct, at least in the implementation you shared. The similarity matrix is processed in blocks, but the embeddings themselves are loaded all at once and stacked into one NumPy matrix, hence the memory peak.

So in larger projects, more than half of the time is spent on the similarity search step, and almost all the memory is spent there as well.

I don't understand what you mean with your first sentence. Both SemHash and Slopo are deduplication libraries right? Regardless, finding similar code is the core functionality that enables (semantic) deduplication.

I also don't think the backend is the "easiest part" here, a lot of the scalability lives there, which is important for monorepos, or cases where you want to deduplicate across projects. For example, from looking at the implementation, you use exact brute-force similarity search (comparing every item to every other item) which is an O(n^2) operation. It also allocates large dense similarity blocks in memory, so memory use won’t scale well either.

Fair, many tools trade off cost for performance/adherence. So far our experience is good with Semble (at least with Anthropic/OpenAI models), but if you have any feedback feel free to reach out. We're working on evaluations for this as well, though it will take some time as this is much harder than benchmarking retrieval quality.

There's a "semble savings" command you can run to see token savings. It varies heavily based on the repo: in general, the larger the project the larger the savings. Output code quality is something we're still trying to measure, but it's much harder than measuring retrieval quality.

Hey, this skepticism is fair and we share it, which is why we don't claim end-to-end agent improvements since we haven't measured those (yet). The benchmark we published measures retrieval quality and token count during search, not overall agent performance. We are working on agent-level evals, but those are unfortunately much harder to get right. However, we do believe that Semble makes agents better based on our own experience of using it for the past months while in development (or at the very least, cheaper).

Wow, thanks for sharing, and cool that you're working on similar things! Feel free to drop any feedback on the repo if you want!

This is mostly done with the prompt. To be fair, we are still evaluating and improving this as well!

Hey, what's the issue with a disk search-lib in python specifically? The library is extremely fast. Yes, we could probably squeeze some more performance in Rust, but that's not our native programming language, so we opted for doing it correctly rather than use a language that we don't understand well enough.

In theory maybe, but in practice it hurts more than it helps I think. Irrelevant context makes the model more likely to reason from the wrong code (and it's slower and more expensive).

Hey, thanks for the detailed feedback. For the bug, would you mind opening an issue with your setup details? This is definitely something we want to investigate and fix. The multiple queries thing is really good feedback, thanks for that, we'll update the prompt/instructions to prevent this from happening and we'll try to add some tests for this. The external connection errors during install are uv fetching deps from PyPI I think, those should not be the reason it's hanging.