To give more context, the current version is already an optimized one I considered good enough and didn't spend more time on it. In the first attempt, I used a vector database with indexes, trying to query for similar vectors. This was uselessly slow even in medium-sized repos. The brute-force NumPy solution is a significant improvement, making it faster than other calculations like clustering.
"vectors are not loaded into memory at once" is not true, I had in mind splitting computation into blocks.
One possible simple optimization is to use 16-bit floats in vector instead of 32-bit. I used this in a different project (halfvec in pgvector) without affecting results.
1536-dim embeddings from your case also can be reduced. This large vector usually doesn't give much benefit compared to smaller ones. And this is something I will compare in my own tests.
To give more context, the current version is already an optimized one I considered good enough and didn't spend more time on it. In the first attempt, I used a vector database with indexes, trying to query for similar vectors. This was uselessly slow even in medium-sized repos. The brute-force NumPy solution is a significant improvement, making it faster than other calculations like clustering.
"vectors are not loaded into memory at once" is not true, I had in mind splitting computation into blocks.
One possible simple optimization is to use 16-bit floats in vector instead of 32-bit. I used this in a different project (halfvec in pgvector) without affecting results.
1536-dim embeddings from your case also can be reduced. This large vector usually doesn't give much benefit compared to smaller ones. And this is something I will compare in my own tests.