> There are a few arguments why adding sparse search doesn't require too much extra specialization
Full-text search != sparse search, that's a naive oversimplification. Btw, sparse search is on Qdrant roadmap, so we should be able to compare it's performance on benchmarks.
> Cross Encoder inference generally doesn't happen in the database itself thus it makes sense to use modules to process the additional ranking logic
That statement makes your argument that `A combined system have better end-to-end latency.` invalid
> Such benchmarks exist as in trengrj's initial response and we are working on them as well.
link or it didn't happen.
In your current benchmarks you advertise everywhere, you're just throwing in disproportionately powerful and expensive hardware. Even a full-scan can give good results under those conditions
Full-text search != sparse search, that's a naive oversimplification. Btw, sparse search is on Qdrant roadmap, so we should be able to compare it's performance on benchmarks.
> Cross Encoder inference generally doesn't happen in the database itself thus it makes sense to use modules to process the additional ranking logic
That statement makes your argument that `A combined system have better end-to-end latency.` invalid
> Such benchmarks exist as in trengrj's initial response and we are working on them as well.
link or it didn't happen.
In your current benchmarks you advertise everywhere, you're just throwing in disproportionately powerful and expensive hardware. Even a full-scan can give good results under those conditions