IMO we are well past peak cosine-similarity-search as a service. Most people I talk to in the space don't bother using specialized vector DBs for that.
I think there's space for a much more interesting product that is longer-lived (since it's harder to implement than just cosine-similarity-search on vectors), which is:
1. Fine-tuning OSS embedding models on your real-world query patterns
2. Storing and recomputing embeddings for your data as you update the fine-tuned models.
MTEB averages are fine, but hardly anyone uses the average result: most use cases are specialized (i.e. classification vs clustering vs retrieval). The best models try to be decent at all of those, but I'd bet that finetuning on a specific use case would beat a general-purpose model, especially on your own dataset (your retrieval is probably meaningfully different than someone else's: code retrieval vs document Q&A, for example). And your queries are usually specialized! People using embeddings for RAG are generally not also trying to use the same embeddings for clustering or classification; and the reverse is true too (your recommendation system is likely different than your search system).
And if you're fine-tuning new models regularly, you need storage + management, since you'll need to recompute the embeddings every time you deploy a new model.
I would pay for a service that made (1) and (2) easy.
I think there's space for a much more interesting product that is longer-lived (since it's harder to implement than just cosine-similarity-search on vectors), which is:
1. Fine-tuning OSS embedding models on your real-world query patterns
2. Storing and recomputing embeddings for your data as you update the fine-tuned models.
MTEB averages are fine, but hardly anyone uses the average result: most use cases are specialized (i.e. classification vs clustering vs retrieval). The best models try to be decent at all of those, but I'd bet that finetuning on a specific use case would beat a general-purpose model, especially on your own dataset (your retrieval is probably meaningfully different than someone else's: code retrieval vs document Q&A, for example). And your queries are usually specialized! People using embeddings for RAG are generally not also trying to use the same embeddings for clustering or classification; and the reverse is true too (your recommendation system is likely different than your search system).
And if you're fine-tuning new models regularly, you need storage + management, since you'll need to recompute the embeddings every time you deploy a new model.
I would pay for a service that made (1) and (2) easy.