j0selit0

I need changes I can understand and review

that makes me even more humbled :)

author here - apologies for it not being clear. the idea here is:

step 1: sparse index retrieval (FTS/BM25) - say with k = 10 step 2: re-rank the 10 records using embeddings

the difference in this approach is during step 2, you convert text to embeddings on the fly - when you're running the retrieval pipeline, meaning you don't need to have all of your corpus pre-embedded in a vector db

you would also need to maintain multiple indexes in multiple languages. I never had to do that - but I assume it's a pain

good one. yes, that's pretty much my experience with query rewriting as well

to be fully transparent, I used LLM to review my draft yes. and interestingly enough, this article had tens of thousands of more views and estimulated significantly more discussions than my other, non-LLM written articles. so there's that.

and for the record, my last employer was still using gpt-4o and gpt-4o mini last year. and they are an F500 (not that it means anything, just for context).

sorry if it came across as name dropping - the point I wanted to make is actually that F500s are chasing RAG (as in pure semantic search without any care for chunking strategies, evals etc) instead of starting with the basics.

and for the record, my last employer was still using gpt-4o and gpt-4o mini last year. and they are an F500 (not that it means anything, just for context).

this is quite nuanced. in financial markets you have a combination of natural language questions that involve technical keywords / slang / acronym. and for these specific terms, an off-the-shelf embeddings model fails miserably.

my personal experience - I have been involved with such projects for the last 2 years. interestingly enough, a lot of times such initiatives didn't take off because people/stakeholders were overcomplicating things and wanting to use semantic search for everything - without having a minimum knowledge of chunking strategies, pros/cons etc

I work on a really similar scenario. We ended up having to create a so-called "semantic layer" containing metadata (table and schema descriptions) and glossary terms. Still, there is a lot of work involved maintaining glossary terms since some of them are ambiguous and people have different understanding/interpretations for some of them.

I wish everyone thought like you, in my experience unfortunately it's not the case