Comment by sandkoan

I'd honestly argue that he might not have even needed OpenAI embeddings—any off-the-shelf Huggingface model would've sufficed.

Because of attention mechanisms, we no longer so heavily depend on the existence of those "pointy words," so generally, Transformers-based semantic search works quite well.