Comment by make3

Importantly though, LLMs can generate questions in parallel, and there can be much fewer questions (q) than segments to evaluate against (n), so the cost of jev on questions (jev(q * n) + llm(q)) vs the cost of putting all the segments in the LLM can be smaller (llm(n)), (llm(n) >> llm(q) + jev(n * q) if n >> q)

The real benchmark that's missing for me is MaxSim on a real embedder from MTEB.