But I wonder whether LLM on its own might have gotten close. If you're going to use an LLM for each call then it seems reasonable to test whether or not adding Jev improves performance.
EDIT: I should have had more coffee before writing this comment. As make3 points out below, I forgot that in the LLM+Jev setup, the LLM only needs to run once per query, whereas in the LLM-only setup the LLM would run once per item/segment being evaluated.
- BM25 only
- Jev only
- LLM + Jev
The latter worked best.
But I wonder whether LLM on its own might have gotten close. If you're going to use an LLM for each call then it seems reasonable to test whether or not adding Jev improves performance.
EDIT: I should have had more coffee before writing this comment. As make3 points out below, I forgot that in the LLM+Jev setup, the LLM only needs to run once per query, whereas in the LLM-only setup the LLM would run once per item/segment being evaluated.