Comment by Espressosaurus

My impression based on a cursory look at the research is that the caching algorithm is absolutely vital, non-obvious, model-dependent, and potentially prompt-dependent to make sure you're holding onto the right experts.

It doesn't look like a solved problem yet.