Comment by lukeplato

I don't see why a mixture of experts couldn't be distilled into a single model and unified latent space