Comment by akomtu

Random thought: right now an LLM returns a probabilities distribution, an RNG sampler picks one and apoends it to the output, then the sequence repeats; but can the RNG instead pick N tokens that approximate the distribution, ask LLM to generate N new distributions, combine them somehow, then pick another set of N tokens from the combined dustribution?