This is very interesting! Seems like a promising direction.
I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?
Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?
Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?
I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?
Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?
Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?