Comment by stared

Before this experiment I tried to run KLD on various context lengths to see a quantization-dependent deterioration.

On wikitext2 there was no difference. I concluded these have no long-term dependency and I should use Linux kernel. Still, the same.

So yes, KLD depends on the dataset. Still, it does not measure what any e2e test does.