> Presciently, Hutter appears to be absolutely right. His enwik8 and enwik9’s benchmark datasets are, today, best compressed by a 169M parameter LLM
Okay, that's not fair. There's a big advantage to having an external compressor and reference file whose bytes aren't counted, whether or not your compressor models knowledge.
More importantly, even with that advantage it only wins on the much smaller enwiki8. It loses pretty badly on enwiki9.
Okay, that's not fair. There's a big advantage to having an external compressor and reference file whose bytes aren't counted, whether or not your compressor models knowledge.
More importantly, even with that advantage it only wins on the much smaller enwiki8. It loses pretty badly on enwiki9.