What's also cool is if you take that n-gram frequency model, and scale it up from just 2 characters ('d' and 'f') to a full alphabet. And increase n towards a very large number then you can in theory do a good job of predicting sentences. Unfortunately you'll have a lot of your long sequences will only have 1 or 2 data points as n gets large, and you'll need an absurd amount of memory to store it all
So if instead you replace it with a "compressed" model that doesn't actually store all the seen n-gram options, but instead predicts them, then you've more or less just re-invented LLMs.
So if instead you replace it with a "compressed" model that doesn't actually store all the seen n-gram options, but instead predicts them, then you've more or less just re-invented LLMs.