Comment by docfort

There is some recent work [0] that explores this idea, scaling up n-gram models substantially while using word2vec vectors to understand similarity. Used to compute something the authors call the Creativity Index [1].

[0]: https://infini-gram.io [1]: https://arxiv.org/abs/2410.04265v1