Comment by sanxiyn

If you are interested in grokking you should have a look at this: Progress measures for grokking via mechanistic interpretability. https://arxiv.org/abs/2301.05217