dbagr

This has been known for a long time to those interested in the field.

You need recursion at some point: you can't account for all possible scenarios of combinations, as you would need an infinite number of layers.

As I said, either by benchmark contamination (it is semi-private and could have been obtained by persons from other companies which model have been benchmarked) or by having more compute.

Either they overtook other LLMs by simply using more compute (which is reasonable to think as they have a lot of GPUs) or I'm willing to bet there is benchmark contamination. I don't think their engineering team came up with any better techniques than used in training other LLMs, and Elon has a history of making deceptive announcements.

This sounds like an RNN with extra steps.