Comment by pavpanchekha

It's about chips with a large enough scale up domain. Larger domain allows for bigger model, which is what's driving this jump. You've got to get the chips, test them, tune kernels, then start a big pre train, mid & post-train, and only then do you actually get the model. So it takes time. Anthropic got there first partly because they use different hardware (TPU I think, maybe Trainium) which had larger scale ups earlier.