I think a lot of it is just time. The quality of a model is E * C
Where:
E = Efficiency, and efficiency gains come from quality of data, quality of algorithms.
C = Compute (Size of model, flops of train run)
So a better company can train a bigger and better model with less required compute which let's anthropic get there first. If another company does the same thing with a worse: model architecture, kernel, optimizer, etc... They will get there as well if they just run there train run with more flops for longer
Mythos was actually ready about 6 months ago. So if you have 6 months later or hardware setup and time to train you can get a lot done.
Where: E = Efficiency, and efficiency gains come from quality of data, quality of algorithms. C = Compute (Size of model, flops of train run)
So a better company can train a bigger and better model with less required compute which let's anthropic get there first. If another company does the same thing with a worse: model architecture, kernel, optimizer, etc... They will get there as well if they just run there train run with more flops for longer
Mythos was actually ready about 6 months ago. So if you have 6 months later or hardware setup and time to train you can get a lot done.