This is not really true. The Chinchilla paper showed that a 4% difference in loss between Chinchilla and Gopher led Chinchilla to blow Gopher out of the water at most tasks, including 30x performance in physics.
Empirically, LLMs have shown to have emergent abilities appear at different loss levels. So, a 10% difference could really matter.
Empirically, LLMs have shown to have emergent abilities appear at different loss levels. So, a 10% difference could really matter.