There isn't a hurdle here. These problems are fundamental to the technology.
If you told me 20 years ago that you could, by connecting a nuclear power plant to a virtually infinite number of GPUs, produce a machine capable of predicting the next token in a sequence, I'dve believed you, but also would have said that it would be an astoundingly huge waste of resources to use to get to that result.
People seem to ascribe some magic here that isn't happening, because they're surprised by the results.
If you told me 20 years ago that you could, by connecting a nuclear power plant to a virtually infinite number of GPUs, produce a machine capable of predicting the next token in a sequence, I'dve believed you, but also would have said that it would be an astoundingly huge waste of resources to use to get to that result.
People seem to ascribe some magic here that isn't happening, because they're surprised by the results.