To me, it is not at all obvious that the "level" of the training set is an upper limit to the capabilities of an LLM.
Sure, the LLM hasn't been exposed to material more advanced than the most capable human domain experts have produced. However, it has seen and learned from a vast amount of information that these domain experts are completely unaware of. Why shouldn't the LLM be able to use that information to produce output that's beyond the capability of a domain expert?
To me, it is not at all obvious that the "level" of the training set is an upper limit to the capabilities of an LLM.
Sure, the LLM hasn't been exposed to material more advanced than the most capable human domain experts have produced. However, it has seen and learned from a vast amount of information that these domain experts are completely unaware of. Why shouldn't the LLM be able to use that information to produce output that's beyond the capability of a domain expert?