This is a recurring sentiment but flawed, I think.
First of all, neural nets do nit return averages per se. They construct space between the points and extrapolate outside
of the points. So even if a point was not in their training data, they will be ok, in many situations, to acknowledge it.
Or in other words - LLMs don’t average. They construct world models. A novel thing that fits their world model will be accepted no prob. A thing that doesn’t may still be accepted but with
challenges.
The same is true though for humans, including scientists. There is a saying that science moves one grave at a time - because often prev gen of scientists needs to die off for a new idea to take root.
Or in yet other words - even if llms produced averages, an average
of a discontinuous set can lie outside of that set. And the set of all human ideas is very much discontinuous.
First of all, neural nets do nit return averages per se. They construct space between the points and extrapolate outside of the points. So even if a point was not in their training data, they will be ok, in many situations, to acknowledge it.
Or in other words - LLMs don’t average. They construct world models. A novel thing that fits their world model will be accepted no prob. A thing that doesn’t may still be accepted but with challenges.
The same is true though for humans, including scientists. There is a saying that science moves one grave at a time - because often prev gen of scientists needs to die off for a new idea to take root.
Or in yet other words - even if llms produced averages, an average of a discontinuous set can lie outside of that set. And the set of all human ideas is very much discontinuous.