I once again feel that a comparison to humans is fitting.
We are also "trained" on a huge amount of input over a large amount of time.
We will also try to guess the most natural continuation of our current prompt (setting). When asked about things it I can at times hallucinate things I was certain to be true.
It seems very natural to me that large advances in reasoning and logic in AI should come at the expense of output predictability and absolute precision.
It seems very natural to me that large advances in reasoning and logic in AI should come at the expense of output predictability and absolute precision.