Very fascinating. If we train on every piece of knowledge, then statistically that should average around 100IQ right? It's not perfect analogy, but the model is shooting more for consensus, a bit hive mind.
It feels like to improve the quality of the model, we need to be more selective of training data.
It feels like to improve the quality of the model, we need to be more selective of training data.