I wonder if someone has redone the experiment of retraining LLMs on LLM output that was conducted a few years back that showed a pretty drastic degradation in quality. I imagine it's hard to perform on the closed source frontier models and the ballooning cost of training doesn't help.
In theory as the models improve their ability to ape human writing the effect should diminish, though iterative reingestion of their own output is probably still detrimental long term.
In theory as the models improve their ability to ape human writing the effect should diminish, though iterative reingestion of their own output is probably still detrimental long term.