If that's the goal then they can do it the other way round.
1. Pick random high quality maths papers that are outside of the training window
2. Convert them into a form which preserves the quality of the results but throws away the quality of the writing. For example, formalise them or keep only a high level summary of the proofs and throw away the introduction/conclusion
3. Generate full maths papers out of these summaries
4. Try to make the output maths papers be as similar as possible to the input papers.
You can see this as similar to training an auto-encoder, and I don't see any real downside to doing this over training on these corrected papers. Yes, the individual results would be less significant but you can't exactly mass produce training data by publishing papers and hoping someone fixes it.
1. Pick random high quality maths papers that are outside of the training window 2. Convert them into a form which preserves the quality of the results but throws away the quality of the writing. For example, formalise them or keep only a high level summary of the proofs and throw away the introduction/conclusion 3. Generate full maths papers out of these summaries 4. Try to make the output maths papers be as similar as possible to the input papers.
You can see this as similar to training an auto-encoder, and I don't see any real downside to doing this over training on these corrected papers. Yes, the individual results would be less significant but you can't exactly mass produce training data by publishing papers and hoping someone fixes it.