Comment by itsumoiru

As you mentioned, the data processing inequality[1] applies here, but I imagine synthetic data could help training squeeze out more from the existing data.

[1] https://en.m.wikipedia.org/wiki/Data_processing_inequality