When you hear a company talk about the promises of synthetic data, you should run far far away. The fundamental problem is that in order for synthetic data to be useful for model training the generative synthetic model must have already solved the problem at hand. Training on the synthetic data is just a charade: you would be better off simply extracting the model you need from the source generative model. For generative models with densities this is as simple as P(Y | X) = P(X, Y) \ P(X).
So, what do you get out of it compared to simply training models as a service for people? Almost nothing useful. All you get are:
- Much worse privacy guarantees. (People selling synthetic data love to talk about improved privacy, but it's actually the reverse. The privacy guarantees for synthetic data are much, much worse than selling direct models to people.)
- Much worse model performance. See the previous notes about how a synthetic data generating procedure must have already solved the problem at hand.
- A much more complicated setup with much more expensive model training. Training generative models is hard and requires a lot of data and compute due to the difficulty in learning such a complex outcome space. This can easily cost 100-1000x as much as simply training a straightforward xgboost model.
So, what do you get out of it compared to simply training models as a service for people? Almost nothing useful. All you get are:
- Much worse privacy guarantees. (People selling synthetic data love to talk about improved privacy, but it's actually the reverse. The privacy guarantees for synthetic data are much, much worse than selling direct models to people.)
- Much worse model performance. See the previous notes about how a synthetic data generating procedure must have already solved the problem at hand.
- A much more complicated setup with much more expensive model training. Training generative models is hard and requires a lot of data and compute due to the difficulty in learning such a complex outcome space. This can easily cost 100-1000x as much as simply training a straightforward xgboost model.