In my PhD research (on computer vision at University of Tokyo), one of the topics I was exploring was reliability of models to synthetic data. Using generative models, we could show that models which were built on binary data sources were somewhat more reliable than others. Consider different geometries of rashes on skin & different skin location - the model could still be closer to ground truth (i.e model performance on real skin lesions)
However when other variables came into the mix (different tones, ethnicity, participant age etc), the model predictions were up for tosses.
Now, medical data is much more harder in ML. The variability is quite extreme since all our bodies are little different from one another. In my thesis, I very firmly proposed that generative models should never ever be used in synthetic medical information as part of the thesis proposal - and there were those field studies to back it up. (The thesis is under non-disclosure at University of Tokyo until September 2023 since an embargo is involved from participating hospitals & data sources. I would have been eager to share otherwise). I am glad people aren't wildly trusting if task A, B & C are doable, task AB, CA, ABC, BC etc should be similarly tractable.
However when other variables came into the mix (different tones, ethnicity, participant age etc), the model predictions were up for tosses.
Now, medical data is much more harder in ML. The variability is quite extreme since all our bodies are little different from one another. In my thesis, I very firmly proposed that generative models should never ever be used in synthetic medical information as part of the thesis proposal - and there were those field studies to back it up. (The thesis is under non-disclosure at University of Tokyo until September 2023 since an embargo is involved from participating hospitals & data sources. I would have been eager to share otherwise). I am glad people aren't wildly trusting if task A, B & C are doable, task AB, CA, ABC, BC etc should be similarly tractable.