"It may well be the case that the gender split is 80% / 20%"
That's a preposterous hypothetical. You're also incorrect in your implication - the existence of minor (and as yet unsubstantiated) fundamental differences in propensity to pursue technology in no way precludes the (well established) social biases and inequities that result in the same. Neither would it prevent these issues from being reflected in statistical models.
Thought experiments about magical perfect AIs are completely and totally irrelevant to this real problem which everyone who uses models should be aware of.
To get on my cranky, gatekeeping high horse, I find it super frustrating to see people who call themselves data scientists misunderstanding this problem. I teach this in introductory statistics to non-technical audiences. I'd hope for better from engineers and computer scientists.
"It may well be the case that the gender split is 80% / 20%"
That's a preposterous hypothetical. You're also incorrect in your implication - the existence of minor (and as yet unsubstantiated) fundamental differences in propensity to pursue technology in no way precludes the (well established) social biases and inequities that result in the same. Neither would it prevent these issues from being reflected in statistical models.
Thought experiments about magical perfect AIs are completely and totally irrelevant to this real problem which everyone who uses models should be aware of.
To get on my cranky, gatekeeping high horse, I find it super frustrating to see people who call themselves data scientists misunderstanding this problem. I teach this in introductory statistics to non-technical audiences. I'd hope for better from engineers and computer scientists.