> "So would any reasonable human."
That's the entire point of the article, our biases, as reflected in the world, are learned by models. Consider the following:
* More computer programmers are men than women (descriptive statement, no problem)
* A predictive model correctly identifies that more computer programmers are men than women (a prediction based on observed data, no problem)
* A recruiting agency uses a predictive model to recruit computer programmers. Due to the way the model was trained, it excludes qualified women (not OK, a clear misapplication of an algorithm)
Feel free to try this thought experiment in other contexts where existing biases can be amplified through algorithms.
That's the entire point of the article, our biases, as reflected in the world, are learned by models. Consider the following:
* More computer programmers are men than women (descriptive statement, no problem)
* A predictive model correctly identifies that more computer programmers are men than women (a prediction based on observed data, no problem)
* A recruiting agency uses a predictive model to recruit computer programmers. Due to the way the model was trained, it excludes qualified women (not OK, a clear misapplication of an algorithm)
Feel free to try this thought experiment in other contexts where existing biases can be amplified through algorithms.