> the dataset contained ... demographic attributes like race ... to predict actual work performance
Regardless of the original intent, that's a request to launder a protected class into an opaque "work performance" metric. Also, while they are not on the unfortunately short list of protected classes, using demographics like age is probably going to create similar discrimination problems.
If you want to know about job performance, use data that is at least theoretically related to someone's performance. While machine learning is going to find some sort opf signal in whatever pile of data you feed it, that doesn't mean it actually predicts something complicated like job performance. When you feed it a bunch of general demographics, it's going to find one of the signals that we know are present throughout existing demographic: the institutional discrimination and similar problems of the past.
Regardless of the original intent, that's a request to launder a protected class into an opaque "work performance" metric. Also, while they are not on the unfortunately short list of protected classes, using demographics like age is probably going to create similar discrimination problems.
If you want to know about job performance, use data that is at least theoretically related to someone's performance. While machine learning is going to find some sort opf signal in whatever pile of data you feed it, that doesn't mean it actually predicts something complicated like job performance. When you feed it a bunch of general demographics, it's going to find one of the signals that we know are present throughout existing demographic: the institutional discrimination and similar problems of the past.