One question that would be interesting to explore is: what is the predictive score of judges/juries/etc? Is it more or less variable than the COMPAS score?
In other words, do judges perform better than COMPAS? And slightly less important: is the variance lower?
If it isn't, then there is no point arguing whether COMPAS is risky: it would be less so than the alternative.
Although I suppose that the 'crowd of non-experts' is essentially what a jury is and they had about the same performance.
At any rate, it sounds like much less of a hassle to input characteristics into COMPAS and get a probability than getting a jury together.
Besides, I think I'd be much more worried that a jury could influence each other and bias its decision than about am algorithm making unfair predictions.
In other words, do judges perform better than COMPAS? And slightly less important: is the variance lower?
If it isn't, then there is no point arguing whether COMPAS is risky: it would be less so than the alternative.
Although I suppose that the 'crowd of non-experts' is essentially what a jury is and they had about the same performance.
At any rate, it sounds like much less of a hassle to input characteristics into COMPAS and get a probability than getting a jury together.
Besides, I think I'd be much more worried that a jury could influence each other and bias its decision than about am algorithm making unfair predictions.