This is an interesting paper on a subject that has provoked debate on HN in the past. I would be interested to read criticism of the methods from those who have expressed support for these kinds of automated systems.
The most obvious criticism is that the uninspiring performance of the COMPAS model does not provide any evidence of the weakness of automated systems in general, which is surely correct. This kind of argument would not, however, address more general concerns about how and why these kinds of systems are adopted - why is this system being used when its performance is not obviously competitive? (The answer to that is presumably "corruption", in a generalised sense, but that is hardly an adequate response.)
Even more broadly, as the complexity of human society increases, it seems likely that the difficulty of making constructive interventions will also increase, and possibly more rapidly ("The Collapse of Complex Societies" by Joseph Tainter investigates this hypothesis). It does seem risky to just plough on, hoping that we can invent our way out of any problems that might arise as a result of what we are doing.
The most obvious criticism is that the uninspiring performance of the COMPAS model does not provide any evidence of the weakness of automated systems in general, which is surely correct. This kind of argument would not, however, address more general concerns about how and why these kinds of systems are adopted - why is this system being used when its performance is not obviously competitive? (The answer to that is presumably "corruption", in a generalised sense, but that is hardly an adequate response.)
Even more broadly, as the complexity of human society increases, it seems likely that the difficulty of making constructive interventions will also increase, and possibly more rapidly ("The Collapse of Complex Societies" by Joseph Tainter investigates this hypothesis). It does seem risky to just plough on, hoping that we can invent our way out of any problems that might arise as a result of what we are doing.