I don't think "black-box" algorithms are the problem per se. In this case the system was doomed from the get-go by using biased and flawed input attributes.
What I find very problematic in the use of automated systems for fraud detection is taking the system decision as final and not offering an easy and clear way of appealing decisions.
What I find very problematic in the use of automated systems for fraud detection is taking the system decision as final and not offering an easy and clear way of appealing decisions.