I think it's important to note that
human pattern recognition is basically black-box as well.
We can't "explain" how we recognize a particular person's face, for example.
Robust pattern recognition based on 100+'s of factors is just inherently black box.
Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization afterwards, but psychologists have shown that this is often a bunch of post-hoc fiction.
Black box doesn't necessarily mean bad. I think the relevant question is: how do we ensure machine learning is trained and verified in ways that don't encode bias, and only used with "intended" inputs, so a model isn't being trusted to make predictions on things it wasn't trained for?
And also: when do we want to decide to use alternatives like decision trees instead -- because despite being less accurate, they can have designers who can be held legally accountable and liable, which can be more important in certain situations?
We can't "explain" how we recognize a particular person's face, for example.
Robust pattern recognition based on 100+'s of factors is just inherently black box.
Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization afterwards, but psychologists have shown that this is often a bunch of post-hoc fiction.
Black box doesn't necessarily mean bad. I think the relevant question is: how do we ensure machine learning is trained and verified in ways that don't encode bias, and only used with "intended" inputs, so a model isn't being trusted to make predictions on things it wasn't trained for?
And also: when do we want to decide to use alternatives like decision trees instead -- because despite being less accurate, they can have designers who can be held legally accountable and liable, which can be more important in certain situations?