As a counterargument: it's precisely because of high-dimensional statistical learning that interpretability is a valuable trait. Yes, the power of modern ML is that it can handle situations that the designers did not explicitly design for--but this doesn't necessarily mean that it handles them
well. For example, if your approval for a loan is subject to an AI and it rejected you, then you want to know
why you were not approved. You'd want the reason your application was not granted to be something reasonable (like a poor credit history) and not something like "the particular combination of inputs triggered some weird path and rejected you offhand." Another example is machine vision for self-driving cars. You want the car to understand what a stop sign
is and not just react to the color, otherwise the first pedestrian with a red jacket will bring the car to a screeching halt. Even though you may not have had red jackets in your training set (or may not have had enough so that misclassifications ended up contributing to your error percentage), you can verify the model works as intended using interpretability.
It's dangerous to treat this sort of models as a black box, as the details of how the model makes a decision is as important as the output; otherwise, how could it be trusted?
-
This topic is the subject of my thesis, so i am currently steeped in it. Let me know if I can answer any more questions!
It's dangerous to treat this sort of models as a black box, as the details of how the model makes a decision is as important as the output; otherwise, how could it be trusted?
-
This topic is the subject of my thesis, so i am currently steeped in it. Let me know if I can answer any more questions!