Perihermeneias

Thanks for the response. So is it more of a feeling of trust for the end-users that people want explainable AI? If an explainable model has the same performance as a black box model, does that really add any real world benefit though?

Or does that explainability potentially offer something to the FSD or fintech engineers/developers that allows them to make better models? And if so, what would that be? That’s where I seem to get stuck on this question, explainability sounds nice but I want to understand what specific, practical thing it would offer.

I’m thinking not just about AI for coding, which I agree doesn’t seem to need explainability, but AI/ML in general. In that case, I suppose your point would generalize to something like explainable AI would allow for performance guarantees of some sort? What would that look like in practice, though? Or is it more a subjective feeling of trust?

Is this something that AI is necessary for, though? Or perhaps you mean it would do this with more complex metrics.

What does this look like in practice though? Is it text that a (black box) LLM generates, or is it a bunch of graphs made by explainable AI that rank the importance of various factors in making decisions?

Thanks for the thoughtful response. What would this look like in something more complicated? In deep learning would this be something like “if you can get these neurons to fire more you can get a better output” and “you can increase these neurons firing by changing the inputs in this way”?

Also, I feel like this sounds a lot like generative AI, and explainability might not be necessary. Or maybe one could say that in the real world people want to have better assurance that things they do are based on a good underlying model. Images and text are easier to evaluate than predictive, speculative decisions about housing markets.