I'd lean more towards emphasizing the importance of explainability. If a system is opaque, it cannot be trusted. This goes both for AI producing stupid results and for software that relies on security-by-obscurity.
If AI is made explainable, then confidence in AI should more closely map to actual capability (that is, bad AI will not be trusted because its flaws are on display, while good AI will be trusted because its strengths are on display). It will also provide AI developers with a better grasp of what they need to work on.
If AI is made explainable, then confidence in AI should more closely map to actual capability (that is, bad AI will not be trusted because its flaws are on display, while good AI will be trusted because its strengths are on display). It will also provide AI developers with a better grasp of what they need to work on.