I think certain classes of artificial neural networks are definitely more interpretable than others.
For example, a trillion-parameter transformer model built using many layers is likely significantly easier to read than one that only has a few million parameters but was constructed using neuroevolution over one gigantic layer.
The degree to which a model can be quantized without losing its performance also likely contributes to interpretability. The redundancy can help humans lock onto patterns. If there is no redundancy at all (everything is a surprise in information theoretical terms), then the stream of parameters will look indistinguishable from cosmic background radiation.
For example, a trillion-parameter transformer model built using many layers is likely significantly easier to read than one that only has a few million parameters but was constructed using neuroevolution over one gigantic layer.
The degree to which a model can be quantized without losing its performance also likely contributes to interpretability. The redundancy can help humans lock onto patterns. If there is no redundancy at all (everything is a surprise in information theoretical terms), then the stream of parameters will look indistinguishable from cosmic background radiation.