Comment by wokwokwok

> Focusing on mathematical abstracta completely misses the problem...

I don’t think it does.

We’ve been studying these networks for a long time now, and “common sense” interpretations are largely “completely bs” when examined closely.

These investigations have a tangible impact on how to prune and design networks.

High level hand waving about measuring the measuring device may seem profound, but I fail to see any meaningful application.

You can reconstruct 3D images from a few samples; see neural radiance fields. The observer is irrelevant.