Comment by EGreg

I wonder how much of their output is simple raw “superpositions” and how much is from far more generalized emergent structures as it is trained and has to learn latent representations.

Only the first layer is a “mashup” of the raw data, other layers start to recognize patterns across the data!