As added support for this view, I can contrast with the relatively uncommon experience of having some experiences unified: when listening to an audio book, my brain often (but not always!) ties the experience of the book with my physical location at the time. When I return to that location, the experience of being there is inextricably tied to the portions of the book I heard in that place. I can't really remember one without the other.
But this is unusual. Most things are not tied together like this. In the author's example, I can separately remember the philosophical musings that were in my head on a drive, and the feeling of the steering wheel (which normally would be too mundane to remember, but on a hot day the cover will get sticky and I'll be very aware of that). There might be a weak association from one to the other, but usually not much at all.
As a tangent, I wonder if LLM attention heads are doing something analogous. Each is associated with a strand, sometimes combined with the others and usually not? (I don't know enough to say whether this matches at all.)
But this is unusual. Most things are not tied together like this. In the author's example, I can separately remember the philosophical musings that were in my head on a drive, and the feeling of the steering wheel (which normally would be too mundane to remember, but on a hot day the cover will get sticky and I'll be very aware of that). There might be a weak association from one to the other, but usually not much at all.
As a tangent, I wonder if LLM attention heads are doing something analogous. Each is associated with a strand, sometimes combined with the others and usually not? (I don't know enough to say whether this matches at all.)