> ... the higher the wordcount, the deeper the "spaces get"...
During training, what I would say is that the "space" gets denser. Imagine you live on a cliff overlooking a lake/sea (some body of water with known boundaries). You notice on some days the winds produces long waves that are spaced far apart, while other days the waves are very short and choppy. If you wanted to encode this, it would take more memory to encode the latter than the former despite the lake being the same size.
If you have more questions about the Deep Schizophrenia model, I'd be happy to discuss. You'll see my email at the bottom of the site.
During training, what I would say is that the "space" gets denser. Imagine you live on a cliff overlooking a lake/sea (some body of water with known boundaries). You notice on some days the winds produces long waves that are spaced far apart, while other days the waves are very short and choppy. If you wanted to encode this, it would take more memory to encode the latter than the former despite the lake being the same size.
If you have more questions about the Deep Schizophrenia model, I'd be happy to discuss. You'll see my email at the bottom of the site.
[1] https://en.wikipedia.org/wiki/Swell_(ocean)