Comment by swyx

> RNNs famously have infinite context in theory and very short context in reality.

any sources to read more about this please? its the first ive heard of it