Comment by junipertea

State size is constant (a matrix), attention is linear (a single unidirectional scan over all states)

Both statements are correct