Comment by nighthawk454

Anyone remember Universal Transformers paper (Dehghani et al) from back in 2018? Recurrent transformers have a history as long as transformers themselves.

Somewhat unclear how particularly novel this is vs a way to save compute.