Comment by visarga

For an in-depth review you can see:

Yannic Kilcher - Big Bird: Transformers for Longer Sequences (Paper Explained)

https://www.youtube.com/watch?v=WVPE62Gk3EM