Comment by gmkiv

How does ALiBi compare to rotary positional embeddings? That method makes similar claims. I find ALiBi much easier to understand, but that’s probably not the best reason to chose it over other methods.