Comment by conradkay

It's not quadratic attention, you get that curve from the input tokens going up linearly, since the graph is measuring cumulative cost at each token count. Basically for y=5 it's 5+4+3+2+1, or f(x) = x(x+1)/2

https://pbs.twimg.com/media/HNFc4Dma8AA76FW.jpg?name=orig