Comment Comment by chillee This is extremely wrong.The attention component that's quadratic is a relatively small portion of compute. parent chillee 2 years ago
The attention component that's quadratic is a relatively small portion of compute.