Joined 1 month ago
are you worried later-layer attention gets drowned out by earlier layers just because there are more of them contributing to the sum?