I too "read Vaswani et al. (2017) multiple times, carefully, and was quite unable to grasp what 'attention' was supposed to be doing. (I could follow the math.) I also read multiple tutorials, for multiple intended audiences, and got nothing from them."
It took years before I finally realized it was just a kernel smoothing (though I never used quite so precise language), all because of a poorly written paper. This is what I mean when I say almost every ML paper is trash. "Attention is All You Need" is even way better than most---ever read the Adam paper?
It took years before I finally realized it was just a kernel smoothing (though I never used quite so precise language), all because of a poorly written paper. This is what I mean when I say almost every ML paper is trash. "Attention is All You Need" is even way better than most---ever read the Adam paper?