Tangential to this post but Peter Bloem’s Transformers from Scratch (2019) was one of the best early tutorials on transformers and self-attention. There have been many more tutorials since then but this was the one that solidified a mental picture for me.
https://peterbloem.nl/blog/transformers