Comment Comment by jayalammar Additional ones that come to mind now are:Transformer Feed-Forward Layers Are Key-Value Memories https://arxiv.org/abs/2012.14913The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention https://arxiv.org/abs/2202.05798https://github.com/neelnanda-io/TransformerLens parent jayalammar 3 years ago
Transformer Feed-Forward Layers Are Key-Value Memories https://arxiv.org/abs/2012.14913
The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention https://arxiv.org/abs/2202.05798
https://github.com/neelnanda-io/TransformerLens