ketothekingdom

nice :)

I use a combination of Hugo with obsidian-style syntax for everything

Definitely! It's more of an intuition but doesn't explain what vector gets put in the weights and the analysis doesn't involve the learning dynamics at all. I link some papers at the end that do think about how memory relates to the dataset. But the architecture having this storing+lookup property probably is important in helping transformers remember so much information.

Thank you! It's been sitting in my head for so long. I'm glad I finally got to share it with people

That's such a neat idea! Now I want to try it out too