alf42red

I've only read 1-4 without prior knowledge of qkv attention and I like the explanation. The only thing I didn't get from the text was why we divide by the square root of d_k, but ChatGPT explained it was to counter variance scaling with the dimension of the matrix. I probably couldn't have come up with this myself, but I feel that it actually makes sense now and I like that deltanet attention "learns a learning rate" according to one interpretation of beta from the text, if I'm understanding it correctly. Great article!

I think it's great that you can still follow people like Lawrence Paulson and Moshe Vardi. It's not only a great time for programmers but also for computer scientists. Platforms like LinkedIn make it easy to share a quick observation or question, and occasionally getting a reaction from leading researchers is one of the things that keeps me interested in the field.

After skimming through this, I want to write a post about how others are living their lives wrong. It must really feel great to be right about things on the internet. How do I start? I guess I'll just ask AI ...