dt23
Joined 5 years ago
Joined 5 years ago
In contrast, an LLM is usually built using a sequence of Transformer modules, which use something called "self-attention": it modifies each piece of input seen so far to include information about its relationship with the other bits of input. In text, this is a natural thing to do (what role does my word play in the sentence); you can also do it with images (giving you Vision Transformers, aka ViTs) but it might be less natural. After self-attention is a little fully connected network, and then the output is passed into a new transformer, etc, for a number of times (commonly 6), until the output of the final transformer is used as the prediction.
In a nutshell: very different architectures, exploiting different aspects of the data (CNN: local structure; Transformer: relationship between all elements in the context).
Every day at 3pm was "coffee and cookies", and my colleagues and I would join the line filled with physicists of all ages and from across the lab, to grab a cookie or maybe two. On Fridays the coffee turned into wine and the cookies into cheese.
The second floor, where the coffee was, had a rotating art installation, which at some point included an acrylic box filled with water. One day this box went from being empty to containing a live goldfish.
I hope Fermilab retains this sense of magic that I have found in so few other places.