ambrozk
Joined 5 years ago
Joined 5 years ago
I do not believe this is true in the slightest. I learned more in a single OS class than i learned in years of industry work. What you learn in industry tends to be distinct from what you learn in college, and if it isn't, then your college curriculum was probably a waste.
Decoder: Fixed representation vector + N decoded text tokens -> N+1th text token
Encoder/Decoder architecture: You take some tokenized text, run an encoder on it to get a fixed representation vector, and then recursively apply the decoder to your fixed representation vector and the 0...N tokens you've already produced to produce the N+1th token.
Decoder-only architecture: You take some tokenized text, and recursively apply a decoder to the 0...N tokens you've already produced to produce the N+1th token (without ever using an encoded representation vector).
Basically, an encoder produces this intermediate output which a decoder knows how to combine with some existing output to create more output (imagine, e.g., encoding a sentence in French, and then feeding a decoder the vector representation of that sentence plus the three words you've translated so far, so that it can figure out the next word in the translation). A decoder can be made to require an intermediate context vector, or (this is how it's done in decoder-only architectures) it can be made to require only the text produced so far.
Like a lot of people here, I have family who died in the Holocaust. It is highly likely that the camps they moved through were using tech supplied by IBM. Would it have been "culture warring" for tech employees circa 1938, to publicize the human rights abuses being enabled by their companies? Because that's what the people who support these protesters think is going on here.