The author has more experience in the field than me, so gotta defer to him for the most part, and while I generally agree with the post, I disagree strongly with one point. The author frames this new era of AI about using transformer models (and diffusion models) but transformer models have been around for a while and have been useful from before GPT3, the model the author claims as the starting point for this new AI. BERT is a transformer model that came out in 2018 and is a very useful transformer style model, which showed the promise of transformers before GPT3.
Edit: Going back through the post, the author’s slide has Transformers labeled as 2017, so he is aware of the history and he’s just emphasizing that GPT3 was the first transformer model that he thinks had something interesting and related to the current AI explosion. I think BERT style models would be worth a mention in the post as the first transformer models found to be widely useful.
Edit: Going back through the post, the author’s slide has Transformers labeled as 2017, so he is aware of the history and he’s just emphasizing that GPT3 was the first transformer model that he thinks had something interesting and related to the current AI explosion. I think BERT style models would be worth a mention in the post as the first transformer models found to be widely useful.