From the POV of an AI practitioner, there is one and only one reason I remain unimpressed with GPT3.
It is nothing more than one big transformer. At a technical level, it does nothing impressive, apart from throw money at a problem.
So in that sense, having already been impressed at Transformers and then ELMO/BERT/GPT-1 (for making massive pretraining popular). There is nothing in GPT3 that is particularly impressive outside of Transformers and massive pre-training, both of which are well known in the community.
So, yeah, I am very impressed by how well transformers scale. But, idk if I'd give OpenAI any credit for that.
It is nothing more than one big transformer. At a technical level, it does nothing impressive, apart from throw money at a problem.
So in that sense, having already been impressed at Transformers and then ELMO/BERT/GPT-1 (for making massive pretraining popular). There is nothing in GPT3 that is particularly impressive outside of Transformers and massive pre-training, both of which are well known in the community.
So, yeah, I am very impressed by how well transformers scale. But, idk if I'd give OpenAI any credit for that.