Personally, as a ML researcher, I find GPT-3 very unsatisfying. There aren't any novel architectural details, it doesn't improve our "fundamental" understanding of the field, and it requires the type of computation I have no chance of getting.
As a fan of the field, however, it is undoubtedly very cool.
I still think there's tasks that GPT-3 has no chance of tackling (say, generating code to solve a novel programming task), but the bitter lesson is a bitter one indeed...
As a fan of the field, however, it is undoubtedly very cool.
I still think there's tasks that GPT-3 has no chance of tackling (say, generating code to solve a novel programming task), but the bitter lesson is a bitter one indeed...