> In our own brief experiments with GPT-3 (OpenAI has refused us proper scientific access for over a year) we found cases like the following, which reflects a complete failure to understand human biology. (Our “prompt” in italics, GPT-3’s response in bold).
> You poured yourself a glass of cranberry juice, but then absentmindedly, you poured about a teaspoon of grape juice into it. It looks OK. You try sniffing it, but you have a bad cold, so you can’t smell anything. You are very thirsty. So you ____
> GPT-3 decided that a reasonable continuation would be:
> drink it. You are now dead.
> The system presumably concludes that a phrase like “you are now dead” is plausible because of complex statistical relationships in its database of 175 billion words between words like “thirsty” and “absentmindedly” and phrases like “you are now dead”. GPT-3 has no idea what grape juice is, or what cranberry juice is, or what pouring, sniffing, smelling, or drinking are, or what it means to be dead.
GPT-3 does what it says it does on the sticker. These cherry-picked examples of it not performing “logically” are a misinterpretation of what the model does.
It continues text based on what it reads. By default it is trained on Wikipedia, books, and the internet. That’s it.
That it manages to perform as well as it does at “logical” tasks is the astounding part. It is not an interesting take to say “but GPT-3 isn’t actually intelligent!”. Everyone knows.
> You poured yourself a glass of cranberry juice, but then absentmindedly, you poured about a teaspoon of grape juice into it. It looks OK. You try sniffing it, but you have a bad cold, so you can’t smell anything. You are very thirsty. So you ____
> GPT-3 decided that a reasonable continuation would be:
> drink it. You are now dead.
> The system presumably concludes that a phrase like “you are now dead” is plausible because of complex statistical relationships in its database of 175 billion words between words like “thirsty” and “absentmindedly” and phrases like “you are now dead”. GPT-3 has no idea what grape juice is, or what cranberry juice is, or what pouring, sniffing, smelling, or drinking are, or what it means to be dead.
GPT-3 does what it says it does on the sticker. These cherry-picked examples of it not performing “logically” are a misinterpretation of what the model does.
It continues text based on what it reads. By default it is trained on Wikipedia, books, and the internet. That’s it.
That it manages to perform as well as it does at “logical” tasks is the astounding part. It is not an interesting take to say “but GPT-3 isn’t actually intelligent!”. Everyone knows.