I’ve also been trying to find the simplest examples where ChatGPT fails. I think these questions are way too hard. The questions I’ve been asking are ones a kindergartener can answer. For example:
> If I have 1 cupcake on a plate and I flip the plate upside-down, how many cupcakes will be left on the plate?
> ChatGPT: If you flip the plate upside-down, there will still be 1 cupcake on the plate. Flipping the plate over does not change the number of cupcakes that are on it.
That being said, it’s just as easy to find difficult questions that ChatGPT can answer shockingly well.
> I hope this is enough to show that ChatGPT doesn’t actually have a grasp on the concepts behind the words it’s using.
I think this is wrong. chatGPT clearly does understand concepts behind the words it uses. It’s not just regurgitating text, it is synthesizing text in original and creative ways.
> If I have 1 cupcake on a plate and I flip the plate upside-down, how many cupcakes will be left on the plate?
> ChatGPT: If you flip the plate upside-down, there will still be 1 cupcake on the plate. Flipping the plate over does not change the number of cupcakes that are on it.
That being said, it’s just as easy to find difficult questions that ChatGPT can answer shockingly well.
> I hope this is enough to show that ChatGPT doesn’t actually have a grasp on the concepts behind the words it’s using.
I think this is wrong. chatGPT clearly does understand concepts behind the words it uses. It’s not just regurgitating text, it is synthesizing text in original and creative ways.