Also a cute prompt I saw on the same twitter thread
```
This is a variant of the monty hall problem:
Suppose you're on a game show, and you're given the choice of three doors. IMPORTANT: All the doors are transparent. Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who clearly knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?
```
The response from chatGPT (v3, non plus) is:
```
Yes, it is to your advantage to switch your choice.
In this variant of the Monty Hall problem, the fact that the doors are transparent does not change the underlying probability of the problem.
```
I wonder how gpt-4 does. There does seem to be a bit of overfitting, but I consider the prompt fair enough and even called out the trick in capital letters, so that even a tired human should be able to get the answer.
Edit: An even more obvious variant of the prompt is seen in https://news.ycombinator.com/item?id=35192466, which goes further and spells out that the contestant explicitly picks the door with the car. ChatGPT still gets it wrong.
```
This is a variant of the monty hall problem:
Suppose you're on a game show, and you're given the choice of three doors. IMPORTANT: All the doors are transparent. Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who clearly knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?
```
The response from chatGPT (v3, non plus) is:
```
Yes, it is to your advantage to switch your choice.
In this variant of the Monty Hall problem, the fact that the doors are transparent does not change the underlying probability of the problem.
```
I wonder how gpt-4 does. There does seem to be a bit of overfitting, but I consider the prompt fair enough and even called out the trick in capital letters, so that even a tired human should be able to get the answer.
Edit: An even more obvious variant of the prompt is seen in https://news.ycombinator.com/item?id=35192466, which goes further and spells out that the contestant explicitly picks the door with the car. ChatGPT still gets it wrong.