Comment by TillE

GPT-4 is not solving logic problems, it's doing an extremely good job of generating text based on recognizing familiar patterns.

If you give it something weird and unfamiliar, it will absolutely fail.