>"How many Rs in strawberry?" is not a trick question.
To a system that sees letters and words, sure. To a system that doesn't, you're just asking a blind man to count how many apples are on the table and patting yourself on the back for his failure. It just doesn't make any sense to begin with.
And this is before the fact that human intelligence is rife with absurd seeming failure modes and cognitive biases.
It's frankly very telling that these token related questions are the most popular kind of questions for these discussions.
This is no testable definition of reasoning or intelligence that will cleanly separate LLMs and Humans. That is the reality today. And it should make anybody pause.
To a system that sees letters and words, sure. To a system that doesn't, you're just asking a blind man to count how many apples are on the table and patting yourself on the back for his failure. It just doesn't make any sense to begin with.
And this is before the fact that human intelligence is rife with absurd seeming failure modes and cognitive biases.
It's frankly very telling that these token related questions are the most popular kind of questions for these discussions.
This is no testable definition of reasoning or intelligence that will cleanly separate LLMs and Humans. That is the reality today. And it should make anybody pause.