We already know the system is really bad at spelling. I have Claude configured to periodically remind me “By the way, I think there are ** n's in 'banana'”, so I don't forget what I am dealing with. It has never gotten this right.
But that doesn't mean that it is not extremely useful. It only means I shouldn't ask it to spell stuff.
If a human is unable to count the n's in 'banana' we expect them to be barely functional. Articles like this one try to draw the same inference about the LLM: it can't count 'n's, so it must not be able to do anything else either.
But it's a bad argument, and I'm tired of hearing it.
But that doesn't mean that it is not extremely useful. It only means I shouldn't ask it to spell stuff.
If a human is unable to count the n's in 'banana' we expect them to be barely functional. Articles like this one try to draw the same inference about the LLM: it can't count 'n's, so it must not be able to do anything else either.
But it's a bad argument, and I'm tired of hearing it.