I just got Claude Opus to fail at counting letters in a string.
Me:
What about in rrorooroororororrrrrrorrororororororrrorooorrorroror
Opus 5.5:
There are 29 r's in that string.
There are 30rs in the string. If you want I have a screenshot.
The reality is this, the models "think" in tokens, not in letters, so they can easily make mistakes like this and it's very hard to stop these kind of dumb mistakes happening. So yes, frontier intelligence at work.
Me: What about in rrorooroororororrrrrrorrororororororrrorooorrorroror
Opus 5.5: There are 29 r's in that string.
There are 30rs in the string. If you want I have a screenshot.
The reality is this, the models "think" in tokens, not in letters, so they can easily make mistakes like this and it's very hard to stop these kind of dumb mistakes happening. So yes, frontier intelligence at work.