This is an artifact of an implementation-specific trick that trades performance at character-level tasks for performance at everything else. It does not reflect anything inherent about this type of model's capabilities: https://www.gwern.net/GPT-3#bpes
GPT-3 does not see individual characters. It sees "djsjcnnrjfkalcr" chunked as [d, js, jc, nn, r, j, f, k, al, cr]. You can see for yourself here: https://beta.openai.com/tokenizer.
GPT-3 does not see individual characters. It sees "djsjcnnrjfkalcr" chunked as [d, js, jc, nn, r, j, f, k, al, cr]. You can see for yourself here: https://beta.openai.com/tokenizer.