There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta.openai.com/tokenizer
This is a solved problem with models trained on byte-level objectives without tokenization like ByT5 (if you tried this task on one of those, it would probably work perfectly with a few samples). In GPT’s case, there’s a trade off between having a long context window vs being good at character level tasks, and OpenAI picked the former.
This is a solved problem with models trained on byte-level objectives without tokenization like ByT5 (if you tried this task on one of those, it would probably work perfectly with a few samples). In GPT’s case, there’s a trade off between having a long context window vs being good at character level tasks, and OpenAI picked the former.