Comment by shuri

One theory I heard about this type of problem is because these algorithms tokenize the text early, and each token can be multiple characters.