This makes it worse IMO. I was starting to think it didn’t have a letter by letter representation of the tokens. It does. In which case the fact it didn’t decide to use it speaks even more towards its unsophistication.
Regardless, I’d love if you would explain a bit more why the transformer internals make this problem so difficult?
Regardless, I’d love if you would explain a bit more why the transformer internals make this problem so difficult?