Comment by isjustintime

Very cool. We use Tiktoken and I'd love to see the performance impact. Pretty great decision to make it drop-in compatible.