Comment Comment by howscrewedami openAI released an official tokenizer recently:https://platform.openai.com/tokenizer parent howscrewedami 3 years ago
https://platform.openai.com/tokenizer