G github.com Code for the Byte Pair Encoding algorithm, commonly used in LLM tokenization 81 points magoghm 2 years ago 31 comments
