Comment by rockinghigh

For small models, tokenization can reach 1-10% of total inference time.