Comment by onlyrealcuzzo

This is awesome, but tokenization is typically <0.1% of total inference time.

Presumably there's a host of applications that just need to tokenize, though, and this would be great for those!