matthewolfe

What exactly doesn't "just work" anymore?

For argument's sake, suppose we live in a world where many high-quality models can be run on-device. Is there any concern from companies/model developers about exposing their proprietary weights to the end user? It's generally not difficult to intercept traffic (weights) sent to and app, or just reverse the app itself.

To echo the other replies, the tokenizer is definitely not the bottleneck. It just happens to be the first step in inference, so it's what I did first.

haha, I thought about it.

Fair chance I'm remembering it wrong :D

Alright, 0.1.1 should now be a true drop-in replacement. I'll write up some examples soon.