thanks!
> ditched oLlama"
yeah! this is interesting.
> 8.1GB per slotserve process is a lot! Is that in your control?
yes, it is hard, but I agree the smaller the better. I'll work on that
> If it's local and open-weight, this could be marketed this way I think.
I like this!
> what's the actual, real use case for slotserve?
I'm working rn on an app on top of it that closes the loop and is a fully local AI app, an experiment. I'll publish it as soon as it is usable!
> built a small html hello world served via Python
What did you use as a harness here?
> ditched oLlama"
yeah! this is interesting.
> 8.1GB per slotserve process is a lot! Is that in your control?
yes, it is hard, but I agree the smaller the better. I'll work on that
> If it's local and open-weight, this could be marketed this way I think.
I like this!
> what's the actual, real use case for slotserve?
I'm working rn on an app on top of it that closes the loop and is a fully local AI app, an experiment. I'll publish it as soon as it is usable!
> built a small html hello world served via Python
What did you use as a harness here?