Comment by kamranjon

Dwarfstar already supports this, curious how it compares, but I use the q4 quant daily and it works really well.

https://github.com/antirez/ds4/blob/main/docs/MODELS.md#qwen...