Comment by chzblck

Sounds interesting would love to test it out but here's what I got when I first tried to get some models downloaded.

On a 64gb Ram and 5080 machine the biggest model suggested was Qwen 9B

can hit 90+ tps on the MoE 35b but mag thinks it wont fit.