I run Qwen3.8 Flash Next on 2x Sparks at $WORK and it has been great! It can run 16 concurrent sessions with decent speed (~30-40 t/s per session) and can hold 8 full 256k contexts in cache. It has turned out to be a great production setup for our company.
I have a half finished repo that sets it all up for production (using systemd, not hacky scripts and one-off docker commands), if someone is interested in this, tell me, and it might give me the push to actually finish up the latest threads and publish it!
I have a half finished repo that sets it all up for production (using systemd, not hacky scripts and one-off docker commands), if someone is interested in this, tell me, and it might give me the push to actually finish up the latest threads and publish it!