spencer9714

Great work on the AVBD implementation! I’m building a decentralized GPU compute platform (using Node.js/WebGPU). Since the solver uses a 'coloring' approach for parallelism, how do you handle compute preemption or state recovery if a worker/tab is closed mid-solve? Is it feasible to serialize the solver state efficiently for a distributed queue?

Interesting concept. One thing I’m curious about if I’m in a cohort for something like DeepSeek V3 and another user spins up a heavy 24/7 job, how do you keep TTFT from degrading? vLLM’s continuous batching helps, but there’s still a physical limit with shared VRAM/compute. I’ve been grappling with this exact 'noisy neighbor' issue while building Runfra. We actually ended up moving toward a credit per task model on idle GPUs specifically to avoid that resource contention entirely.

Curious how you’re thinking about isolation here. Is there any hard guarantee on a 'slice' of the GPU, or is it mostly just handled by the vLLM scheduler?

That BitTorrent analogy is spot on. For Runfra, I’m mostly focusing on the orchestration and inference pipeline. Trying to fix the waiting problem, but also thinking a lot about yield, like how many actually usable outputs you get per run. Right now, I’m just dogfooding a small cluster of my own 3x 4060s. Since these are basically idle home GPUs, I have the luxury to bias toward quality over latency. If this ever turns into a marketplace, something like Mycellm for node reputation would be a lifesaver. Honesty is definitely the hardest nut to crack in decentralized compute. The batch first idea is really my attempt to kill that “slot machine” workflow where you have to babysit every single prompt. Instead, you just fire off a bunch and walk away. The scoring layer (CLIP / aesthetic filters) acts as an automated quality gate, quietly filtering out the "AI slop" in the background and only surfacing the winners.

Since I need $Temperature > 0$ for that creative randomness, I inevitably get a lot of junk. So I’m essentially treating these idle GPUs as a distributed filtration layer, and trading idle time for guaranteed better outputs.