Comment by eblanshey

Why not just run FP8 on vLLM with that much vRAM? It's plenty fast.