Comment by lostmsu

Is there a workload that actually needs more than 24GB of VRAM that can't be offloaded to regular RAM.

I honestly don't have any examples out of my head. LLM inference is compute bound AFAIK.