Comment by thelastparadise

Could it run on a 4x 3090 24GB rig?

These can be built for about $4500 or less all-in.

Inference FLOPs will be roughly equivalent to ~1.8X A100 perf.