Comment Comment by D4Ha Do you find it useful or worthwhile to split a large LLM across two GPUs on a desktop?If you've tried it, what worked well and what didn't? I'm especially interested in mismatched VRAM setups, e.g. a 16 GB GPU + a 24 GB GPU.How much overhead did you see from inter-GPU transfers, and did the extra usable VRAM outweigh the performance hit? parent D4Ha 2 months ago
If you've tried it, what worked well and what didn't? I'm especially interested in mismatched VRAM setups, e.g. a 16 GB GPU + a 24 GB GPU.
How much overhead did you see from inter-GPU transfers, and did the extra usable VRAM outweigh the performance hit?