Comment by tempoponet

My take as well. The dual-3090 and dual-spark configs have an outsized voice in this conversation. I'd add pipeline parallelism vs tensor parallelism and in the case of TP setting your lanes and speeds per card.