Comment Comment by brucethemoose2 FYI: Cerebras's nodes are very different than your typical Nvidia training nodes:https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...Each individual "chip" has 40GB of SRAM vs ~76MB for the Nvidia H100, and networked pools of external RAM, SSDs and such. Thats why the training architecture is so different. parent brucethemoose2 3 years ago
https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...
Each individual "chip" has 40GB of SRAM vs ~76MB for the Nvidia H100, and networked pools of external RAM, SSDs and such. Thats why the training architecture is so different.