In contrast, a typical DDR4 RAM on your typical desktop is 0.05 TBps, so yeah, 7.2TBps external bandwidth between GPUs is quite a lot.
-----------
For Frontier, the Slingshot NICs are 100GBps each. So each node can communicate with more bandwidth to each other than your typical Desktop computer has RAM-bandwidth.
For example, a typical DGX supercomputer system from NVidia is pushing 7.2TBps (https://www.nvidia.com/en-us/data-center/dgx-h100/) GPU-to-GPU communications.
In contrast, a typical DDR4 RAM on your typical desktop is 0.05 TBps, so yeah, 7.2TBps external bandwidth between GPUs is quite a lot.
-----------
For Frontier, the Slingshot NICs are 100GBps each. So each node can communicate with more bandwidth to each other than your typical Desktop computer has RAM-bandwidth.
The diagrams imply that there's 4x Slingshot NICs per node on Frontier, suggesting 400GBps bandwidth to the interconnect. (https://www.olcf.ornl.gov/wp-content/uploads/2020/02/frontie...)