Comment by rcxdude

There is a difference between a supercomputer and just a large cluster of compute nodes: mainly this is in the bandwidth between the nodes. I suspect industry uses a larger number of smaller groups of highly-connected GPUs for AI work.