Comment by arcanus

While FP64 is indeed important for supercomputers, the largest supercomputers have a great deal in common with AI infrastructure.

For example, high bandwidth, low latency interconnects, supporting GPU direct network messaging and IO, are important.

High memory bandwidth is also quite important.

Debugging and performance profiling at scale also commonly uses similar tools.