Comment by Lwerewolf

You need latency for token parallelism, not bandwidth. Hence actual RDMA that bypasses the software TCP stack (ROCe or whatever).