Comment by apaszke

PyTorch uses NVIDIA NCCL for multi-GPU communication (under BSD license). Gloo is only one of the three backends that can be used for distributed communication out of the box.