ROCE do require proper nic/switch tuning. But if you go all in on Mellanox NIC's and switches, its basically on knob on the switch: roce
If you buy CX3 and Mellanox SX6036(easy fan adjustment and low power usage) switches on Ebay for cheap. Which are great up to 56Gbps. You can run infiniband which is way easier to setup, just remember you still need to enable IP over IB to start the handshake for the RDMA connection.
If you buy CX3 and Mellanox SX6036(easy fan adjustment and low power usage) switches on Ebay for cheap. Which are great up to 56Gbps. You can run infiniband which is way easier to setup, just remember you still need to enable IP over IB to start the handshake for the RDMA connection.