What Is RDMA?

Zero-copy, CPU bypass, and why that is the point

A normal network transfer copies data through the kernel and leans on the CPU at both ends. RDMA removes both. The adapter reads directly from application memory on the sender and writes directly into application memory on the receiver, a pattern called zero-copy with kernel bypass. Nothing traverses the CPU on the data path. That is what delivers RDMA's combination of very low latency, very high throughput, and almost no CPU cost, and it is why RDMA became the standard for GPU-to-GPU communication.

How RoCEv2 carries it over Ethernet

RoCEv2 (RDMA over Converged Ethernet, version 2) wraps RDMA in UDP and IP so it can be routed across a standard Layer 3 Ethernet fabric, no separate network required. The RDMA operation still runs adapter to adapter with the same zero-copy behavior; RoCEv2 simply gives it a routable Ethernet path. This is what lets a multi-vendor Ethernet fabric stand in for InfiniBand on GPU clusters.

Why AI collectives depend on it

AI training runs collective operations such as AllReduce: after each training step, every GPU exchanges gradients with every other GPU, then all wait for that exchange before the next step. The data volumes are large and the schedule is tight. RDMA keeps each exchange fast and off the CPU, so the collective that every GPU is blocked on finishes sooner. Multiply that across millions of steps and RDMA is a direct lever on total training time.