Rail-optimized network topology for AI fabrics

What rail-optimized means, and how it differs from rail-only

Modern GPU servers ship with 8 NICs. A rail is one NIC position taken from every server: NIC-1 from every server forms rail-1, and so on to rail-8. Rail-optimized homes each rail to its own dedicated leaf, so every server has one link into each of the 8 rail leaves. The payoff is locality: xCCL libraries schedule the dominant AllReduce inside a single rail, and because that rail is one leaf, the traffic never leaves the leaf. Only the smaller cross-rail remainder reaches the spine.

• Rail-only keeps the rail-aligned leaves but drops the spine tier, so cross-rail traffic leans on the in-server GPU scale-up domain rather than a network path. Cheaper for a handful of racks, but with no fabric route for cross-rail flows once the cluster outgrows one scale-up domain.
• Rail-optimized adds a 1:1 non-blocking spine above the rail leaves, giving cross-rail flows a real network path. It is the standard single-pod scalable unit; rail-only is the entry point below it.

Why rail-optimized matters

Distributed training is dominated by collectives such as AllReduce for gradient synchronization, run through xCCL libraries (NCCL, RCCL, oneCCL). Rail alignment puts same-rank GPUs on a shared leaf, so those collectives complete with the lowest hop count and no spine transit for the common case. That holds down tail latency, and tail latency sets the pace of a synchronous step: it finishes only when the slowest GPU finishes its exchange.

How far a rail-optimized fabric scales

Scale is set by the switch radix and by how each 800G port maps to GPUs. On radix-64 Broadcom Tomahawk 5 (51.2 Tbps, 64×800G), the designs run from a 2-tier leaf-spine up to a 3-stage Clos, shown below. Because most collective traffic stays rail-local, the super-spine plane is sized only to the cross-pod ratio you actually need. For context, Meta has publicly described a 24,000-GPU Ethernet-based AI training cluster, which shows Ethernet fabrics operating well beyond a single pod.

State dimensionando il vostro cluster? The AI Fabric Design Suite sizes a non-blocking two-tier leaf-spine pod at one fabric NIC per GPU and flags when you cross into three-tier scale. For the full topology picture, see the Topologie AI fabric reference.

Rail vs ToR: aligning the fabric to GPU ranks

The rail-versus-ToR choice is about what the fabric aligns to. A rail layout aligns each GPU rank to a network plane, so same-rank GPUs across the cluster share a leaf and the collective completes with the lowest hop count. A ToR layout aligns to the server: every NIC in one server lands on its rack switch, which is simpler to cable but forces same-rank GPUs in different racks up to the spine and back. For GPU training fabrics, where AllReduce dominates, rail alignment is the standard; ToR stays a fine choice for storage and general-purpose racks where traffic is not rank-synchronous.