Rail-optimized · Two-tier and three-tier · Coherent DCI

AI Fabric Topologies: Rail-Optimized Designs to 16K GPUs

Match the topology to the GPU count: a rail-optimized two-tier leaf-spine pod up to 4,096 GPUs, a third super-spine tier above that up to 16,384, and coherent DCI to span sites. Pick the smallest non-blocking design that keeps every GPU's link saturated during collectives. All three run on OcNOS-DC over a lossless RoCEv2 (PFC + ECN) L3 fabric, sized here in concrete port-counts on Broadcom Tomahawk 4 and Tomahawk 5.

Book an architecture review Open the AI fabric design suite
up to 16,384GPUs, one design system
1:1non-blocking GPU plane
up to 800GTH5 leaf and spine
1 NOSOcNOS-DC, every tier

The smallest non-blocking design that fits your GPU count

An AI fabric topology has one job: keep every GPU's outbound link saturated during a collective without creating tail-latency outliers. The right topology is the smallest one that does this for your GPU count, with a fall-back path for the next size up. Three reference points, with concrete port maths on Broadcom Tomahawk silicon.

One NOS, every tier of scale
256 GPUs
Entry non-blocking pod
One rack-row of rail-aligned leaves over a small spine tier. Two-tier leaf-spine, 1:1 non-blocking.
8 leaves · 4 spines · TH5 · 2 × 400G per GPU
4,096 GPUs
Rail-optimized 2-tier pod
Rail-aligned leaves with a 1:1 non-blocking spine, the largest two-tier build on Tomahawk 5. Each GPU's 800G lands on two 400G leaf ports. Intra-rail AllReduce stays on the leaf; cross-rail traffic uses the spine.
128 leaves · 64 spines · TH5 · 2 × 400G per GPU
16,384 GPUs
Rail-optimized 3-tier fabric
Above 4,096 GPUs, a super-spine plane joins two-tier pods. DLB at every tier; GLB end-to-end in OcNOS 7.1. Sized for the trillion-parameter training class.
512 leaves · 384 spines · 128 super-spines (estimate) · TH5 · 800G

A two-tier leaf-spine reaches 4,096 GPUs at 800G per GPU; a 1,024-GPU build is 32 leaves and 16 spines, 48 switches. Above 4,096 GPUs, add the super-spine tier. Every tier runs OcNOS-DC, in the edition that fits its role, so the fabric scales out with the cluster instead of being redesigned at each step.

Sizing your own cluster? The AI Fabric Design Suite gives a fast first cut: it sizes a non-blocking two-tier leaf-spine pod and flags when you cross into three-tier scale. The reference designs below use the same non-blocking leaf/spine math and add a super-spine tier above 4,096 GPUs, so the switch counts line up with the tool. Rail-optimized here is the wiring discipline of an 8-NIC GPU server (one rail per leaf, so intra-rail AllReduce stays on the leaf) layered on that non-blocking fabric: it changes traffic locality, not the switch count. Use the tool for a ballpark; use these designs for the build.

Two-tier pod, three-tier fabric, and coherent DCI

Three reference topologies cover single-pod, multi-pod, and multi-site AI fabrics, each sized on HCL-listed Broadcom hardware.

Reference design 1

Rail-optimized single pod

Each GPU server's 8 NICs land on 8 dedicated rail leaves, so same-rail AllReduce stays on one leaf and never traverses the spine. A non-blocking spine carries cross-rail flows. The single-pod scalable unit, up to 4,096 GPUs on Tomahawk 5.

Rail-optimized network
Reference design 2

Three-tier rail-optimized fabric

Leaf, spine, and super-spine tiers take the fabric past 4,096 GPUs, to 16,384 in the reference design. Non-blocking within each pod, with the super-spine plane setting the cross-pod ratio. DLB at every hop, GLB end-to-end in OcNOS 7.1.

GLB fabric-wide balancing
Reference design 3

Coherent multi-DC DCI

When a training run spans data halls, extend the fabric with 400G or 800G ZR/ZR+ coherent optics on the spine. DCI between sites with no transponder or muxponder shelves, while the in-site fabric remains unchanged.

Coherent DCI

Rail-optimized single pod

Each GPU server has 8 NICs, one per rail (a dedicated xCCL (NCCL / RCCL / oneCCL) collective channel). Each rail is its own dedicated leaf, so all 8 NICs on every server land on a different leaf. AllReduce across rail-N stays inside leaf-N, so there is no east-west pressure on the spine for the dominant collective pattern.

Rail-optimized AI fabric topology: four 800G Tomahawk 5 spines above eight rail leaves, with four GPU servers each mapping one of its eight NICs to a different rail leaf, so intra-rail AllReduce stays on one leaf.
Rail-optimized single pod: each GPU server's 8 NICs map one per rail to 8 dedicated leaves, so same-rail AllReduce stays on the leaf and only cross-rail traffic reaches the spine.

OcNOS pieces: BGP-unnumbered L3 underlay, RoCEv2 lossless (PFC + ECN) on every leaf, DLB at the spine tier. Built on HCL-listed hardware: every tier from the 256-GPU pod to the 4,096-GPU two-tier ceiling uses TH5 64x800G leaves and spines (Edgecore AIS800-64D or UfiSpace S9321-64E), run as 128x400G so each GPU lands on two 400G leaf ports. Tomahawk 4 (Edgecore AS9736-64D, 64x400G) fits fabrics with one 400G NIC per GPU.

Three-tier rail-optimized fabric: above 4,096 GPUs

A two-tier rail-optimized pod on Tomahawk 5 tops out at 4,096 GPUs at 800G per GPU, when the spines run out of ports. Above that, the same rail-optimized design adds a third tier: leaf, spine, super-spine. Any two GPUs are at most four switch hops apart; same-leaf and same-pod peers are closer.

The hard part on a Clos is spreading flows evenly so no link becomes a hot spot. Approaches run from per-flow ECMP, through dynamic load balancing, to per-packet spray, the model Ultra Ethernet uses with reordering handled at the NIC. A separate family, cell-based scheduled fabrics such as Broadcom DDC, segments traffic into cells and schedules it inside the fabric. OcNOS keeps the GPU plane balanced with DLB and adds fabric-wide GLB in OcNOS 7.1.

The diagram is schematic: it draws a reduced tier count. At 16,384 GPUs on TH5 800G with a ~2:1 cross-pod plane, the build is about 512 leaves / 384 spines / 128 super-spines, an estimate from Clos port math; exact counts per design, with an IP Infusion engineer.

Three-tier rail-optimized AI fabric topology: a super-spine tier over a spine tier over leaf switches feeding GPU pods, sized to 16,384 GPUs on 800G Tomahawk 5 with DLB at every hop.
Three-tier rail-optimized fabric: leaf, spine, and super-spine tiers scale a non-blocking GPU plane to thousands of GPUs, with DLB at every hop and fabric-wide GLB in OcNOS 7.1.

OcNOS pieces: eBGP-unnumbered L3 underlay, RoCEv2 lossless (PFC + ECN), DLB at every tier, GLB end-to-end in OcNOS 7.1, and gNMI streaming telemetry to your observability stack. Built on HCL-listed TH5 64x800G chassis throughout.

Subscription is a dial, not a fixed rule. These counts keep each pod 1:1 non-blocking and use a cost-optimized ~2:1 super-spine for cross-pod traffic, the rail-optimized approach hyperscale Ethernet fabrics rely on (published large-scale designs oversubscribe the top tier far more, because collective traffic stays pod-local). Want maximal any-to-any headroom instead? A fully non-blocking 1:1 build is 512 / 512 / 256 at 16,384 GPUs; only the spine and super-spine counts change. Model either in the AI Fabric Design Suite.

Multi-DC AI fabric: coherent DCI

When a single training run spans more than one data hall, the fabric extends across the WAN. OcNOS-DC supports coherent optics directly on the spine for DCI across sites without transponder or muxponder shelves: 400G ZR/ZR+ on the Tomahawk 4 and Trident 4 switches, and 800G ZR/ZR+ on the Tomahawk 5 Edgecore AIS800-64D and UfiSpace S9321-64E. The fabric inside each site is unchanged.

Multi-data-center AI fabric topology: two leaf-spine sites joined by 400G ZR/ZR+ coherent optics on the spine, extending the fabric across the WAN, with no transponder or muxponder shelves.
Coherent multi-DC DCI: two AI data centers joined by 400G or 800G ZR/ZR+ optics on the spine, extending the fabric across sites without transponder or muxponder shelves.

OcNOS pieces: 400G or 800G ZR/ZR+ pluggable coherent optics on a DWDM-capable spine or border-leaf port, with gNMI telemetry across sites. No transponder, muxponder or regenerator shelves required (regenerators within ZR/ZR+ reach). Reach: 400ZR to roughly 120 km amplified; OpenZR+ reaches farther with oFEC.

Design rules of thumb

A handful of rules keep an AI fabric on the right side of the performance and cost curve, whichever reference design you build from.

  • Match the topology to the GPU count. Smallest pods (under one leaf's NIC radix): rail-only is enough. Single-pod scale, up to 4,096 GPUs on Tomahawk 5: rail-optimized leaf-spine. Above that: add a super-spine tier to the same rail-optimized design, with the pod non-blocking and the cross-pod ratio set to the workload.
  • Always 1:1 subscription on the AI plane. Storage and CPU racks can run higher oversubscription ratios. The GPU plane should not.
  • Plan the rail count from xCCL, not from cabling convenience. 8 rails is the current de-facto standard for 8-NIC GPU servers. Do not combine rails into fewer leaves.
  • Pick the silicon by power and density, not the badge. TH4 (25.6T) and TH5 (51.2T) are the workhorses; the choice between them is rack power and breakout-cable cost.
  • Validate against the HCL. Every reference here is built on hardware listed in the OcNOS Hardware Compatibility List; pick from there for first-class support.
RoCEv2 lossless fabric What is RDMA DLB: dynamic load balancing GLB: fabric-wide (7.1) Ultra Ethernet (UEC) EVPN-VXLAN BGP underlay OcNOS feature matrix Qualified hardware (HCL) OcNOS-DC AI Fabric solution

AI fabric topology FAQ

What is a rail-optimized topology, and how is it different from rail-only?
Rail-optimized wiring connects each of a GPU server’s 8 NICs to its own dedicated rail leaf, so the dominant same-rail AllReduce traffic stays on one leaf and never traverses the spine. Rail-only is the small-cluster case: a single rack-row of rail-aligned leaves with no spine tier, where cross-rail traffic relies on the GPU scale-up domain. Rail-optimized adds a non-blocking spine so cross-rail flows have a network path.
When does an AI fabric need a third tier?
Only above the two-tier ceiling. On Tomahawk 5 switches, a 2-tier leaf-spine reaches 4,096 GPUs at 800G per GPU, 1:1 non-blocking: each switch runs as 128×400G and each GPU lands on two 400G leaf ports, for 128 leaves and 64 spines. Adding a super-spine tier to the same rail-optimized design extends that up to 16,384 GPUs in the reference designs above. Because most collective traffic stays rail-local, the super-spine plane is sized to the cross-pod ratio you actually need.
Should I use Tomahawk 4 or Tomahawk 5?
Both run OcNOS-DC. Tomahawk 4 (25.6 Tbps, 64×400G) is the cost-optimized choice for entry pods and 400G GPU NICs. Tomahawk 5 (51.2 Tbps, 64×800G) is the workhorse for 800G GPU servers and larger fabrics. Tomahawk 4 has no native 800G, so match the switch to your NIC speed.
Do I need InfiniBand, or is Ethernet enough?
Ethernet is now a first-class AI-fabric transport. RoCEv2 with PFC and ECN delivers lossless RDMA today. Ultra Ethernet (UEC) removes the network-wide PFC dependency using endpoint packet-spray, selective retransmission, and link-level retry on UEC-capable NICs. OcNOS-DC runs the RoCEv2 fabric today, and IP Infusion is a member of the Ultra Ethernet Consortium.
Where does scale-up end and scale-out begin?
Inside a GPU server and its NVLink domain (for example GB200 NVL72), GPUs communicate over the scale-up fabric at terabit speeds. The rail, leaf-spine, and Clos network is the scale-out fabric between servers and pods. Collective libraries use the scale-up domain to move data onto the destination rail first, so the network mostly carries same-rail and cross-pod traffic, which is why 1:1 non-blocking matters most on the GPU plane.

Go deeper. Take it with you.

The product datasheet plus short, technical downloads that go further than this page.

Datasheet

OcNOS-DC Datasheet

Full OcNOS-DC specification: the EVPN-VXLAN and Ethernet for AI feature set, software SKUs, supported hardware platforms, and the solution ordering guide.

Get the datasheet
Solution Brief

OcNOS 800G Lossless AI Fabric

Lossless RoCEv2 fabric on Tomahawk 5 spines: SKU tiers, qualified platforms, and deployment architecture.

Get the brief
Solution Brief

EVPN-VXLAN Data Center Fabric

Leaf-spine data center fabric on EVPN-VXLAN: a VRF and VNIs per tenant, EVPN multihoming, BGP unnumbered and route-target filtering.

Get the brief

Design the whole AI fabric with OcNOS

From the business case to the port-count maths, pick up wherever you are in the build.

Solution Open AI Fabric The complete 800G AI fabric: open switches, OcNOS-DC, and support under one contract.
You are here AI Fabric Topologies Rail-optimized two-tier and three-tier fabrics, and coherent DCI, sized in real port counts.
Size & build AI Fabric Design Suite Size a GPU fabric: leaf, spine, and super-spine counts with a component and power summary.
New to AI fabrics? Start here What is an AI fabric? What is a GPU fabric? What is lossless Ethernet? What is RDMA? AI networking switches
The technology inside RoCEv2 lossless Rail-optimized network DLB GLB (7.1) Ultra Ethernet Tomahawk 5 Tomahawk 6 at 1.6T Broadcom silicon DCQCN PFC deadlock InfiniBand vs Ethernet RoCE vs InfiniBand AI fabric architecture Coherent DCI
Evaluate the fabric How to choose an AI networking fabric AI fabric proof points Neocloud networking

Designing your AI fabric? We'll do the port-count maths with you.

Tell us the workload and the GPU scale, and an IP Infusion engineer will size the leaf, spine, and super-spine tiers with you, or start with a first-pass layout in the AI Fabric Design Suite.

Book an architecture review Open the AI fabric design suite