Home · Documentation · Technology · What Is a GPU Fabric
Networking Definition

What Is a GPU Fabric?

A GPU fabric is the back-end network that interconnects GPUs across servers and pods for training and inference. It is the scale-out network beyond the in-server scale-up domain, and it carries the RDMA traffic GPUs exchange.

A GPU fabric is the back-end network that interconnects GPUs across servers and pods so they can work together on one training or inference job. It carries the RDMA traffic GPUs exchange during collective operations, separate from the front-end network used for management and storage. It is the scale-out network that begins where the in-server scale-up domain (such as NVLink) ends, built from rail-optimized and Clos topologies running lossless RoCEv2. OcNOS-DC provides this fabric on Broadcom Tomahawk 5 hardware.

Scale-up ends, scale-out begins

A GPU fabric sits on one side of a clear boundary.

Inside the server

Scale-up domain

The very high bandwidth interconnect inside a GPU server and its tightly coupled domain, for example NVLink within an NVL72 rack. GPUs here talk at terabit speeds as if they were one large accelerator. This is not the GPU fabric.

Between servers

Scale-out GPU fabric

The Ethernet network between servers and pods: rail, leaf, spine, and Clos tiers. This is the GPU fabric. A collective uses scale-up first inside the domain, then this fabric carries the cross-server remainder.

Getting this boundary right matters for sizing. Because same-rail traffic is often absorbed by scale-up first, the GPU fabric carries the cross-rail and cross-pod remainder, which is why 1:1 non-blocking is worth paying for on the GPU plane specifically and not on storage or CPU racks.

How the fabric is shaped: rails and Clos

Most GPU fabrics are rail-optimized leaf-spine networks: each of a GPU server’s 8 NICs connects to its own dedicated rail leaf, so the dominant same-rail AllReduce traffic stays on one leaf and never crosses the spine. Past a single pod, the design extends to a 3-stage Clos (leaf, spine, super-spine). On radix-64 Tomahawk 5 switches, a 2-tier fabric reaches roughly 2,048 GPUs at 1:1 non-blocking and a 3-stage Clos scales to 16,000+ GPUs, up to about 65,535 at the fat-tree limit. See rail-optimized network and AI fabric topologies for the port maths.

Lossless RoCEv2 on the GPU plane

GPUs move data with RDMA over RoCEv2, which performs poorly under packet loss. So the GPU fabric is engineered lossless with Priority Flow Control and ECN, and kept evenly loaded with dynamic load balancing so no link becomes a hot spot during a collective. OcNOS-DC supplies PFC (including over L3), ECN, Dynamic ECN, and DLB on Broadcom Tomahawk 5 (51.2 Tbps, 64x800G) platforms such as the Edgecore AIS800-64D and UfiSpace S9321-64E. That no-drop, evenly-loaded behavior is what keeps a training step from stalling.

Rail-optimized network AI fabric topologies RoCEv2 lossless fabric What is an AI fabric AI Fabric solution OcNOS-DC

Standing up a GPU fabric? Let's size the plane together.

Book an Architecture Review →
FAQ

What Is a GPU Fabric FAQ

What is a GPU fabric?
A GPU fabric is the back-end network that interconnects GPUs across servers and pods so they can work together on a single training or inference job. It carries the RDMA traffic GPUs exchange during collective operations, and it is separate from the front-end network used for management and storage. In practice a GPU fabric is a leaf-spine (Clos) Ethernet network, usually rail-optimized, running lossless RoCEv2. OcNOS-DC provides it on Broadcom Tomahawk 5 hardware.
What is the difference between scale-up and scale-out?
Scale-up is the very high bandwidth interconnect inside a GPU server and its tightly coupled domain, for example NVLink within an NVL72 rack, where GPUs talk at terabit speeds as if they were one large accelerator. Scale-out is the network between servers and pods, the GPU fabric this page describes, built from rail, leaf, spine, and Clos tiers over Ethernet. A collective first uses scale-up inside the domain, then the scale-out GPU fabric carries the remainder between servers.
What topology does a GPU fabric use?
Most GPU fabrics use a rail-optimized leaf-spine design: each of a GPU server’s 8 NICs connects to its own dedicated rail leaf, so the dominant same-rail AllReduce traffic stays on one leaf. As the cluster grows past a single pod, the design extends to a 3-stage Clos with a super-spine tier. On radix-64 Tomahawk 5, a 2-tier fabric reaches about 2,048 GPUs at 1:1 non-blocking and a 3-stage Clos scales to 16,000+ GPUs. See the rail-optimized network and AI fabric topologies pages.
Does a GPU fabric have to be lossless?
Yes, on the GPU plane. GPUs move data with RDMA over RoCEv2, which performs poorly under packet loss, so the GPU fabric is engineered lossless with Priority Flow Control and ECN, and kept evenly loaded with dynamic load balancing. That no-drop behavior is what keeps a collective from stalling. OcNOS-DC supplies PFC, ECN, Dynamic ECN, and DLB on Broadcom Tomahawk 5 for exactly this.
AI Fabric

Design the whole AI fabric with OcNOS

From the business case to the port-count maths, pick up wherever you are in the build.

Solution Open AI Fabric The complete 800G AI fabric: open switches, OcNOS-DC, and support under one contract. Reference designs AI Fabric Topologies Rail-optimized, scheduled 3-stage Clos, and coherent DCI, sized in real port counts. Size & build AI Fabric Design Suite Size a GPU fabric: leaf, spine, and super-spine counts with a component and power summary.
New to AI fabrics? Start here What is an AI fabric? What is a GPU fabric? What is lossless Ethernet? What is RDMA?
The technology inside RoCEv2 lossless Rail-optimized network DLB adaptive routing GLB (7.1) Ultra Ethernet DCQCN PFC deadlock InfiniBand vs Ethernet RoCE vs InfiniBand AI fabric architecture Coherent DCI
Datasheet and Solution Briefs

Go deeper. Take it with you.

Two short, technical downloads that go further than this page: the full OcNOS-DC datasheet and the lossless 800G AI fabric architecture.

Datasheet

OcNOS-DC Datasheet

Full OcNOS-DC specification: the EVPN-VXLAN and Ethernet for AI feature set, software SKUs, supported hardware platforms, and the solution ordering guide.

Get the datasheet
Solution Brief

OcNOS 800G Lossless AI Fabric

Non-blocking RoCEv2 fabric on Broadcom Tomahawk 4/5 spines: SKU tiers, validated platforms, and deployment architecture.

Get the brief