Home · Documentazione · Tecnologia · What Is a GPU Fabric
Networking Definition

What Is a GPU Fabric?

A GPU fabric is the back-end network that interconnects GPUs across servers and pods for training and inference. It is the scale-out network beyond the in-server scale-up domain, and it carries the RDMA traffic GPUs exchange.

A GPU fabric is the back-end network that interconnects GPUs across servers and pods so they can work together on one training or inference job. It carries the RDMA traffic GPUs exchange during collective operations, separate from the front-end network used for management and storage. It is the scale-out network that begins where the in-server scale-up domain (such as NVLink) ends, built from rail-optimized and Clos topologies running lossless RoCEv2. OcNOS-DC provides this fabric on Broadcom Tomahawk 5 hardware.

Scale-up ends, scale-out begins

A GPU fabric sits on one side of a clear boundary.

Inside the server

Scale-up domain

The very high bandwidth interconnect inside a GPU server and its tightly coupled domain, for example NVLink within an NVL72 rack. GPUs here talk at terabit speeds as if they were one large accelerator. This is not the GPU fabric.

Between servers

Scale-out GPU fabric

The Ethernet network between servers and pods: rail, leaf, spine, and Clos tiers. This is the GPU fabric. A collective uses scale-up first inside the domain, then this fabric carries the cross-server remainder.

Getting this boundary right matters for sizing. Because same-rail traffic is often absorbed by scale-up first, the GPU fabric carries the cross-rail and cross-pod remainder, which is why 1:1 non-blocking is worth paying for on the GPU plane specifically and not on storage or CPU racks.

How the fabric is shaped: rails and Clos

Most GPU fabrics are rail-optimized leaf-spine networks: each of a GPU server's 8 NICs connects to its own dedicated rail leaf, so the dominant same-rail AllReduce traffic stays on one leaf and never crosses the spine. Past a single pod, the design extends to a Clos a 3 stadi (leaf, spine, super-spine). On radix-64 Tomahawk 5 switches, a 2-tier fabric reaches roughly 2,048 GPUs at 1:1 non-blocking and a 3-stage Clos scales to 16,000+ GPUs, up to about 65,535 at the fat-tree limit. See rail-optimized network e Topologie AI fabric for the port maths.

Lossless RoCEv2 on the GPU plane

GPUs move data with RDMA over RoCEv2, which performs poorly under packet loss. So the GPU fabric is engineered lossless with Priority Flow Control and ECN, and kept evenly loaded with dynamic load balancing so no link becomes a hot spot during a collective. OcNOS-DC supplies PFC (including over L3), ECN, Dynamic ECN, and DLB on Broadcom Tomahawk 5 (51.2 Tbps, 64x800G) platforms such as the Edgecore AIS800-64D and UfiSpace S9321-64E. That no-drop, evenly-loaded behavior is what keeps a training step from stalling.

Rail-optimized network Topologie AI fabric Fabric lossless RoCEv2 What is an AI fabric Soluzione AI fabric OcNOS-DC

Standing up a GPU fabric? Let's size the plane together.

Prenoti una Architecture Review →
FAQ

What Is a GPU Fabric FAQ

Cos'è una GPU fabric?
A GPU fabric is the back-end network that interconnects GPUs across servers and pods so they can work together on a single training or inference job. It carries the RDMA traffic GPUs exchange during collective operations, and it is separate from the front-end network used for management and storage. In practice a GPU fabric is a leaf-spine (Clos) Ethernet network, usually rail-optimized, running lossless RoCEv2. OcNOS-DC provides it on Broadcom Tomahawk 5 hardware.
What is the difference between scale-up and scale-out?
Scale-up is the very high bandwidth interconnect inside a GPU server and its tightly coupled domain, for example NVLink within an NVL72 rack, where GPUs talk at terabit speeds as if they were one large accelerator. Scale-out is the network between servers and pods, the GPU fabric this page describes, built from rail, leaf, spine, and Clos tiers over Ethernet. A collective first uses scale-up inside the domain, then the scale-out GPU fabric carries the remainder between servers.
What topology does a GPU fabric use?
Most GPU fabrics use a rail-optimized leaf-spine design: each of a GPU server's 8 NICs connects to its own dedicated rail leaf, so the dominant same-rail AllReduce traffic stays on one leaf. As the cluster grows past a single pod, the design extends to a 3-stage Clos with a super-spine tier. On radix-64 Tomahawk 5, a 2-tier fabric reaches about 2,048 GPUs at 1:1 non-blocking and a 3-stage Clos scales to 16,000+ GPUs. See the rail-optimized network and AI fabric topologies pages.
Does a GPU fabric have to be lossless?
Yes, on the GPU plane. GPUs move data with RDMA over RoCEv2, which performs poorly under packet loss, so the GPU fabric is engineered lossless with Priority Flow Control and ECN, and kept evenly loaded with dynamic load balancing. That no-drop behavior is what keeps a collective from stalling. OcNOS-DC supplies PFC, ECN, Dynamic ECN, and DLB on Broadcom Tomahawk 5 for exactly this.
AI fabric

Design the whole AI fabric with OcNOS

From the business case to the port-count maths, pick up wherever you are in the build.

Soluzione Open AI Fabric The complete 800G AI fabric: open switches, OcNOS-DC, and support under one contract. Design di riferimento Topologie AI Fabric Rail-optimized, scheduled 3-stage Clos, and coherent DCI, sized in real port counts. Size & build AI Fabric Design Suite Size a GPU fabric: leaf, spine, and super-spine counts with a component and power summary.
Nuovo alle AI fabric? Inizia qui Cos'è una AI fabric? Cos'è una GPU fabric? Cos'è l'Ethernet senza perdite? Cos'è RDMA?
The technology inside RoCEv2 lossless Rail-optimized network DLB adaptive routing GLB (7.1) Ultra Ethernet DCQCN deadlock PFC InfiniBand vs Ethernet RoCE vs InfiniBand AI fabric architecture Coherent DCI
Datasheet e schede soluzione

Approfondite. Portatelo con voi.

Two short, technical downloads that go further than this page: the full OcNOS-DC datasheet and the lossless 800G AI fabric architecture.

Datasheet

Datasheet OcNOS-DC

Specifica completa di OcNOS-DC: il set di funzionalità EVPN-VXLAN e Ethernet for AI, gli SKU software, le piattaforme hardware supportate e la guida all'ordine della soluzione.

Ottieni il datasheet
Solution brief

OcNOS 800G Lossless AI Fabric

Fabric RoCEv2 non bloccante su spine Broadcom Tomahawk 4/5: tier di SKU, piattaforme validate e architettura di deployment.

Scarica il brief