Networking Definition

What Is an AI Fabric?

In networking, an AI fabric is the GPU back-end network that carries RDMA traffic between GPUs during training and inference. It is a scale-out fabric of leaf and spine switches, and its speed decides how fast your GPUs finish a job.

An AI fabric is the GPU back-end (scale-out) network that carries RDMA traffic between GPUs during distributed training and inference. It connects every GPU NIC across servers and pods using leaf and spine switches in rail-optimized and Clos topologies, and it runs a lossless Ethernet transport (RoCEv2 with PFC and ECN) so no packet is dropped under load. Because GPUs exchange data in lockstep collectives, the fabric's tail latency sets job completion time. Note the word overlap: this is the networking sense, not the enterprise "data fabric" used in data management. OcNOS-DC provides this fabric on Broadcom Tomahawk 5 hardware.

The networking sense, not "data fabric"

Two different fields use the word "fabric". A data fabric is a software architecture for unifying and governing data across an enterprise. An AIファブリック, the subject of this page, is a physical network: the wires, switches, and transport that move RDMA traffic between GPUs. When an AI infrastructure team says "the fabric was the bottleneck", they mean this network, the one that decided how long the training run took.

How the fabric is built: rails and Clos

An AI fabric is a leaf-spine (Clos) network tuned for one traffic pattern: many GPUs sending large flows to each other at once. The common building block is rail-optimized wiring, where each of a GPU server's 8 NICs connects to its own dedicated rail leaf, so the dominant same-rail AllReduce traffic stays on one leaf and never touches the spine. As the cluster grows past a single pod, the design extends to a 3ステージClos (leaf, spine, super-spine). On radix-64 Tomahawk 5 switches, a 2-tier fabric reaches roughly 2,048 GPUs at 1:1 non-blocking, and a 3-stage Clos scales to 16,000+ GPUs. See the AIファブリックトポロジー and rail-optimized network pages for the port maths.

Why it has to be lossless

GPUs move data with RDMA, which performs poorly when packets are dropped. So the AI fabric is engineered to be lossless: Priority Flow Control (PFC) stops a switch queue from overflowing, and Explicit Congestion Notification (ECN) marks packets early so the sending NIC slows down before loss happens. That combination is what carries RoCEv2, the transport GPUs use over Ethernet. OcNOS-DC ships this fabric with PFC (including over L3), ECN, Dynamic ECN, WRED, and dynamic load balancing to keep every path evenly loaded.

Where this fits

  • Ethernet is enough. RoCEv2 with PFC and ECN delivers lossless RDMA on standard, multi-vendor hardware. Meta has published a 24,000-GPU training cluster built on Ethernet.
  • The switch sets the ceiling. OcNOS-DC runs on Broadcom Tomahawk 5 (51.2 Tbps, 64x800G) platforms such as the Edgecore AIS800-64D and UfiSpace S9321-64E, the density an 800G GPU fabric needs.
  • Design by GPU count. Rail-optimized single pod up to roughly 1,000 GPUs, then a 3-stage Clos to 16,000+ and up to about 65,535 at the fat-tree limit.

Building an AI fabric? Let's size it together.

アーキテクチャレビューを予約 →
よくある質問

What Is an AI Fabric FAQ

AIファブリックとは?
In networking, an AI fabric is the GPU back-end (scale-out) network that carries RDMA traffic between GPUs during distributed training and inference. It connects the GPU NICs across every server and pod so a collective operation such as AllReduce completes as fast as the slowest link allows. It is built from leaf and spine switches in rail-optimized and Clos topologies, and it runs a lossless Ethernet transport (RoCEv2 with PFC and ECN). OcNOS-DC provides that fabric on Broadcom Tomahawk 5 hardware.
Is an AI fabric the same as a data fabric?
No. They are different terms that share a word. A data fabric is an enterprise data-management architecture for unifying and governing data across systems. An AI fabric, in the networking sense used here, is a physical GPU back-end network that moves RDMA traffic between GPUs. This page is about the networking meaning.
Why does the AI fabric decide job completion time?
GPU training runs in lockstep. After each step, GPUs exchange gradients in a collective, and every GPU waits for that exchange to finish before the next step starts. If one link stalls or drops a packet, the whole job waits. Because the fabric carries every one of those exchanges, its tail latency, not its average, sets how long a training run takes, which is why the fabric is engineered to stay lossless and evenly loaded.
Do I need InfiniBand, or is Ethernet enough for an AI fabric?
Ethernet is a first-class AI-fabric transport today. RoCEv2 with PFC and ECN delivers lossless RDMA on standard, multi-vendor hardware, and Meta has published a 24,000-GPU training cluster built on Ethernet. OcNOS-DC runs that RoCEv2 fabric today and aligns with the Ultra Ethernet Consortium 1.0 fabric profile for the next generation of endpoints.