AI data center fabric architecture
An AI data center runs four network planes, not one flat network: an AI back-end fabric for GPU-to-GPU collectives, a storage fabric, a front-end fabric, and an isolated out-of-band management plane. Coherent DCI extends training across sites, and OcNOS-DC runs every plane on a single NOS over qualified Broadcom Tomahawk hardware.
Design the planes, not just the switches
Most AI fabric mistakes are scoping mistakes: teams size the GPU plane, then bolt storage, tenant access, and management onto it as an afterthought.
Separate traffic by plane instead, and give each the subscription ratio and isolation it needs. Only the AI back-end fabric has to be non-blocking and lossless; the other three exist to keep traffic off it that does not belong. All four planes, plus the coherent links that join sites, run on OcNOS-DC and HCL-listed Tomahawk and Trident hardware.
Each plane has one job
Size each plane for its own job, keep its traffic on its own wires, and the fabric stays predictable under a full training run.
AI back-end fabric
The GPU-to-GPU plane that carries collectives. Built 1:1 non-blocking as a rail-optimized leaf-spine pod, or three tiers with a super-spine, on a pure Layer 3 eBGP underlay, and kept lossless with the RoCEv2 fabric RDMA requires: PFC (including over L3, with DCBX/LLDP) and ECN. DLB spreads flows so no link becomes a hot spot during AllReduce.
Storage fabric
A leaf-spine Clos that moves checkpoints and datasets between the GPU racks and NVMe-oF or NFS storage. Typically sized near 3:1, since storage bursts tolerate more oversubscription than collectives, and kept lossless so RDMA storage traffic does not drop under load.
Front-end fabric
The north-south plane for tenant access, inference serving, and the path out to the rest of the data center and the internet. Runs 400G or 800G access, sized to the service traffic it carries rather than to the collective pattern of the back-end plane.
Out-of-band management
A separate Clos or MLAG plane on its own wires, so bring-up and monitoring never share fate with the data planes. Carries ZTP, SNMP and syslog, and gNMI/OpenConfig streaming telemetry, and keeps working when a data-plane fabric is being reconfigured.
Four fabrics plus DCI, on one NOS
One data center, four planes, extended to a second site over coherent optics. The AI back-end plane is 1:1 non-blocking; storage, front-end, and management each run at the ratio their traffic needs. Every plane runs OcNOS-DC, and the ZR+ DCI link scales training across sites without transponder or muxponder shelves.
OcNOS pieces: a routed eBGP-unnumbered underlay per plane (the AI back-end is pure Layer 3 eBGP, with EVPN-VXLAN kept as a tenant overlay rather than the AI underlay), RoCEv2 lossless (PFC + ECN, PFC over L3 with DCBX/LLDP) and DLB on the back-end and storage planes, an isolated management plane for ZTP and gNMI telemetry, and 400G ZR/ZR+ or 800G ZR coherent optics for DCI on the AIS800-64D, S9321-64E and AS9736-64D. The back-end planes are built on HCL-listed Tomahawk 5 (Edgecore AIS800-64D, UfiSpace S9321-64E / 64EO) and Tomahawk 4 (Edgecore AS9736-64D) hardware; the management plane runs the OcNOS-DC MGMT image on Trident 3 switches.
Coherent DCI, no transponder or muxponder shelves
When a single training run outgrows one data hall, the fabric extends across the WAN on 400G ZR+ or 800G ZR coherent pluggable optics, straight off a border port.
On OcNOS-DC, 800G ZR optics run on the Tomahawk 5 Edgecore AIS800-64D and UfiSpace S9321-64E, and 400G ZR/ZR+ optics on the Tomahawk 4 Edgecore AS9736-64D. 400ZR covers metro distances up to roughly 120 km, and OpenZR+ with oFEC reaches farther, while each site keeps its own leaf-spine fabric unchanged. See the coherent DCI deep-dive for optics and reach detail.
What hardware runs each plane
The AI back-end, storage and front-end planes run OcNOS-DC on Broadcom Tomahawk silicon. The 800G planes and DCI ride Tomahawk 5; entry and 400G planes ride Tomahawk 4. Both are on-chip shared-buffer switches. The out-of-band management plane runs the OcNOS-DC MGMT image on Trident 3 1G/10G switches, and every platform is on the OcNOS Hardware Compatibility List.
| Silicon layer | Tomahawk 5800G planes + DCI | Tomahawk 4entry + 400G planes |
|---|---|---|
| Broadcom part | BCM78900 | BCM56990 |
| Switch capacity | 51.2 Tbps | 25.6 Tbps |
| Port configuration | 64×800G | 64×400G |
| Buffer | On-chip shared buffer; HCL-listed. | On-chip shared buffer; HCL-listed. |
| Platforms | Edgecore AIS800-64D, UfiSpace S9321-64E, UfiSpace S9321-64EO. 800G ZR optics on the AIS800-64D and S9321-64E. | Edgecore AS9736-64D. |
| Role in the design | AI back-end and storage planes at 800G, plus coherent DCI across sites. | Entry builds and 400G front-end or management planes. |
The power of one NOS
The four planes and the DCI links are not four products. They are one operating system in four roles: OcNOS-DC runs the AI back-end, storage, front-end, and out-of-band management fabrics, plus coherent DCI, with one configuration model. The team learns one CLI, one automation surface, and one set of counters for every plane.
One operational model
The same routing, QoS, and telemetry model applies whether a port is a GPU-facing back-end leaf, a storage leaf, a front-end border, or a management switch.
One support contract
When you buy the switches as OcNOS Systems, one IP Infusion contract covers the software and the hardware, with one TAC and one SLA across every plane. For hardware you buy yourself, IP Infusion supports the software and the manufacturer supports the hardware.
One hardware list
Every plane is built from the same OcNOS Hardware Compatibility List, so support, optics, and firmware stay consistent across the design.
One roadmap to grow into
Fabric-wide GLB is coming in OcNOS 7.1. Upcoming Tomahawk 6 silicon (BCM78910 / BCM78914, 102.4 Tbps, TSMC 3nm) rides the OcNOS 7.2 train and extends the same design at higher radix.
AI fabric architecture FAQ
What fabrics make up an AI data center?
What oversubscription should each fabric use?
How do you connect two AI data centers?
Can one NOS run all the fabrics?
What hardware runs each plane?
Go deeper. Take it with you.
The product datasheet plus short, technical downloads that go further than this page.
OcNOS-DC Datasheet
Full OcNOS-DC specification: the EVPN-VXLAN and Ethernet for AI feature set, software SKUs, supported hardware platforms, and the solution ordering guide.
Get the datasheet
OcNOS 800G Lossless AI Fabric
Lossless RoCEv2 fabric on Tomahawk 5 spines: SKU tiers, qualified platforms, and deployment architecture.
Get the brief
EVPN-VXLAN Data Center Fabric
Leaf-spine data center fabric on EVPN-VXLAN: a VRF and VNIs per tenant, EVPN multihoming, BGP unnumbered and route-target filtering.
Get the briefOcNOS-DC Datasheet
Quick form. Your PDF opens in a new tab immediately after submit.
✓ Opening your PDF in a new tab…
If it didn't open, use the link below.
OcNOS 800G Lossless AI Fabric
Quick form. Your PDF opens in a new tab immediately after submit.
✓ Opening your PDF in a new tab…
If it didn't open, use the link below.
EVPN-VXLAN Data Center Fabric
Quick form. Your PDF opens in a new tab immediately after submit.
✓ Opening your PDF in a new tab…
If it didn't open, use the link below.
Design the whole AI fabric with OcNOS
From the business case to the port-count maths, pick up wherever you are in the build.
Designing the full AI data center? We will plan every plane with you.
Tell us the GPU scale and the workload, and an IP Infusion engineer will size the back-end, storage, front-end, management, and DCI planes with you, or start with a first-pass layout in the AI Fabric Design Suite.