Networking Definition · Clos · EVPN-VXLAN

What Is an IP Fabric?

An IP fabric is a data centre network built as a routed leaf-spine Clos topology: every switch-to-switch link is a Layer 3 link, all paths between any two leaves carry traffic at once, and there is no spanning tree. The underlay is a routing protocol, usually eBGP following RFC 7938. Where a workload still needs Layer 2, an EVPN-VXLAN overlay provides it on top of the routed fabric. OcNOS-DC builds this fabric on open Broadcom Trident and Tomahawk switches.

See the DC fabric solution How EVPN-VXLAN works
Layer 3on every fabric link
ECMPall paths forward at once
RFC 7938eBGP underlay design
EVPN-VXLANoverlay where L2 is needed
Why it is routed

The fabric that replaced spanning tree

The data centre networks of the 2000s were Layer 2 trees. One broadcast domain stretched across access, aggregation, and core switches, and spanning tree kept it loop-free by blocking every redundant link. Half the fabric sat idle, a topology change touched every switch, and the largest fault domain was the whole network. The IP fabric removes all three problems by making every link a routed link.

  • Every path forwards. With a Layer 3 adjacency on each leaf-spine link, equal-cost multipath spreads traffic across every spine at once. Adding a spine adds capacity linearly.
  • Failures stay small. A failed link or spine is withdrawn by the routing protocol and the leaves reconverge around it. No broadcast domain, no spanning-tree recalculation, no fabric-wide flush.
  • It scales sideways. Leaves are added for ports, spines for bandwidth, and a third tier of super-spines when one pod is full. The protocol and the design do not change with size.
How it is built

Topology, underlay, overlay

An IP fabric is three decisions stacked on each other. The topology says how switches connect, the underlay says how they route between themselves, and the overlay, if there is one, says how tenant traffic rides across.

Topology

Leaf-spine Clos

Servers connect to leaves; every leaf connects to every spine; spines connect to nothing but leaves. Any two servers are the same number of hops apart. A second pod adds a super-spine tier above the spines.

DC Clos fabric solution
Underlay

eBGP, one AS per switch

RFC 7938 puts eBGP on every link, with each leaf in its own private autonomous system and the spines sharing one. BGP unnumbered (RFC 8950) brings sessions up over IPv6 link-local addresses, so no link needs an IPv4 subnet. OcNOS supports BGP unnumbered with EVPN-VXLAN since OcNOS 6.0.

BGP and unnumbered peering
Overlay

EVPN-VXLAN when L2 is needed

BGP EVPN learns MAC and IP reachability, VXLAN carries frames across the routed fabric, and IRB routes between subnets at the leaf. Tenants get a Layer 2 segment without the fabric giving up its routed underlay.

EVPN-VXLAN explained
With or without an overlay

When the underlay is enough, and when it is not

Not every IP fabric needs EVPN. The question is whether anything on the fabric needs a Layer 2 segment, tenant isolation, or an address that survives a move.

NeedUnderlay onlyWith EVPN-VXLAN overlay
Server to server routingYes, nativeYes
Layer 2 between racksNoYes, per VNI
VM or container mobilityNo, address is bound to the leafYes, MAC and IP advertised in BGP
Tenant separationVRF onlyVRF plus per-tenant L2 and L3 VNIs, inter-VRF leaking
First-hop gatewayOn the leafDistributed anycast gateway on every leaf (IRB)
Dual-homed serversRouted, per linkEVPN multi-homing with an Ethernet Segment, all-active
Typical useAI training back-end, storage, HPCEnterprise, cloud, multi-tenant, VM-heavy
  • OcNOS-DC overlay capabilities in the Feature Matrix: Layer 2 EVPN for VXLAN (RFC 7348, 7432, 8365), EVPN multi-homing, IRB (RFC 9135) since 4.1, Type-5 prefix routes (RFC 9136) since 4.1, inter-VRF route leaking since 4.2, overlay ECMP since 5.0, DHCP relay on IRB since 5.0, multiple anycast gateway addresses since 6.3.1, BFD over IRB since 6.6.1, and Layer 3 gateway stitching since 7.0.
  • Lossless transport on the same fabric: PFC and ECN over VXLAN since OcNOS 7.0, so an overlay fabric can also carry RoCEv2 storage or inference traffic.
Where this fits

One fabric design, three workloads

The same routed Clos serves very different traffic. What changes is the overlay decision and the transport tuning, not the fabric.

Enterprise and cloud

Multi-tenant fabric

EVPN-VXLAN overlay, IRB with a distributed anycast gateway, per-tenant VRFs, and EVPN multi-homing for dual-attached servers.

Multi-tenant fabric
AI training

AI back-end fabric

Usually no overlay. Rail-optimized leaves, a scheduled Clos spine, and lossless RoCEv2 with PFC and ECN, because job completion time is set by the slowest flow.

What is an AI fabric
Between sites

Data centre interconnect

Type-5 routes and Layer 3 gateway stitching carry tenant prefixes between fabrics. Over distance, 400G ZR and ZR+ coherent optics in the router port remove the transponder.

Coherent DCI

What Is an IP Fabric FAQ

What is an IP fabric?
An IP fabric is a data centre network built as a routed leaf-spine topology. Every link between a leaf and a spine is a Layer 3 link with its own routing adjacency, so all paths between any two leaves are used at once through equal-cost multipath, and there is no spanning tree to block links. Layer 2 services, where a workload still needs one, run as an EVPN-VXLAN overlay on top of the routed fabric. OcNOS-DC builds this fabric on open Broadcom Trident and Tomahawk switches.
Is an IP fabric the same as a Clos fabric?
Nearly. Clos describes the topology: a multi-stage arrangement in which every leaf connects to every spine, first described by Charles Clos for telephone switching. IP fabric describes how that topology is run: routed at Layer 3 on every link. Almost every Clos fabric built today is an IP fabric, so the terms are used interchangeably in practice.
What is the difference between an IP fabric and a Layer 2 fabric?
A Layer 2 fabric extends one broadcast domain across the switches and relies on spanning tree or a proprietary multi-chassis protocol to avoid loops, which leaves links idle and makes failures wide. An IP fabric routes on every link, so every path forwards, failures are contained by the routing protocol, and the fabric scales by adding spines. Where a Layer 2 segment is genuinely needed, EVPN-VXLAN provides it as an overlay without giving up the routed underlay.
Which routing protocol runs the underlay of an IP fabric?
Usually eBGP, following RFC 7938, with each leaf in its own private autonomous system, the spines sharing one, and sessions brought up over IPv6 link-local addresses using BGP unnumbered (RFC 8950), so no per-link addressing is needed. An IGP such as OSPF or IS-IS also works and is common in smaller fabrics. OcNOS supports both, including BGP unnumbered with EVPN-VXLAN since OcNOS 6.0.
Do I need EVPN-VXLAN on an IP fabric?
Only if workloads need Layer 2 adjacency, tenant separation, or a virtual machine to keep its address when it moves. A fabric that only routes between servers can stop at the underlay. EVPN-VXLAN adds the overlay: BGP learns MAC and IP addresses, VXLAN carries the frames across the routed fabric, and IRB routes between subnets at the leaf. Most enterprise and multi-tenant fabrics run it; many AI training fabrics do not.
How many tiers does an IP fabric have?
Two tiers for a single pod: leaves that connect servers and spines that connect leaves, which form a three-stage Clos. When one pod cannot hold the ports, a third tier of super-spines connects several pods into a five-stage Clos. The tier count is set by port count and oversubscription, not by the protocol; the same BGP underlay and EVPN overlay run at every scale.
Is an AI fabric an IP fabric?
Yes, in construction. An AI back-end fabric is a Clos IP fabric tuned for one traffic pattern: many GPUs sending large flows at once. It adds lossless transport (RoCEv2 with PFC and ECN) and often a rail-optimized leaf layer, and it usually runs without an EVPN overlay. The general IP fabric described here is what the AI fabric specialises.

Designing an IP fabric? Start from the port count.

Tell us the server count, the oversubscription you can accept, and whether tenants need Layer 2. An IP Infusion engineer will size the leaf, spine, and super-spine tiers on open Broadcom hardware running OcNOS-DC.

Book an architecture review See qualified switches
Datasheet and Solution Briefs

Go deeper. Take it with you.

Two short, technical downloads that go further than this page: the full OcNOS-DC datasheet and the EVPN-VXLAN data center reference.

Datasheet

OcNOS-DC Datasheet

Full OcNOS-DC specification: the EVPN-VXLAN and Ethernet for AI feature set, software SKUs, supported hardware platforms, and the solution ordering guide.

Get the datasheet
Solution Brief

EVPN-VXLAN Data Center Fabric

Carrier-grade leaf-spine data center fabric: symmetric IRB, Type-2/Type-5 routes, and distributed anycast gateway.

Get the brief