OcNOS-DC  ·  800G  ·  RoCEv2  ·  PFC / ECN / DLB  ·  24/7 SLA

AI Fabric & Lossless RoCEv2

Your GPU cluster is only as fast as the network connecting it. OcNOS-DC delivers a production-grade 800G lossless RoCEv2 fabric on validated open hardware — with the carrier-grade SLA your AI investment demands.

51.2 Tbps
Max switch throughput (TH5)
800G
Native port speed — Broadcom TH5
PFC + ECN
Lossless DCB stack — RoCEv2 ready
24/7
Carrier-grade global SLA
The Problem

One dropped packet stalls every GPU in the job.

RDMA has no retransmission. A single packet drop in an AllReduce restarts the entire collective operation across every GPU in the cluster. Your network must be lossless — or your cluster is running slower than it should be.

OcNOS-DC ships pre-tuned for RoCEv2 on every supported Broadcom ASIC. PFC ECN ETS DCBX DLB — correct from Day 1, on open hardware.

Reference Architecture

800G Spine-Leaf AI Fabric — Lossless RoCEv2

A 3-stage Clos fabric with eBGP unnumbered underlay, ECMP at every tier, and PFC/ECN tuned per priority group. ZTP provisions each rack-level leaf switch automatically at boot.

800G AI fabric topology with full-mesh eBGP and isolated OOB management Horizontal 800G AI fabric. Three GPU racks on the left feed two leaf VTEPs running OcNOS-DC, which connect to two 51.2 Tbps spines over a full-mesh eBGP ECMP underlay with DLB. An isolated out-of-band management bus across the top carries ZTP and telemetry. Leaf-attached NVMe-oF/NFS GPU storage sits to the right. ISOLATED OOB MANAGEMENT BUS OOB Mgmt Isolated Network ZTP · Telemetry GPU Rack 1 8× GPU nodes RoCEv2 / RDMA GPU Rack 2 8× GPU nodes RoCEv2 / RDMA GPU Rack 3 8× GPU nodes RoCEv2 / RDMA Leaf-01 OcNOS-DC 64 × 400G Tomahawk 4 PFC / DCBX / ZTP LOSSLESS RoCEv2 MLAG PEER Leaf-02 OcNOS-DC 32 × 400G Tomahawk 3 PFC / DCBX / ZTP LOSSLESS RoCEv2 eBGP ECMP Full Mesh Spine-01 OcNOS-DC 51.2 Tbps · DLB eBGP · ECMP · DLB Spine-02 OcNOS-DC 51.2 Tbps · DLB eBGP · ECMP · DLB GPU Storage NVMe-oF / NFS RDMA-optimized OcNOS-DC — AI FABRIC — HORIZONTAL CLOS · PFC · ECN · DLB · 800G
OcNOS-DC leaf/spine
OcNOS-DC spine (DLB)
GPU servers / storage

Hover nodes for capability and platform details  ·  Full HCL: 40+ validated platforms at ipinfusion.com/hcl

600+Operator Deployments
60+Countries
26Years in Networking
Technical Architecture

Four layers of losslessness — built into OcNOS-DC.

Most AI fabric failures come from a single misconfigured PFC priority group or an ECN threshold set for cloud workloads rather than RDMA. OcNOS-DC ships with buffer profiles validated for RoCEv2 on each supported Broadcom ASIC — so Day-1 configuration is correct, not trial-and-error.

PFC + ECN — Priority-group lossless control

PFC (Priority Flow Control) pauses per-priority traffic before buffer overflow. ECN marks packets early to trigger sender-side slowdown. Together they prevent drops without stalling the entire port. OcNOS-DC supports PFC over L3 for routed AI fabrics.

Dynamic Load Balancing (DLB) — flow-level ECMP

Standard consistent-hash ECMP creates hotspots when many GPU-to-GPU flows collide on the same spine link. DLB in OcNOS-DC monitors real-time queue depth and non-disruptively reassigns elephant flows to less-loaded paths — maximizing fabric utilization during AllReduce.

DCBX — automated server-to-switch configuration

DCB Exchange Protocol (DCBX) runs over LLDP and pushes the correct PFC and ETS configuration from OcNOS-DC leaf switches to attached GPU servers automatically — eliminating the risk of manual misconfiguration that silently breaks losslessness.

gNMI on-change telemetry — PFC counter visibility

PFC pause counters, ECN marking rates, and per-priority buffer depths are exposed as gNMI sensor paths with on-change subscriptions. Feed directly to Prometheus and Grafana to detect congestion events in milliseconds — before they cascade into training stalls.

ai-leaf01 — gNMI lossless fabric telemetry STREAMING
$gnmic subscribe --path /qos/pfc/ \
--mode ON_CHANGE --encoding proto
RoCEv2 Priority Group 3 — real-time
et-0/0/1 PG3 PFC-Rx: 0 Tx: 0 Drop: 0
et-0/0/2 PG3 PFC-Rx: 0 Tx: 0 Drop: 0
et-0/0/3 PG3 PFC-Rx: 0 Tx: 0 Drop: 0
$gnmic subscribe --path /interfaces/counters/
et-0/0/1 in: 780 Gbps out: 776 Gbps
et-0/0/2 in: 795 Gbps out: 791 Gbps
→ Telegraf → Prometheus → Grafana
✓ lossless — 0 drops — fabric healthy

Validated AI Fabric Platforms

AIS800-64D
Edgecore — Spine
800GTH5
S9321-64E
UfiSpace — Spine
800GTH5
AS9736-64D
Edgecore — Leaf
400G / 25.6T
AS9716-32D
Edgecore
400G / 12.8T

40+ validated platforms — view full HCL →

Open vs Proprietary

Why operators are moving AI fabric to open hardware.

Proprietary AI switch vendors charge a premium for switching ASICs that are, in most cases, the same Broadcom merchant silicon available in open ODM hardware. OcNOS-DC gives you the same lossless RoCEv2 performance — without the lock-in.

❌  Proprietary Vendor

Hardware and software bundled — you pay the vendor margin on both, every refresh cycle.

PFC/ECN profiles are vendor-tuned and not exposed to operators — you trust defaults you cannot inspect.

Single-vendor ECMP implementation — no DLB, or DLB locked to a specific proprietary protocol.

Proprietary telemetry stack — data only flows into the vendor's own observability products.

Support requires separate hardware and software contracts from the same vendor.

✓  OcNOS-DC on Open Hardware

Hardware from Edgecore or UfiSpace. Software from IP Infusion. One combined SLA — two vendor relationships eliminated.

Broadcom Tomahawk buffer profiles fully configurable and documented. Validated PFC/ECN settings ship with OcNOS-DC for each platform.

DLB (Dynamic Load Balancing) standard in OcNOS-DC — monitors real-time queue depth, reassigns flows non-disruptively.

gNMI with on-change subscriptions — all PFC/ECN/buffer data feeds into standard Prometheus, InfluxDB, or OpenTelemetry pipelines.

Single IP Infusion support contract covers software, TAC, and hardware RMA coordination globally, 24/7.

Deployment Scenarios

Where OcNOS AI Fabric is deployed today.

USE CASE 01

GPU-Dense AI Training Clusters

Large-scale GPU training clusters running distributed jobs require a non-blocking lossless fabric with consistent latency across all GPU-to-GPU paths. OcNOS-DC delivers PFC/ECN and DLB on 800G spine-leaf topologies, ensuring AllReduce operations complete without collective restarts.

USE CASE 02

AI Inference at Scale

High-throughput inference clusters serving real-time API endpoints require predictable low-latency paths between GPU nodes. OcNOS-DC's ETS scheduling ensures inference traffic is never queued behind batch jobs, and streaming telemetry provides per-flow visibility to detect latency regression in production.

USE CASE 03

GPU-as-a-Service / Cloud AI

Cloud providers offering GPU compute to tenants need multi-tenant fabric isolation alongside lossless RoCEv2. OcNOS-DC combines EVPN-VXLAN tenant isolation from OcNOS-DC's data center fabric feature set with the RoCEv2 lossless stack — both in a single NOS instance on the same hardware.

Get Started

Bring your topology. We'll show you the path.

Every IPI demo is led by a network architect with production OcNOS deployments — no slides, no sales theatre. Just your specific AI fabric topology and real configuration walkthroughs.

Common Technical Questions

AI Fabric with OcNOS-DC

What is RoCEv2 and why does it require a lossless Ethernet fabric?
RoCEv2 (RDMA over Converged Ethernet v2) enables direct GPU-to-GPU memory transfers without CPU involvement, eliminating software overhead in AI collective operations such as AllReduce and AllGather. Unlike TCP, RDMA has no retransmission mechanism — a single dropped packet halts the entire operation and forces a restart across all participating GPUs. A lossless fabric using Priority Flow Control (PFC) and Explicit Congestion Notification (ECN) is therefore a hard requirement for any production AI training environment. OcNOS-DC ships with buffer profiles pre-tuned for RoCEv2 traffic classes.
How does OcNOS-DC guarantee zero packet loss in an AI fabric?
OcNOS-DC implements a complete Data Center Bridging (DCB) stack: PFC (Priority Flow Control) pauses traffic at the per-priority-group level before buffers overflow, ECN (Explicit Congestion Notification) marks packets early to signal receivers to slow down, and ETS (Enhanced Transmission Selection) ensures AI/RDMA flows are never scheduled behind lower-priority traffic. These three mechanisms — combined with per-port buffer tuning on Broadcom Tomahawk ASICs — deliver a deterministic lossless environment. OcNOS also supports PFC over L3 for fabrics that extend across routed boundaries.
What is Dynamic Load Balancing (DLB) and why does it matter for AI workloads?
Standard Equal-Cost Multi-Path (ECMP) hashes flows to uplinks using a consistent hash that can create hotspots when many GPU flows land on the same spine link. Dynamic Load Balancing (DLB) in OcNOS-DC monitors real-time queue depth and non-disruptively reassigns flows to less-congested links. In an AllReduce pattern — where every GPU communicates with every other GPU simultaneously — DLB prevents the bandwidth imbalance that would otherwise force PFC pause storms and degrade Job Completion Time (JCT).
What scale does OcNOS AI Fabric support?
OcNOS-DC supports 400G and 800G leaf-spine fabrics. On Broadcom Tomahawk 5-based spine platforms such as the Edgecore AIS800-64D and UfiSpace S9321-64E, each spine switch delivers up to 51.2 Tbps of non-blocking throughput with 64 × 800G ports. Leaf switches run Tomahawk 4 platforms such as the Edgecore AS9736-64D at 400G / 25.6 Tbps, providing the standard oversubscription ratio expected in production AI fabrics. A 3-stage Clos topology with ZTP (Zero Touch Provisioning) and eBGP unnumbered underlay scales to large GPU clusters. The feature matrix page lists full per-platform SKU support.
How does OcNOS AI Fabric compare to proprietary solutions like NVIDIA Spectrum-X or Cisco?
OcNOS-DC runs on the same Broadcom Tomahawk merchant silicon used inside many proprietary AI switch products, but on open ODM hardware from Edgecore and UfiSpace. This eliminates per-port licensing premiums and vendor lock-in. Operators get equivalent RoCEv2 lossless performance — PFC, ECN, ETS, DLB — with IP Infusion's 24/7 carrier-grade SLA and a transparent SKU model. The capital freed by switching to open hardware can be reallocated directly into additional GPU capacity.
Does OcNOS-DC support automation and telemetry for AI fabric operations?
Yes. OcNOS-DC supports DCBX (DCB Exchange Protocol) for automated server-to-switch RoCEv2 configuration, ZTP over IPv4 and IPv6 for zero-touch rack onboarding, and gNMI streaming telemetry with on-change subscriptions. PFC pause counters, ECN marking rates, and per-priority buffer depths are all exposed as gNMI sensor paths — consumable directly by Prometheus, InfluxDB, or any OpenTelemetry pipeline. Ansible playbooks and a Terraform provider are available for Day-0 through Day-2 automation.