Startseite · Dokumentation · Technologie · What Is Lossless Ethernet
Networking Definition

What Is Lossless Ethernet?

Lossless Ethernet is standard Ethernet tuned so that RDMA traffic is not dropped when the network gets congested. It is the no-drop substrate a GPU fabric needs, built from PFC, ECN, and DCQCN.

Lossless Ethernet is standard Ethernet configured so a chosen class of traffic, typically RDMA, is not dropped under congestion. It combines Priority Flow Control (PFC), which pauses a specific priority queue before it overflows, and Explicit Congestion Notification (ECN), which marks packets early so the sender slows down before any queue fills. DCQCN is the end-to-end algorithm that ties switch ECN marking to NIC rate control. RoCEv2, the transport GPUs use over Ethernet, needs this no-drop path, which is why every production GPU fabric runs it. OcNOS-DC ships these controls on Broadcom Tomahawk 5 hardware.

The three controls, and how they layer

PFC (the backstop)

Priority Flow Control pauses one priority queue hop by hop when its buffer is about to overflow, so nothing is dropped. It is coarse and fast, used as the last line of defense, not the everyday control.

ECN (the primary control)

Explicit Congestion Notification marks packets as a queue starts to build. The receiving NIC signals the sender, which lowers its rate before the queue ever fills. Fine-grained and continuous.

DCQCN (the loop)

DCQCN is the end-to-end algorithm that binds ECN marking on the switch to rate control on the NIC. The switch role is the ECN marking; the NIC does the rate adjustment. Together they hold queues low without triggering PFC.

The design intent is a ladder. ECN and DCQCN do the everyday work of keeping queues shallow so traffic keeps flowing. PFC only engages when a burst outruns them, pausing a priority for a moment rather than dropping. A fabric that is triggering PFC constantly is a fabric that needs its ECN thresholds tuned, not more PFC. See the DCQCN und PFC-Deadlock pages for how each is configured and protected.

Why RDMA and RoCEv2 require it

RDMA was built for a no-drop transport. RoCEv2 carries RDMA over UDP and IP so it can be routed across a data center, but it inherits that low tolerance for loss: a single dropped packet forces recovery that spikes tail latency and can stall a GPU collective, which stalls the whole training step. Making the Ethernet path lossless gives RoCEv2 the behavior it was designed for. That is the entire reason lossless Ethernet exists in AI fabrics.

What OcNOS-DC ships

On Broadcom Tomahawk 5 platforms, OcNOS-DC provides PFC (including PFC over L3, with DCBX and LLDP), ECN, Dynamic ECN, Enhanced Transmission Selection (ETS), WRED, and PFC Deadlock Detection and Recovery, plus buffer tuning and RTAG7 hashing to spread flows. Congestion and queue counters stream over gNMI and OpenConfig telemetry, so thresholds can be tuned in a closed loop during cluster bring-up rather than guessed.

Verlustfreie RoCEv2-Fabric DCQCN PFC-Deadlock-Schutz What is RDMA AI-Fabric-Lösung OcNOS-DC

Tuning a lossless fabric? Let's do the thresholds together.

Architecture Review buchen →
FAQ

What Is Lossless Ethernet FAQ

Was ist verlustfreies Ethernet?
Lossless Ethernet is standard Ethernet configured so that a chosen class of traffic, typically RDMA, is not dropped when the network gets congested. It is achieved with Priority Flow Control (PFC), which pauses a specific priority queue before its buffer overflows, and Explicit Congestion Notification (ECN), which marks packets early so the sender slows down before any queue fills. The goal is a no-drop path for traffic that reacts badly to loss, without turning the whole network no-drop.
How do PFC, ECN, and DCQCN work together?
They operate at different speeds. ECN is the primary, fine-grained control: a switch marks packets when a queue starts to build, and the receiving NIC signals the sender to reduce its rate. DCQCN is the end-to-end algorithm that ties ECN marking on the switch to rate control on the NIC in a continuous loop; the switch role in DCQCN is the ECN marking. PFC is the fast backstop: when a queue is about to overflow anyway, PFC pauses that priority hop by hop so nothing is dropped while ECN and DCQCN take effect. Well-tuned fabrics lean on ECN and DCQCN and keep PFC as the last resort.
Why does RoCEv2 need lossless Ethernet?
RoCEv2 is RDMA carried over UDP and IP. RDMA was designed for a no-drop transport, so a dropped packet forces recovery that is expensive in tail latency and can stall a GPU collective. Making the Ethernet path lossless with PFC and ECN gives RoCEv2 the no-drop behavior it expects, which is why a production RoCEv2 GPU fabric always runs on a lossless-configured network.
What lossless features does OcNOS-DC support?
On Broadcom Tomahawk 5 hardware, OcNOS-DC supports PFC (including PFC over L3, with DCBX and LLDP), ECN, Dynamic ECN, Enhanced Transmission Selection (ETS), WRED, and PFC Deadlock Detection and Recovery, along with buffer tuning and RTAG7 hashing. Congestion state and queue counters stream over gNMI and OpenConfig telemetry for closed-loop tuning. See the PFC deadlock and DCQCN pages for the detail behind each.
AI-Fabric

Design the whole AI fabric with OcNOS

From the business case to the port-count maths, pick up wherever you are in the build.

Lösung Open AI Fabric The complete 800G AI fabric: open switches, OcNOS-DC, and support under one contract. Referenzdesigns AI-Fabric-Topologien Rail-optimized, scheduled 3-stage Clos, and coherent DCI, sized in real port counts. Size & build AI Fabric Design Suite Size a GPU fabric: leaf, spine, and super-spine counts with a component and power summary.
Neu bei AI-Fabrics? Hier starten Was ist ein AI-Fabric? Was ist ein GPU-Fabric? Was ist verlustfreies Ethernet? Was ist RDMA?
The technology inside RoCEv2 lossless Rail-optimized network DLB adaptive routing GLB (7.1) Ultra Ethernet DCQCN PFC-Deadlock InfiniBand vs. Ethernet RoCE vs. InfiniBand AI fabric architecture Coherent DCI
Datenblatt und Solution Briefs

Gehen Sie tiefer. Nehmen Sie es mit.

Two short, technical downloads that go further than this page: the full OcNOS-DC datasheet and the lossless 800G AI fabric architecture.

Datenblatt

OcNOS-DC Datenblatt

Vollständige OcNOS-DC Spezifikation: Funktionsumfang für EVPN-VXLAN und Ethernet for AI, Software-SKUs, unterstützte Hardware-Plattformen und der Solution Ordering Guide.

Datasheet abrufen
Solution Brief

OcNOS 800G verlustfreie AI Fabric

Non-blocking RoCEv2-Fabric auf Broadcom Tomahawk 4/5 Spines: SKU-Stufen, validierte Plattformen und Deployment-Architektur.

Brief anfordern