Home · Documentazione · Tecnologia · What Is Lossless Ethernet
Networking Definition

What Is Lossless Ethernet?

Lossless Ethernet is standard Ethernet tuned so that RDMA traffic is not dropped when the network gets congested. It is the no-drop substrate a GPU fabric needs, built from PFC, ECN, and DCQCN.

Lossless Ethernet is standard Ethernet configured so a chosen class of traffic, typically RDMA, is not dropped under congestion. It combines Priority Flow Control (PFC), which pauses a specific priority queue before it overflows, and Explicit Congestion Notification (ECN), which marks packets early so the sender slows down before any queue fills. DCQCN is the end-to-end algorithm that ties switch ECN marking to NIC rate control. RoCEv2, the transport GPUs use over Ethernet, needs this no-drop path, which is why every production GPU fabric runs it. OcNOS-DC ships these controls on Broadcom Tomahawk 5 hardware.

The three controls, and how they layer

PFC (the backstop)

Priority Flow Control pauses one priority queue hop by hop when its buffer is about to overflow, so nothing is dropped. It is coarse and fast, used as the last line of defense, not the everyday control.

ECN (the primary control)

Explicit Congestion Notification marks packets as a queue starts to build. The receiving NIC signals the sender, which lowers its rate before the queue ever fills. Fine-grained and continuous.

DCQCN (the loop)

DCQCN is the end-to-end algorithm that binds ECN marking on the switch to rate control on the NIC. The switch role is the ECN marking; the NIC does the rate adjustment. Together they hold queues low without triggering PFC.

The design intent is a ladder. ECN and DCQCN do the everyday work of keeping queues shallow so traffic keeps flowing. PFC only engages when a burst outruns them, pausing a priority for a moment rather than dropping. A fabric that is triggering PFC constantly is a fabric that needs its ECN thresholds tuned, not more PFC. See the DCQCN e deadlock PFC pages for how each is configured and protected.

Why RDMA and RoCEv2 require it

RDMA was built for a no-drop transport. RoCEv2 carries RDMA over UDP and IP so it can be routed across a data center, but it inherits that low tolerance for loss: a single dropped packet forces recovery that spikes tail latency and can stall a GPU collective, which stalls the whole training step. Making the Ethernet path lossless gives RoCEv2 the behavior it was designed for. That is the entire reason lossless Ethernet exists in AI fabrics.

What OcNOS-DC ships

On Broadcom Tomahawk 5 platforms, OcNOS-DC provides PFC (including PFC over L3, with DCBX and LLDP), ECN, Dynamic ECN, Enhanced Transmission Selection (ETS), WRED, and PFC Deadlock Detection and Recovery, plus buffer tuning and RTAG7 hashing to spread flows. Congestion and queue counters stream over gNMI and OpenConfig telemetry, so thresholds can be tuned in a closed loop during cluster bring-up rather than guessed.

Fabric lossless RoCEv2 DCQCN Protezione deadlock PFC What is RDMA Soluzione AI fabric OcNOS-DC

Tuning a lossless fabric? Let's do the thresholds together.

Prenoti una Architecture Review →
FAQ

What Is Lossless Ethernet FAQ

Cos'è l'Ethernet senza perdite?
Lossless Ethernet is standard Ethernet configured so that a chosen class of traffic, typically RDMA, is not dropped when the network gets congested. It is achieved with Priority Flow Control (PFC), which pauses a specific priority queue before its buffer overflows, and Explicit Congestion Notification (ECN), which marks packets early so the sender slows down before any queue fills. The goal is a no-drop path for traffic that reacts badly to loss, without turning the whole network no-drop.
How do PFC, ECN, and DCQCN work together?
They operate at different speeds. ECN is the primary, fine-grained control: a switch marks packets when a queue starts to build, and the receiving NIC signals the sender to reduce its rate. DCQCN is the end-to-end algorithm that ties ECN marking on the switch to rate control on the NIC in a continuous loop; the switch role in DCQCN is the ECN marking. PFC is the fast backstop: when a queue is about to overflow anyway, PFC pauses that priority hop by hop so nothing is dropped while ECN and DCQCN take effect. Well-tuned fabrics lean on ECN and DCQCN and keep PFC as the last resort.
Why does RoCEv2 need lossless Ethernet?
RoCEv2 is RDMA carried over UDP and IP. RDMA was designed for a no-drop transport, so a dropped packet forces recovery that is expensive in tail latency and can stall a GPU collective. Making the Ethernet path lossless with PFC and ECN gives RoCEv2 the no-drop behavior it expects, which is why a production RoCEv2 GPU fabric always runs on a lossless-configured network.
What lossless features does OcNOS-DC support?
On Broadcom Tomahawk 5 hardware, OcNOS-DC supports PFC (including PFC over L3, with DCBX and LLDP), ECN, Dynamic ECN, Enhanced Transmission Selection (ETS), WRED, and PFC Deadlock Detection and Recovery, along with buffer tuning and RTAG7 hashing. Congestion state and queue counters stream over gNMI and OpenConfig telemetry for closed-loop tuning. See the PFC deadlock and DCQCN pages for the detail behind each.
AI fabric

Design the whole AI fabric with OcNOS

From the business case to the port-count maths, pick up wherever you are in the build.

Soluzione Open AI Fabric The complete 800G AI fabric: open switches, OcNOS-DC, and support under one contract. Design di riferimento Topologie AI Fabric Rail-optimized, scheduled 3-stage Clos, and coherent DCI, sized in real port counts. Size & build AI Fabric Design Suite Size a GPU fabric: leaf, spine, and super-spine counts with a component and power summary.
Nuovo alle AI fabric? Inizia qui Cos'è una AI fabric? Cos'è una GPU fabric? Cos'è l'Ethernet senza perdite? Cos'è RDMA?
The technology inside RoCEv2 lossless Rail-optimized network DLB adaptive routing GLB (7.1) Ultra Ethernet DCQCN deadlock PFC InfiniBand vs Ethernet RoCE vs InfiniBand AI fabric architecture Coherent DCI
Datasheet e schede soluzione

Approfondite. Portatelo con voi.

Two short, technical downloads that go further than this page: the full OcNOS-DC datasheet and the lossless 800G AI fabric architecture.

Datasheet

Datasheet OcNOS-DC

Specifica completa di OcNOS-DC: il set di funzionalità EVPN-VXLAN e Ethernet for AI, gli SKU software, le piattaforme hardware supportate e la guida all'ordine della soluzione.

Ottieni il datasheet
Solution brief

OcNOS 800G Lossless AI Fabric

Fabric RoCEv2 non bloccante su spine Broadcom Tomahawk 4/5: tier di SKU, piattaforme validate e architettura di deployment.

Scarica il brief