Adaptive routing · Flowlet · RoCEv2

Dynamic Load Balancing: Adaptive Routing for AI Fabrics

On an AI fabric, use adaptive Dynamic Load Balancing rather than static-hash ECMP. DLB re-evaluates the path on every flowlet, a sub-flow chunk delimited by an inter-packet gap, using live ASIC egress queue-depth and port-utilisation telemetry, so GPU collectives stop colliding on a single uplink. It runs in the switch ASIC on Broadcom Tomahawk 4 and 5, with no NIC or collective-library changes.

Per-flowletadaptive path selection
TH4 + TH5Broadcom Tomahawk 25.6T / 51.2T
0 NIC changesruns in the switch ASIC
1 SKUOcNOS-DC PLUS, no add-on
Adaptive routing in motion

Adaptive routing on a leaf-spine fabric

A four-spine, two-leaf slice carrying GPU AllReduce traffic. DLB measures local egress queue depth in real time. When Spine-3 saturates, the leaf re-bins the next flowlet onto a less-loaded spine, keeping all four uplinks balanced.

Dynamic Load Balancing across an AI leaf-spine fabric A four-spine, two-leaf AI fabric. GPU servers attached to the leaves send AllReduce flows. Three spine paths carry balanced flowlets. The fourth spine is congested, and Dynamic Load Balancing re-bins the next flowlet onto a less-loaded spine. Bottom band labels DLB metrics: queue depth, port utilisation, flowlet rebind. Spine-3 congested · next flowlet re-bound to Spine-2 Spine-1 queue 18% Spine-2 queue 22% Spine-3 queue 92% Spine-4 queue 25% Leaf-1 DLB · flowlet Leaf-2 DLB · flowlet GPU-0 GPU-1 GPU-2 GPU-3 DLB · QUEUE-DEPTH FEEDBACK · FLOWLET REBIND · CONGESTION-AWARE ECMP
Why static ECMP fails on AI fabrics

Static ECMP and DLB, axis by axis

Standard ECMP hashes the 5-tuple at flow start and pins the flow for its lifetime. On north-south web traffic, millions of short-lived flows smooth utilisation across paths. On an AI fabric a handful of GPU elephant flows, each holding a whole 400G or 800G uplink for seconds, cause hash polarisation: some uplinks saturate while parallel paths sit idle. DLB re-evaluates the path on every flowlet using live ASIC telemetry.

Axis Static ECMPhash and pin DLBadaptive, per-flowlet
Path decisionHashes the 5-tuple once at flow start and pins the flow for its entire lifetime.Re-evaluates the path on every flowlet boundary and moves traffic to a less-congested uplink.
GranularityWhole flow. A single hash choice governs seconds of elephant-flow traffic.Flowlet: a sub-flow chunk delimited by a small inter-packet gap, safe to re-bind without reordering.
Congestion signalNone. The hash is blind to real-time link state.Live ASIC egress queue-depth and port-utilisation telemetry, recomputed every few microseconds.
Under GPU elephant flowsHash polarisation: two elephants can collide on one uplink while another sits idle.Saturated members drop out of the candidate set within one flowlet, so uplinks stay balanced.
Reordering riskNone; the flow is pinned, but at the cost of hot-spots and tail-latency outliers.None in practice; the inter-packet gap means RoCEv2 and TCP see correct in-order delivery.
NIC / collective libraryNo changes; standard behaviour.No changes. Runs in the switch ASIC, so RoCEv2 NICs and xCCL stacks are untouched.
HardwareAny ECMP-capable switch.Broadcom Tomahawk 4 (25.6T) and Tomahawk 5 (51.2T) in 64x400G and 64x800G configurations.
TelemetryStandard interface counters.Per-member rebind counts, flowlet-gap distributions, and member quality scores over gNMI.
Inside the implementation

The OcNOS DLB implementation

DLB is not a control-plane bolt-on. It runs in the Broadcom forwarding pipeline, scoring every ECMP next-hop from live congestion signals and re-binding flowlets in hardware, co-tuned with the RoCEv2 lossless stack.

Flowlet detection

Sub-millisecond gap timer

An ASIC-native flowlet inactivity timer (typical 16 to 256 µs) splits long elephant flows into chunks safe to spray across paths without TCP or RoCEv2 reordering.

Path quality

Live queue-depth feedback

DLB consumes per-egress port queue-occupancy and link-utilisation signals from the Tomahawk pipeline to score every ECMP next-hop in real time.

Re-bind

Adaptive next-hop selection

On each flowlet boundary the highest-quality member is selected. Member quality is recomputed every few microseconds, so a saturated spine drops out of the candidate set within one flowlet.

Lossless

Co-tuned with PFC & ECN

DLB integrates with the RoCEv2 lossless stack (PFC, ECN/DCQCN, headroom math), so flowlet rebinding happens before pause frames propagate upstream.

Telemetry

gNMI export

Per-member rebind counts, flowlet-gap distributions, and member quality scores stream over gNMI dial-out for closed-loop fabric tuning.

Hardware

TH4 / TH5 native

Validated on Broadcom Tomahawk 4 (25.6T) and Tomahawk 5 (51.2T) spine platforms, in 64x400G and 64x800G port configurations, with no software fast-path penalty.

Part of the adaptive-routing stack

Where DLB sits in the fabric

DLB is the single-hop, congestion-aware layer of a larger open Ethernet transport. It rides on lossless RoCEv2, hands off to fabric-wide GLB for end-to-end scoring, and keeps a clean path to Ultra Ethernet as UEC NICs ship.

Lossless transport underneath

DLB rebinds flowlets on top of a RoCEv2 lossless fabric (PFC, ECN, DCQCN), so adaptive routing and lossless RDMA are tuned together rather than fighting each other.

Fabric-wide scoring next

Where DLB scores the local next-hop, GLB (OcNOS 7.1) extends the decision to end-to-end path quality across the whole leaf-spine fabric.

Spray-friendly transport ahead

DLB is the adaptive-routing bridge to Ultra Ethernet, whose packet spray and multi-path RDMA carry the same idea into the NIC as UEC hardware ships.

In production

What DLB delivers in production AI fabrics

By keeping flowlets off the congested uplinks that static ECMP overloads, DLB turns a symmetric leaf-spine fabric into one that stays balanced under GPU collective traffic.

  • Higher utilisation. Flowlet rebalancing keeps traffic off the congested uplinks that static ECMP overloads, so a well-run fabric can target utilisation above 90 percent on the same hardware, without buying more uplinks.
  • Lower tail latency. P99.9 collective completion time tightens because no single link saturates while others sit idle.
  • Faster training. Less GPU idle time waiting on the slowest rank means measurable wall-clock improvement on AllReduce-heavy workloads.
  • No NIC changes. DLB lives in the switch ASIC. Existing RoCEv2 NICs and xCCL (NCCL, RCCL, oneCCL) collective stacks see correct in-order delivery without code changes.
  • One license. DLB is part of the OcNOS-DC PLUS SKU: same image, same support contract, no per-feature add-on.
The IP Infusion view

Adaptive routing is table stakes for an Ethernet AI fabric

Static ECMP was built for north-south web traffic. Once a fabric carries GPU collectives, adaptive routing is the difference between balanced uplinks and stalled training jobs, and OcNOS delivers it on open hardware today.

ECMP alone is not enough

A handful of elephant flows defeat static hashing. Hash polarisation leaves some uplinks saturated while parallel paths sit idle.

DLB closes the routing gap

Per-flowlet, congestion-aware path selection keeps an Ethernet fabric balanced under collective traffic, one of the axes where InfiniBand had an edge.

OcNOS is the enabler

DLB today, GLB next, UEC as NICs ship. One NOS on validated open hardware from vendors such as Edgecore and UfiSpace, with no NIC or collective-library changes.

FAQ

Dynamic Load Balancing, answered

What is a flowlet and why does DLB use it?
A flowlet is a sub-flow chunk delimited by a small inter-packet gap. DLB re-evaluates the path at each flowlet boundary instead of pinning a whole flow, so traffic moves to a less-congested uplink without causing RoCEv2 or TCP reordering.
How is DLB different from static ECMP?
Static ECMP hashes the 5-tuple at flow start and pins the flow for its lifetime, so two GPU elephant flows can collide on one uplink while another sits idle. This hash polarisation lets some uplinks saturate while parallel paths stay idle. DLB scores every next-hop in real time using live queue depth and port utilisation, and rebinds flowlets onto the best path.
Does DLB require new NICs or changes to my collective library?
No. DLB runs in the switch ASIC, so existing RoCEv2 NICs and xCCL (NCCL, RCCL, oneCCL) collective stacks see correct in-order delivery with no code changes.
Which hardware supports OcNOS DLB?
DLB is supported on Broadcom Tomahawk 4 (25.6T) and Tomahawk 5 (51.2T) platforms in 64x400G and 64x800G configurations, and is included in the OcNOS-DC PLUS SKU with no per-feature add-on.

Tuning DLB for your GPU fabric? Get a workload-specific review

Tell us the GPU scale and the collective pattern, and an IP Infusion engineer will tune flowlet timers and path scoring with you, or start with a first-pass leaf-spine layout in the AI Fabric Design Suite.