OcNOS 7.1 roadmap · End-to-end · Layered on DLB

Global Load Balancing: Fabric-Wide Adaptive Routing

DLB makes the right call at one hop; GLB makes the right call across the whole fabric. On the OcNOS 7.1 roadmap, Global Load Balancing extends adaptive routing from a per-port view to end-to-end path quality, closing the multi-hop hot-spot gap on 3-stage Clos AI fabrics up to the 16,384-GPU ceiling. GLB layers on top of DLB rather than replacing it.

OcNOS 7.1GLB target release
End-to-endpath scoring, not local hop
16,384GPU ceiling, reference design
Layered on DLBextends, never replaces
End-to-end path telemetry

Scoring the whole path, not just the local hop

A 3-stage Clos slice (leaf, spine, super-spine) carrying GPU AllReduce. Every tier streams queue-occupancy and link-utilisation telemetry back toward the ingress leaves. GLB picks the path with the best end-to-end score, not the best local-egress score, so a clean uplink that lands on a congested downlink is no longer chosen blindly.

Global Load Balancing across a 3-stage Clos AI fabric Three-stage Clos AI fabric. Two super-spines on top, four spines in the middle, two leaves at the bottom. Telemetry arrows flow upward and back down so the ingress leaf sees end-to-end path quality. One spine-to-super-spine link is congested and bypassed in favour of an alternative end-to-end path. end-to-end telemetry Super-Spine-1TH5 · 51.2T Super-Spine-2TH5 · 51.2T Spine-1e2e ok Spine-2e2e ok Spine-3uplink hot Spine-4e2e ok Ingress LeafGLB · ranks paths Egress Leaftarget rack GLB · END-TO-END PATH SCORING · MULTI-HOP CONGESTION AWARENESS · OcNOS 7.1
Where the local view breaks

DLB and GLB, by scope of the path decision

DLB scores each ECMP next-hop using local egress queue-depth, which is optimal on a 2-tier leaf-spine. Scale to a 3-tier Clos and you can pick a spine with a clean uplink, only to land on a super-spine whose downlink back to the egress leaf is congested. The local view is correct; the end-to-end view is wrong. At 1,024-GPU and larger fabrics, where 3-stage Clos with super-spines becomes standard, this is the dominant remaining source of tail-latency outliers.

Axis DLBlocal, shipping today GLBglobal, OcNOS 7.1 roadmap
Scope of decisionPer-hop: each switch ranks its own ECMP next-hops.End-to-end: ingress leaves rank complete leaf-to-leaf paths.
Signal usedLocal egress queue-depth and link-utilisation on this switch.Congestion telemetry aggregated from every tier, back to ingress.
Best fit2-stage fabrics and the leaf-to-spine hop in 3-stage.3-stage Clos with super-spines, up to the 16,384-GPU ceiling.
Hot-spot caughtLocal egress congestion only.Downstream hot-spots the local hop cannot see.
RebindingFlowlet-aligned, in-order for RoCEv2 and TCP.Flowlet-aligned, same in-order guarantee, full-fabric input.
HardwareTH4 and TH5 today.Same TH4 and TH5 hardware, no new silicon.
RelationshipIndependent decision at each switch.Layers on top of DLB; does not replace it.
How GLB closes it

One decision, informed by the whole fabric

GLB reuses the adaptive-routing machinery operators already run and adds fabric-wide awareness on top. It builds directly on DLB and RoCEv2, and stays aligned with where Ultra Ethernet is heading.

Builds on

The DLB decision

GLB extends the DLB flowlet decision rather than replacing it. Mixed fabrics work correctly: non-GLB switches simply contribute local-only path quality during a rolling upgrade.

Runs over

A lossless RoCEv2 fabric

GLB rebinds at flowlet boundaries, preserving the in-order delivery that RoCEv2 RDMA needs. The transport stays production-grade; only the path input gets smarter.

Aligned with

Ultra Ethernet signalling

The path-quality plane is being designed to interoperate with Ultra Ethernet Consortium signalling as UEC NIC ecosystems mature, so the roadmap stays forward-compatible.

Inside OcNOS 7.1

How the planned GLB implementation fits together

GLB is enabled by OcNOS as the network operating system on open Broadcom-based hardware. The design keeps the control plane quiet and gives operators the telemetry they need to trust the fabric's decisions.

Telemetry plane

Path-quality publish

Every spine and super-spine publishes per-port queue-occupancy and utilisation deltas to a fabric-wide adjacency. Updates are sub-millisecond over existing in-band signalling, with no extra control-plane chatter.

Path scoring

End-to-end aggregation

Ingress leaves combine local egress quality with downstream telemetry into an aggregate score per candidate path. The worst hop dominates the score, the same intuition operators use when troubleshooting.

Selection

Flowlet-aligned

Like DLB, GLB rebinds at flowlet boundaries, preserving in-order delivery for RoCEv2 and TCP. The difference is what feeds the decision: full-fabric quality, not local-port quality.

Backwards-compatible

Layered on DLB

GLB extends the DLB decision; it does not replace it. Mixed fabrics with GLB-capable and DLB-only switches behave correctly, so brownfield upgrades from 7.0 are safe.

Scale

Up to the 16k-GPU ceiling

Reference designs use 256 spine switches and 128 super-spine switches, each a 64x800G Tomahawk 5, sized to the 16,384-GPU architectural ceiling.

Telemetry out

gNMI for the ops team

Per-path scores, rebind events, and worst-hop attribution stream over gNMI and OpenConfig, so SREs can correlate fabric decisions with xCCL collective job behaviour without a black box.

Roadmap and availability

What to expect when GLB lands

GLB is on the OcNOS 7.1 roadmap. It targets the same hardware and licensing operators already run, so adopting it is an upgrade rather than a forklift.

  • OcNOS 7.1 roadmap. GLB targets the 7.1 OcNOS-DC train, on the same TH4 and TH5 hardware running DLB today. Schedule and feature scope at the OcNOS releases page.
  • Same SKU. Planned for OcNOS-DC PLUS: no per-feature paywall, no new license keys at upgrade time.
  • In-place upgrade. Brownfield upgrade from 7.0 to 7.1 is supported; mixed-version fabrics keep working with DLB-only behaviour during the upgrade window.
  • UEC-aligned. The path-quality plane is being designed to interoperate with Ultra Ethernet Consortium signalling once UEC NIC ecosystems mature. See Ultra Ethernet (UEC).
  • Architecture review available. If you are sizing a 1,000-plus GPU fabric, we will run a sizing exercise that includes the GLB telemetry plane. Get a first-pass leaf-spine layout with the AI fabric design suite.
The IP Infusion view

DLB is the floor today, GLB raises the ceiling next

Adaptive routing is already solving real tail-latency at 2-tier scale. GLB carries that same discipline up into the 3-stage fabrics where the largest clusters live, on open hardware you already validate.

DLB solves the common case

On 2-tier leaf-spine and the leaf-to-spine hop, local adaptive routing already removes most ECMP collisions. That is shipping on TH4 and TH5 today.

GLB solves the scale case

At 1,024 GPUs and up, multi-hop hot-spots become the dominant outlier. GLB scores complete paths so the ingress leaf stops chasing a clean uplink into a congested downlink.

OcNOS is the enabler

One NOS, one feature roadmap: DLB today, GLB next, RoCEv2 and UEC-aligned throughout, on validated open hardware rather than a single-vendor fabric.

FAQ

Global Load Balancing, answered

How is GLB different from DLB?
DLB scores each next-hop using local egress queue-depth on one switch, which is optimal on a 2-tier leaf-spine. GLB aggregates congestion telemetry from every tier so ingress leaves rank complete leaf-to-leaf paths, catching downstream hot-spots the local view misses on 3-stage Clos fabrics.
When is GLB available?
GLB is on the OcNOS 7.1 roadmap, targeting the OcNOS-DC train, on the same Tomahawk 4 and 5 hardware that runs DLB today. It is planned for the OcNOS-DC PLUS SKU with no new license keys at upgrade time.
Do I have to replace DLB to use GLB?
No. GLB layers on top of the DLB decision rather than replacing it. Mixed fabrics work correctly: switches that are not GLB-capable simply contribute local-only path quality, and brownfield upgrades from 7.0 are supported.
How large a fabric does GLB support?
Reference designs use 256 spine switches and 128 super-spine switches, each a 64x800G Tomahawk 5, sized to the 16,384-GPU architectural ceiling.

Sizing a multi-thousand-GPU fabric? Let's run the numbers together

Tell us the workload and the GPU scale, and an IP Infusion engineer will size the GLB telemetry plane with you, or start with a first-pass leaf-spine layout in the AI fabric design suite.