Overlay ECMP with EVPN-VXLAN in OcNOS: Part 2 — Multi-VM Subnet Scaling

In Part 1, we distributed traffic across multiple hypervisors each hosting a single VM with the same service IP. Part 2 addresses the more complex scenario: multiple VMs sharing the same IP subnet hosted on the same hypervisor, and how OcNOS handles ECMP routing in this case.

Topology: Multiple VMs per Hypervisor

Key Difference: Subnet Routes vs. Host Routes

When multiple VMs on the same hypervisor share the same IP subnet, we cannot use EVPN type-2 (MAC+IP) routes for load balancing — those are per-host. Instead, each VM establishes an EBGP session to the leaf switch’s VRF and advertises its loopback IP (the service IP-X) as a host route. The leaf then aggregates these into a subnet advertisement (EVPN type-5) that the entire fabric learns.

EBGP: VM to Leaf VRF Configuration

! OcNOS -- Leaf-3: VRF and EBGP toward HV-3 VMs
!
! Step 1: Create VRF for tenant isolation
vrf TENANT-A
  vni 10100
!
! Step 2: Configure EBGP session to each VM's connected IP
router bgp 65003
  !
  vrf TENANT-A
    bgp router-id 10.1.1.3
    !
    ! VM-A connected IP: 172.16.3.1/31 (leaf side: 172.16.3.0)
    neighbor 172.16.3.1 remote-as 65100
    neighbor 172.16.3.1 activate
    !
    ! VM-B connected IP: 172.16.3.3/31
    neighbor 172.16.3.3 remote-as 65100
    neighbor 172.16.3.3 activate
    !
    ! VM-C connected IP: 172.16.3.5/31
    neighbor 172.16.3.5 remote-as 65100
    neighbor 172.16.3.5 activate
    !
    ! Redistribute VM host routes into EVPN
    address-family ipv4 unicast
      redistribute connected
      neighbor 172.16.3.1 activate
      neighbor 172.16.3.3 activate
      neighbor 172.16.3.5 activate
    exit-address-family
  !
!
! Step 3: Advertise VRF routes into EVPN (type-5 prefix)
router bgp 65003
  address-family l2vpn evpn
    advertise ipv4 unicast            ! Advertise VRF routes as EVPN type-5
  exit-address-family

VM-Side EBGP Configuration

! Linux VM (FRR/BIRD) -- EBGP to Leaf-3 VRF
! Each VM advertises its loopback (service IP-X) to the leaf
!
router bgp 65100
  bgp router-id 192.168.100.10
  neighbor 172.16.3.0 remote-as 65003    ! Leaf-3 VRF IP
  !
  address-family ipv4 unicast
    network 192.168.100.10/32            ! Service IP-X (loopback)
    neighbor 172.16.3.0 activate
    neighbor 172.16.3.0 next-hop-self
  exit-address-family

Verifying Multi-VM ECMP

! OcNOS -- Verify ECMP across fabric for IP-X
!
! On any leaf: check EVPN type-5 route for IP-X subnet
show bgp l2vpn evpn route type prefix

! Expected: IP-X/32 with multiple next-hops (one per VTEP)
! B>* 192.168.100.10/32 [200/0]
!   via 10.1.1.1 (vxlan), weight 1    <- HV-1
!   via 10.1.1.2 (vxlan), weight 1    <- HV-2
!   via 10.1.1.3 (vxlan), weight 1    <- HV-3 (aggregated from 3 VMs)
!
! Verify VRF route table on Leaf-3:
show ip route vrf TENANT-A
!
! Check BGP neighbor state for each VM:
show bgp vrf TENANT-A summary

• Overlay ECMP Part 1 — Fundamentals
• EVPN-ELAN Configuration Guide
• EVPN-L3VPN over ISIS-SR
• OcNOS-DC Product Page
• Contact IP Infusion

IP Infusion Engineering Team