Data Center Interconnect using MPLS

Chapter 1

Data Center Interconnect Overview

Data Center Interconnect Use Cases

Data Center Interconnect Technologies

Multipoint Interconnect using VPLS

Chapter 2

OcNOS DCI MPLS Overview

OcNOS DCI VPLS Architecture

Role determination using ICCP

VPLS Redundancy

MC-LAG in a Data Center using MPLS DCI

Basic Example Configuration

Conclusion

Glossary

Chapter 1

Data Center Interconnect Overview

• Datacenter interconnect use-cases
• Datacenter interconnect technologies
• Multipoint interconnect using VPLS

Data Center Interconnect Use Cases

Data centers are usually not local to a particular geographical location. They span across regions and countries throughout the globe. Data centers often manage critical and voluminous data which poses a challenge to the networking technology.

The applications running inside the data center are agnostic to the interconnection technology and often treat data centers on different sites transparently. One requirement of having multiple data centers is to provide a High-

Availability clusters in which if one data center site fails, traffic can switch to the other without disruption. Another use case is server mobility across data center which is often done for load balancing or better user experience. A general requirement therefore is to have layer-2 connectivity between the data centers to expose a unified network towards the applications or the servers running in different data centers. MPLS data center interconnection is a viable choice as the sections below describe.

Data Center Interconnect Technologies

The popular ways of data center interconnects are;

• Dark Fiber
• IPBased
  • VxLAN
  • OTVand LISP
• MPLS Based
  • Pointto Point interconnect using VPWS
  • Multipointinterconnect using VPLS

Dark Fiber can give end to end layer 2 connectivity but is not always a feasible solution when sites are separated geographically over very wide areas. Using VxLAN for data center connectivity is an emerging technology. OcNOS has support for this but it will not be elaborated in this current document. OTV is a proprietary solution.

The preferred choice of data center interconnect (DCI) is using MPLS. This is a proven technology and most modern data centers are interconnected using MPLS. Sometimes large data center enterprises prefer owning the MPLS core nodes themselves, to have better flexibility in designing their data center interconnect solution. OcNOS enables them with this by its MPLS interconnect solution.

Layer 2VPN connectivity can be achieved by either VPWS or VPLS. VPWS can be only used when interconnecting two sites, while VPLS allows for a multi-site connectivity and therefore is more flexible. VPLS works by essentially creating multiple point to point MPLS tunnels carrying the Layer2 frame encapsulated. It uses a combination of LDP and/or BGP for peer discovery and signaling. This validated solution guide will explain how OcNOS switches can be used to connect multisite data centers using VPLS.

Multipoint Interconnect using VPLS

The following diagram shows a typical case of data center interconnect where three data center sites are involved. Generally it is preferred to have an IP/MPLS loop connection such that there is a backup path available in case of failure. In each data center two DCI core nodes are deployed as redundant to each other and there are two links between each data center site, each terminating on one of the redundant nodes.

DataCenter

SiteB

Data Center SiteA Data Center SiteC

There are two physical links between each data center

Note: Only the DCI core switches of each data center is shown.

Redundant Core Nodes (VPLS Enabled)

The data center core nodes terminate the Layer-2 network of the data center and map the traffic over VPLS to the other data center sites. Thus, the layer 2 domains of the each data center is extended onto the other.

Chapter 2

OcNOS DCI MPLS Overview

• OcNOSDCI MPLS architecture
• Roledetermination using ICCP
• VPLSredundancy
• MC-LAGin the data center using MPLS DCI

OcNOS DCI VPLS Architecture

Figure2. DCI VPLS Architecture

The above topology diagram demonstrates the network architecture on which this solution is developed.

The topology is a standard 3-tier architecture of access, aggregation and core nodes. OcNOS is running on all the nodes. The DCI core node is a layer 3 switch with MPLS enabled and is serving as the layer 2/layer 3 edge node. The aggregation and access nodes are layer 2 switches.

• The layer 3 switches (the DCI core nodes), terminate the layer 2 connections on one end and have VPLS deployed over the Data center The layer 3 switches run Inter-Chassis Communication protocol (ICCP). The same VPLS instance is running on both the DCI core nodes. ICCP synchronizes which node will act as the active/standby for a particular VPLS instance.
• Onthe layer 2 switches (the aggregation and access nodes), an IPI proprietary OcNOS Multi-Chassis Lag (MC-LAG) is deployed, which provides node redundancy as well as link MC-LAG is working in Active-Active mode.

The DCI Core nodes does load sharing based on VLANs which will be mapped to different VPLS instances. Some key features of this solution is

• Aggregation nodes are in MLAG Core Nodes are not in MLAG Domain.
• All the nodes have redundancy, and all are active in steady
• All the links have redundancy and working in active/active

Role determination using ICCP

ICCP protocol is defined in RFC 7275. The OcNOS implementation of ICCP is modelled on RFC 7275 with some proprietary implementations for better results. Two nodes form a ‘Redundancy Group’. An ICCP connection is established between the two nodes over a LDP connection.

ICCP exchanges the VPLS information between the redundant groups, and decides which VPLS instance will be Active/Standby. The administrator can give a preference primary/secondary for a certain VPLS instance on a node and based on that ICCP will choose on which node the VPLS instance will be made Active. The other will be standby. If no preference is given, admin preference is taken as primary, and then ICCP will do a tie-breaker and chooses one on its own logic.

As shown in the diagram below, VPLS-A had the admin role of primary but the operational role determined by ICCP by communicating with its redundant pair is secondary. So this VPLS instance will be standby and will not be forwarding any traffic mapped to it.

VPLS Redundancy

As mentioned earlier (See Figure DCI VPLS Architecture), the DCI core nodes terminate the layer 2 connections of the data center and extend it over to the other data center using VPLS. The core node which is running VPLS are connected to each other on different data centers. If the core node fails the entire data center connectivity will be lost.

Thus, it is very critical for the data center to have a redundancy on the core nodes. For the core node redundancy, VPLS redundancy is also required.

The following diagram shows how typically two data centers are interconnected. The PE nodes (DCI Core nodes) are dual homed to CEs nodes which are part of a MCLAG domain. MC-LAG expands the concept of link aggregation so that it provides node-level redundancy by allowing two or more nodes to share a common LAG endpoint. It emulates multiple nodes to represent as a single logical node to the remote node running Link aggregation. From the perspective of the PE nodes it can be considered that its ‘dual homed’ to a single CE node. The VPLS instances running in the

PE node, have attachment circuits as LAG ports (not MCLAG). Pseudo wires( which are essentially MPLS tunnels encapsulating the Layer2 circuits) are set up between each PE node in one data center to both the PE nodes in other data center. ICCP decides which VPLS instance will be made Active/Standby and based on that the pseudo wires corresponding to the VPLS instances exchange pseudo wire status messages. After negotiations, one pseudo wire per VPLS will be active between the data centers. In the figure, the standby Pseudo wires are shown in dotted lines and

the active in bold lines. For VPLS RED the pseudo wire between PE1 and PE4 is active. For VPLS Blue pseudo wire between PE2 and PE3 is active.

Figure3.VPLS Redundancy

VPLS redundancy always operates in Active/Standby mode, that is both nodes cannot be active simultaneously. Thus if there is a single VPLS instance only one core node in a data center will have an active VPLS and the other node will be standby and unutilized. That is why in solution design for data center adopted an approach to split the layer 2 traffic based on VLAN ranges and map them to two VPLS instances. Each node will have one active VPLS instance, and one standby VPLS instance. This is applicable if the VPLS attachment circuits are in Port+VLAN mode.

With reference to the topology in figure 3, the VLANs 2-200 are distributed between VPLS-Red and VPLS-Blue. In

data center 1, PE1 has VPLS-Red as ‘Active’, serving VLANs 2-100 and PE2 has VPLS-Blue as ‘Active’ serving VLANs 101-200. Thus the traffic is load balanced between the two redundant Nodes. The pseudo wires send status TLV corresponding to the active/standby status of the VPLS its part of and after negotiation one pseudo wire between the data centers will be up per VPLS.

Handling Failure Scenarios

In practice we can even have more than two VPLS instances dedicated to different ranges of VLANs, or even mapped to particular VLANs. If the preference given for a particular VPLS instance on both the redundant DCI core node is the same, ICCP will do a tie-breaker and select only one as active.

Note: The same VLAN should not be mapped to two different VPLS instances.

If the attachment circuit connected to a DCI core node is down, or the Node itself is down, the redundant node will take up all the operations.

The figure above(Figure: Handling Failure Scenarios) shows that PE1 is down. ICCP communication between the DCI core nodes detects this failure and thus PE2 will now host all the operations. Thus both the VPLS instances become active in PE2 node. The pseudo wire status is exchanged accordingly and again there will be one pseudo wire active per VPLS between the two data centers, in this case for VPLS Red between PE2 and PE4 and for VPLS Blue between PE2 and PE3.

MC-LAG in a Data Center using MPLS DCI

As explained earlier MC-LAG expands the concept of link aggregation so that it provides node-level redundancy by allowing two or more nodes to share a common LAG endpoint. It emulates multiple nodes to represent as a single logical node to the remote node running Link aggregation. As a result even if one of the nodes is down there exists a path to reach the destination via other nodes.

The topology in the figure “Handling Failure Scenarios” shows that DCI only deploys MC-LAG only at the Aggregation Nodes. The DCI Core nodes are not running it. However we still have link redundancy between Aggregation and DCI layer. This is because as shown in figure 4, there is a dual homing between each Aggregation node towards both the DCI core node. If the link CE1-PE1 fails, then CE2-PE1 link takes up by virtue of MC- LAG. If both the CE1-PE1, CE2-PE1 link fails, then ICCP will trigger switchover and the PE2 node will be active for all the VPLS instances.

Also there is no duplicate traffic going out of the DCI core nodes because they have VLAN-based load sharing, and a particular VLAN is taken up only by one of the two core nodes when both nodes are up. Thus, even though duplicate traffic reaches to the core nodes, it is dropped by the node which is not serving that particular VLAN. Guaranteeing a loop free topology.

Basic Example Configuration

PE1

PE1 Cont.

PE1 Cont.

CE1

CE2

CE2Cont.

PE3

PE3Cont.

PE3Cont.

PE4

PE4Cont.

PE4Cont.

CE3

CE3Cont.

CE4

CE4Cont.

PE2

PE2Cont.

PE2Cont.

Conclusion

The OcNOS data center interconnect solution provides connectivity between the data centers enabling them to extend their Layer 2 network.

The OcNOS data center interconnect solution handles redundancy at all levels, including the core nodes using ICCP and VPLS redundancy. The links between aggregation and core are dual homed and redundant. In the aggregation and access nodes, the OcNOS MC-LAG solution provides both node and link level redundancy. Also, all the nodes and links are active and there are no unused links.