A Flexible Framework for Automated Testing of Disaggregated Networking Systems

The software running on networking devices traditionally has been bound to specific vendor hardware platforms. With the effort of the Open Compute Project (OCP) and open-source hardware platforms, there are a variety of network operating systems and protocol stacks available today via subscription or open-source licensing.

Disaggregation in switching and routing devices decouples hardware and software, decomposing systems into open, independent, functional components.

Disaggregation avoids vendor lock-in and enables access to inexpensive products that are integrated using individual network elements. In mobile transport networks, operators are exploring open, disaggregated solutions to improve efficiency and performance, and avoid the limitations and expense of traditional, vertically-integrated, hardware products.

Disaggregation is getting widely accepted in the many applications, including data centers, Optical Line Termination (OLT) devices, Cell Site Gateways (CSG), access switches, and core routers.

Challenges in testing disaggregated networks:

As the saying goes “Flexibility comes with complexity” — it is becoming more and more difficult to test disaggregated devices manually.

While it’s true that the industry is getting the benefit of reduced development costs by choosing open-source software packages, there is no open standard or tool available to test network devices for features, scalability, and performance.

At IP Infusion, we are addressing this challenge by developing an automated test framework that is agnostic to the OS and underlying chip.

This framework satisfies following major requirements of an automated test tool.

  1. Open-source based
  2. Agnostic to OS, chip, topology and test-bed (virtual/real)
  3. Customized for networking domain needs
  4. Multi-layer design
  5. Reduced development and porting efforts
  6. Support for management via YANG (NETCONF)/REST (HTTP)/CLI (terminal)
  7. One framework for feature/sanity/long-run/performance testing
  8. Scalable
  9. CI/CD (continuous integration/continuous delivery) integration support
  10. Integration with existing frameworks based on Robot Framework or TCL
  11. Reduced manual intervention for traffic-based tests
  12. Low development and maintenance cost
  13. Parallel execution

The python-based framework, shown above, contains the core components needed to define the elements of a test setup — one that is flexible and supports plug-and-play extensions. The layered architecture for configuration and verification results in a framework that is agnostic to choice of OS, chip, and topology. The framework includes reporting and notification modules that help in understanding and back tracing the test execution results.

IP Infusion has validated the framework for solutions on white-box switches. IP Infusion is actively contributing to the Virtual OLT Hardware Abstraction (VOLTHA) testing framework that is part of openCORD initiatives. IP Infusion has domain and python expertise to satisfy our customers’ requirements for developing and maintaining the test suite for various domains (Switches/Routers/DCSG/VOLT/PON/DC).