In VXLAN-based networks, there are a range of complexities and challenges in determining the destination virtual tunnel endpoints (VTEPs) for any given VXLAN. At scale, various solutions, including Lightweight Network Virtualization (LNV), controller-based options like Midokura MidoNet or VMware NSX and even new standards like EVPN are attempts to address these complexities, however do retain their own complexities.
Enter static VXLAN tunnels, which simply serve to connect two VTEPs in a given environment. Static VXLAN tunnels are the simplest deployment mechanism for small scale environments and are interoperable with other vendors that adhere to VXLAN standards. Because you are simply mapping which VTEPs are in a particular VNI, you can avoid the tedious process of defining connections to every VLAN on every other VTEP on every other rack.
Cumulus Networks supports static VXLAN tunnels only on switches in the Cumulus Linux HCL using the Broadcom Tomahawk, Trident II+ and Trident II ASICs, as well as the Mellanox Spectrum ASIC.
For a basic VXLAN configuration, make sure that:
- The VXLAN has a network identifier (VNI); do not use 0 or 16777215 as the VNI ID, which are reserved values under Cumulus Linux.
- The VXLAN link and local interfaces are added to bridge to create the association between port, VLAN, and VXLAN instance.
Each traditional bridge on the switch has only one VXLAN interface. Cumulus Linux does not support more than one VXLAN ID per traditional bridge.
When deploying VXLAN with a VLAN-aware bridge, there is no restriction on using a single VNI. This limitation is only present when using the traditional bridge configuration.
The VXLAN registration daemon (
vxrd) is not running. Static VXLAN tunnels do not interoperate with LNV or EVPN. If
vxrdis running, stop it with the following command:
The following topology is used in this chapter. Each IP address corresponds to the loopback address of the switch.
Configure Static VXLAN Tunnels
To configure static VXLAN tunnels, do the following for each leaf:
- Specify an IP address for the loopback
- Create a VXLAN interface using the loopback address for the local tunnel IP address
- Create the tunnels by configuring the remote IP address to each other leaf switch's loopback address
To configure leaf01, run the following commands:
These commands create the following configuration in the
Repeat these steps for leaf02, leaf03, and leaf04:
|Node||NCLU Commands||/etc/network/interfaces Configuration|
Verify the Configuration
After you configure all the leaf switches, check for replication entries:
Caveats and Errata
Cumulus Linux does not support different
bridge-learning settings for different VNIs of VXLAN tunnels between 2 VTEPs. For example, the following configuration in the
/etc/network/interfaces file is not supported.