Early Access Feature
OVSDB server high availability is an early access feature in Cumulus Linux 3.7.
Cumulus Linux supports integration with VMware NSX in both standalone mode and OVSDB server high availability mode (where the data plane is running in active-active mode). For information about VMware NSX in standalone mode and for a description of the components that work together to integrate VMware NSX and Cumulus Linux, see Integrating Hardware VTEPs with VMware NSX-MH or Integrating Hardware VTEPs with VMware NSX-V.
With OVSDB server high availability mode, you use two peer Cumulus Linux switches in an MLAG configuration. Both the MLAG primary and MLAG secondary switch contain OVSDB server and VTEPd. The OVSDB servers synchronize their databases with each other and always maintain the replicated state unless failover occurs; for example, the peer link bond breaks, a switch fails, or the OVSDB server goes down. Both of the VTEPd components talk to the active OVSDB server to read the configuration and then push the configuration to the kernel. Only the active OVSDB server communicates with the NSX controller, unless failover occurs and then the standby OVSDB server takes over automatically. Although the Cumulus switches are configured as an MLAG pair, the NSX controller sees them as a single system (the NSX controller is not aware that multiple switches exist).
The following examples show OVSDB server high availability mode.
Example 1: The OVSDB server on the MLAG primary switch is active. The OVSDB server on the MLAG secondary switch is the hot standby. Only the active OVSDB server communicates with the NSX controller.
Example 2: If failover occurs, the OVSDB server on the MLAG secondary switch becomes the active OVSDB server and communicates with the NSX controller.
When the OVSDB server on the MLAG primary switch starts responding again, it resynchronizes its database, becomes the active OVSDB server, and connects to the controller. At the same time, the OVSDB server on the MLAG secondary switch stops communicating with the NSX controller, synchronizes with the now active OVSDB server, and takes the standby role again.
Before you configure OVSDB server high availability, make sure you have two switches running Cumulus Linux in an MLAG configuration. Cumulus Linux includes OVSDB server (
ovsdb-server) and VTEPd (
ovs-vtepd), which support VLAN-aware bridges.
The following example configuration in the
/etc/network/interfaces file shows the minimum MLAG configuration required (the MLAG peerlink configuration and the dual-connected bonds on the peer switches). The dual-connected bonds are identified in the NSX controller by their
clag-id (single-connected bonds or ports are identified by their usual interface names prepended with the name of the particular switch to which they belong). When you create the Gateway Service for the dual-connected bonds (described in Configuring the Transport and Logical Layers, below), make sure to select the
clag-id named interfaces instead of the underlying individual physical ports. All the logical network configurations are provisioned by the NSX controller.
To configure OVSDB server high availability, you need to:
- Determine on which switch you want to run the active OVSDB server (the MLAG primary switch or the MLAG secondary switch).
- Configure the NSX integration on both switches.
- Configure the Transport and Logical Layers from the NSX Manager.
- Verify the VXLAN Configuration.
The OVSDB server cannot select the loopback interface as the source IP address, causing top of rack registration to the controller to fail. To work around this issue, run the
net add bgp redistribute connected command followed by the
net commit command.
Configure the NSX Integration on the Switch
Before you start configuring the gateway service, the logical switches, and ports that comprise the VXLAN, you need to enable and start the
openvswitch-vtep service, then run the configuration script on both the MLAG primary and MLAG secondary switches. Follow these steps:
Enable and start the
Run the configuration script provided with Cumulus Linux:
- On the switch where you want to run the active OVSDB server, run the
vtep-bootstrapcommand with these options:
db_ha activespecifies that the OVSDB server on this switch is the active server.
db_ha_vipis any unused IP address in the subnet used by the peerlink control subinterface (4094 is typically used). This creates a /32 route that can be reached from either MLAG switch (
169.254.0.11:9998in the example below).
db_ha_repl_svspecifies the IP address of the active OVSDB server (
169.254.0.9:9999in the example command below). The standby OVSDB server uses this IP address to synchronize the database.
controller_portis the port used to communicate with the NSX controller.
controller_ipis the IP address of the NSX controller (192.168.100.17 in the example command below).
The ID for the VTEP (
vtep7in the example command below).
The datapath IP address of the VTEP (
172.16.20.157in the example command below). This is the VXLAN anycast IP address.
- The IP address of the management interface on the switch (
192.168.100.157in the example command below). This interface is used for control traffic.
On the switch where you want to run the standby OVSDB server, run the the
vtep-bootstrapcommand with the same options as above but replace
From the switch running the active OVSDB server, copy the certificate files (
hostname-privkey.pem) to the same location on the switch with the standby OVSDB server.
The certificate and key pairs for authenticating with the NSX controller are generated automatically when you run the configuration script and are stored in the
/home/cumulusdirectory. The same certificate must be used for both switches.
On the switch running the active OVSDB server and then the switch running the standby OVSDB server, run the following commands in the order shown to complete the configuration process:
- On the switch where you want to run the active OVSDB server, run the
For information about the configuration script, read
man vtep-bootstrap or run the command
Configure the Transport and Logical Layers
After you finish configuring the NSX integration on both the MLAG primary and MLAG secondary switch, you need to configure the transport and logical layers from the NSX Manager. Refer to Configuring the Transport and Logical Layers (NSX-MH) or Configuring the Transport and Logical Layers (NSX-V).
After you configure OVSDB server high availability, you can check that configuration is successful.
To check the sync status on the active OVSDB server, run the following command:
To check the sync status on the standby OVSDB server, run the following command:
To check that the active OVSDB server is connected to the NSX controller, run the
ovsdb-client dump Manager command:
To make sure the MLAG configuration is correct, run the
The following example command output shows that MLAG is configured correctly on the active OVSDB server:
The following example command output shows that MLAG is not configured correctly on the active OVSDB server or that the peer is down:
To make sure that the BFD sessions are up and running, run the
ptmctl -b command.
switchd and networking services are restarted, the BFD sessions might be interrupted. If you notice that the sessions are down, restart the
If you encounter interface or VXLAN bridge configuration issues after adding the hardware bindings, run the
ifreload -a command to reload all network interfaces.
If you still encounter issues with high availability after you restart
ifreload -a, and restart
networking.service, reboot the switch running the standby OVSDB server.