Switches running Cumulus Linux can integrate with VMware NSX-V to act as hardware VTEP gateways. The VMware NSX-V controller provides consistent provisioning across virtual and physical server infrastructures.
Cumulus Linux also supports integration with VMware NSX in high availability mode. Refer to OVSDB Server High Availability.
Before you integrate VXLANs with NSX-V, make sure you have a layer 2 gateway; a Broadcom Tomahawk, Trident II+, Trident II, Maverick or Mellanox Spectrum switch running Cumulus Linux. Cumulus Linux includes OVSDB server (
ovsdb-server) and VTEPd (
ovs-vtepd), which support VLAN-aware bridges.
To integrate a VXLAN with NSX-V, you need to:
- Configure the NSX-V integration on the switch.
- Configure the transport and logical layers from the NSX Manager.
- Verify the VXLAN configuration.
Cumulus Linux supports security protocol version TLSv1.2 for SSL connections between the OVSDB server and the NSX controller.
The OVSDB server cannot select the loopback interface as the source IP address, causing top of rack registration to the controller to fail. To work around this issue, run the
net add bgp redistribute connected command followed by the
net commit command.
Configure the Switch for NSX-V Integration
Before you start configuring the gateway service, and logical switches and ports that comprise the VXLAN, you need to enable and start the
openvswitch-vtep service, and configure the NSX integration on the switch, either using the script or performing the manual configuration.
Start the openvswitch-vtep Service
To enable and start the
openvswitch-vtep service, run the following command:
In previous versions of Cumulus Linux, you had to edit the
/etc/default/openvswitch-vtep file and then start the
openvswitch-vtep service. Now, you just have to enable and start the
Configure the NSX-V Integration Using the Configuration Script
A script is available so you can configure the NSX-V integration on the switch automatically.
In a terminal session connected to the switch, run the
vtep-bootstrap command with these options:
controller_ipis the IP address of the NSX controller (192.168.100.17 in the example command below).
The ID for the VTEP (
vtep7in the example command below).
The datapath IP address of the VTEP (
172.16.20.157in the example command below). This is the VXLAN anycast IP address.
The IP address of the management interface on the switch (
192.168.100.157in the example command below). This interface is used for control traffic.
Run the following commands in the order shown to complete the configuration process:
Configure the NSX-V Integration Manually
You can configure the NSX-V integration manually for standalone mode only; manual configuration for OVSDB server high availability is not supported.
If you do not want to use the configuration script to configure the NSX-V integration on the switch automatically, you can configure the integration manually, which requires you to perform the following steps:
- Generate a certificate and key pair for authentication by NSX-V.
- Configure a switch as a VTEP gateway.
Generate the Credentials Certificate
In Cumulus Linux, generate a certificate that the NSX controller uses for authentication.
In a terminal session connected to the switch, run the following commands:
/usr/share/openvswitch/scripts/ovs-ctl-vtepfile, make sure the lines containing private-key, certificate, and bootstrap-ca-cert point to the correct files; bootstrap-ca-cert is obtained dynamically the first time the switch talks to the controller:
If files have been moved or regenerated, restart the OVSDB server and VTEPd:
Define the NSX Controller Cluster IP address in OVSDB. This causes the OVSDB server to start contacting the NSX controller:
Define the local IP address on the VTEP for VXLAN tunnel termination. First, find the physical switch name as recorded in OVSDB:
Then set the tunnel source IP address of the VTEP. This is the datapath address of the VTEP, which is typically an address on a loopback interface on the switch that is reachable from the underlying layer 3 network:
After you generate the certificate, keep the terminal session active; you need to paste the certificate into NSX Manager when you configure the VTEP gateway.
Enable ovs-vtepd to Use the VLAN-aware Bridge
By default, in stand-alone mode, the ovs-vtep daemon creates traditional bridges for each VXLAN VTEP. To use the VLAN-aware bridge with the VTEPs, edit the
/usr/share/openvswitch/scripts/ovs-ctl-vtep file and uncomment the
Then restart the OVSDB server and VTEPd:
Provision VMware NSX-V
Configure the Switch as a VTEP Gateway
After you create a certificate, connect to NSX Manager in a browser to configure a Cumulus Linux switch as a hardware VTEP gateway. In this example, the IP address of the NSX Manager is 192.168.110.23.
In NSX Manager, add a new HW VTEP gateway. Click the Network & Security icon, Service Definitions category, then the Hardware Devices tab. Under Hardware Devices, click +. The Create Add Hardware Devices window opens.
In the Name field, provide a name for the HW VTEP gateway.
Enable the BFD service to the service nodes. Select the Enable BFD check box.
From the terminal session connected to the switch where you generated the certificate, copy the certificate and paste it into the Certificate text field. Copy only the bottom portion, including the
END CERTIFICATElines. For example, copy all the highlighted text in the terminal terminal and paste it into NSX Manager:
Click OK to save the gateway.
After communication is established between the switch and the controller, a
controller.cacert file is downloaded onto the switch.
Verify that the controller and switch handshake is successful. In a terminal connected to the switch, run this command:
Configure the Transport and Logical Layers
Configure the Transport Layer
After you finish configuring NSX-V integration on the switch, configure the transport zone and segment ID.
In NSX Manager, click the Logical Network Preparation tab in the Installation category, then click the Segment ID tab.
Click Edit and add the segment IDs (VNIDs) to be used. Here VNIs 5000-5999 are configured.
Click OK to save and provision the segment IDs.
Click the Transport Zones tab, choose the name of the transport zone.
Select Unicast to choose the NSX-V Controller Cluster to handle the VXLAN control plane.
Click OK to save the new transport zone.
Configure the Logical Layer
To complete the integration with NSX-V, you need to configure the logical layer, which requires defining a logical switch (the VXLAN instance) and all the logical ports needed.
To define the logical switch:
In NSX Manager, select the Logical Switches category. Click + to add a logical switch instance.
In the Name field, enter a name for the logical switch.
- In the Transport Zone field, add the transport zone that you created earlier.
In the Replication Mode field, select Unicast for replication by the service node. Then check the Enable IP Discovery check box.
To configure the Replication Cluster:
- Select the Service Definitions category, then click the Hardware Devices tab. Next to the Replication Cluster field, click Edit.
- Hypervisors connected to the NSX controller for replication appear in the Available Objects list. Select the required service nodes, then click the green arrow to move them to the Selected Objects list.
- Click OK to save the replication node configuration.
Define Logical Switch Ports
To define the logical switch ports (you can define a VLAN-to-VNI binding for each switch port associated with a particular logical switch):
In NSX Manager, add a new logical switch port. Click the Logical Switches category. Under Actions, click Manage Hardware Bindings. The Manage Hardware Binding wizard appears.
Click + to add a logical port to the logical switch.
Select the logical switch that you created earlier (5000).
Select the switch port and the corresponding VLAN binding for logical switch 5000. This creates the logical switch port and also maps VLAN 16 of switch port swp2 to VNI 5000.
Click OK to save the logical switch port. Connectivity is established. Repeat this procedure for each logical switch port you want to define.
Verify the VXLAN Configuration
After configuration is complete, you can verify the VXLAN configuration using either or both of these Cumulus Linux commands in a terminal connected to the switch:
To check that the active OVSDB server is connected to the NSX controller, run the
ovsdb-client dump Manager command: