The Cumulus Linux bridge driver supports two configuration modes, one that is VLAN-aware, and one that follows a more traditional Linux bridge model.
For traditional Linux bridges, the kernel supports VLANs in the form of VLAN subinterfaces. Enabling bridging on multiple VLANs means configuring a bridge for each VLAN and, for each member port on a bridge, creating one or more VLAN subinterfaces out of that port. This mode poses scalability challenges in terms of configuration size as well as boot time and run time state management, when the number of ports times the number of VLANs becomes large.
The VLAN-aware mode in Cumulus Linux implements a configuration model for large-scale L2 environments, with one single instance of Spanning Tree. Each physical bridge member port is configured with the list of allowed VLANs as well as its port VLAN ID (either PVID or native VLAN — see below). MAC address learning, filtering and forwarding are VLAN-aware. This significantly reduces the configuration size, and eliminates the large overhead of managing the port/VLAN instances as subinterfaces, replacing them with lightweight VLAN bitmaps and state updates.
Configuring a VLAN-aware Bridge
VLAN-aware bridges can be configured with the Network Command Line Utility (NCLU). The example below shows the NCLU commands required to create a VLAN-aware bridge configured for STP, that contains two switch ports, and includes 3 VLANs — the tagged VLANs 100 and 200 and the untagged (native) VLAN of 1:
The following attributes are useful for configuring VLAN-aware bridges:
bridge-vlan-aware: Is automatically set to yes to indicate that the bridge is in VLAN-aware mode.
bridge-pvid: A PVID is the bridge's Primary VLAN Identifer. The PVID defaults to 1; specifying the PVID identifies that VLAN as the native VLAN.
bridge-vids: A VID is the VLAN Identifier, which declares the VLANs associated with this bridge.
bridge-access: Declares the physical switch port as an access port. Access ports ignore all tagged packets; put all untagged packets into the
bridge-allow-untagged: When set to no, it drops any untagged frames for a given switch port.
If you specify
bridge-pvid at the bridge level, these configurations are inherited by all ports in the bridge. However, specifying any of these settings for a specific port overrides the setting in the bridge.
For a definitive list of bridge attributes, run
ifquery --syntax-help and look for the entries under bridge, bridgevlan and mstpctl.
bridge-pvid 1 is implied by default. You do not have to specify
bridge-pvid for a bridge or a port; in this case, the VLAN is untagged. And while it does not hurt the configuration, it helps other users for readability.
The following configurations are identical to each other and the configuration above:
VLAN Filtering/VLAN Pruning
By default, the bridge port inherits the bridge VIDs. A port's configuration can override the bridge VIDs, by using the
Access ports ignore all tagged packets. In the configuration below, swp1 and swp2 are configured as access ports, while all untagged traffic goes to VLAN 100, as specified in the example below:
Dropping Untagged Frames
With VLAN-aware bridge mode, a switch port can be configured to drop any untagged frames. To do this, add
bridge-allow-untagged no to the switch port. This leaves the bridge port without a PVID and drops untagged packets.
Consider the following example bridge:
Here is the VLAN membership for that configuration:
To configure swp2 to drop untagged frames, add
When you check VLAN membership for that port, it shows that there is no untagged VLAN.
VLAN Layer 3 Addressing — Switch Virtual Interfaces and Other VLAN Attributes
When configuring the VLAN attributes for the bridge, specify the attributes for each VLAN interface, each of which is named vlan<vlanid>. If you are configuring the SVI for the native VLAN, you must declare the native VLAN and specify its IP address. Specifying the IP address in the bridge stanza itself returns an error.
These commands create the following configuration in the
In the above configuration, if your switch is configured for multicast routing, you do not need to specify
bridge-igmp-querier-src, as there is no need for a static IGMP querier configuration on the switch. Otherwise, the static IGMP querier configuration helps to probe the hosts to refresh their IGMP reports.
You can specify a range of VLANs as well. For example:
Configuring ARP Timers
Cumulus Linux does not often interact directly with end systems as much as end systems interact with one another. Thus, after a successful address resolution protocol (ARP) places a neighbor into a reachable state, Cumulus Linux may not interact with the client again for a long enough period of time for the neighbor to move into a stale state. To keep neighbors in the reachable state, Cumulus Linux includes a background process (
/usr/bin/arp_refresh) that tracks neighbors that move into a stale, delay or probe state, and attempts to refresh their state ahead of any removal from the Linux kernel, and thus before it would be removed from the hardware forwarding.
The ARP refresh timer defaults to 1080 seconds (18 minutes). You can change this setting by following the procedures outlined in this knowledge base article.
Configuring Multiple Ports in a Range
bridge-ports attribute takes a range of numbers. The "swp1-52" in the example below indicates that swp1 through swp52 are part of the bridge, which is a shortcut that saves you from enumerating each port individually:
These commands create the following configuration in the
Access Ports and Pruned VLANs
The following example configuration contains an access port and switch port that are pruned; they only sends and receive traffic tagged to/from a specific set of VLANs declared by the
bridge-vids attribute. It also contains other switch ports that send and receive traffic from all the defined VLANs.
Large Bond Set Configuration
The configuration below demonstrates a VLAN-aware bridge with a large set of bonds. The bond configurations are generated from a Mako template.
VXLANs with VLAN-aware Bridges
Cumulus Linux supports using VXLANs with VLAN-aware bridge configuration. This provides improved scalability, as multiple VXLANs can be added to a single VLAN-aware bridge. A 1:1 association is used between the VXLAN VNI and the VLAN, using the bridge access VLAN definition on the VXLAN, and the VLAN membership definition on the local bridge member interfaces.
The configuration example below shows the differences between a VXLAN configured for traditional bridge mode and one configured for VLAN-aware mode. The configurations use head end replication (HER), along with the VLAN-aware bridge to map VLANs to VNIs.
The current tested scale limit for Cumulus Linux 3.2 is 512 VNIs.
Configuring a Static MAC Address Entry
You can add a static MAC address entry to the layer 2 table for an interface within the VLAN-aware bridge by running a command similar to the following:
Caveats and Errata
Spanning Tree Protocol (STP)
VLAN-aware mode supports a single instance of STP across all VLANs, as STP is enabled on a per-bridge basis. A common practice when using a single STP instance for all VLANs is to define every VLAN on every switch in the spanning tree instance.
mstpd remains the user space protocol daemon.
Cumulus Linux supports Rapid Spanning Tree Protocol (RSTP).
IGMP snooping and group membership are supported on a per-VLAN basis, though the IGMP snooping configuration (including enable/disable and mrouter ports) are defined on a per-bridge port basis.
Reserved VLAN Range
For hardware data plane internal operations, the switching silicon requires VLANs for every physical port, Linux bridge, and layer 3 subinterface. Cumulus Linux reserves a range of 1000 VLANs by default; the reserved range is 3000-3999. The reserved range can be modified if it conflicts with any user-defined VLANs, as long the new range is a contiguous set of VLANs with IDs anywhere between 2 and 4094, and the minimum size of the range is 300 VLANs.
To configure the reserved range:
/etc/cumulus/switchd.confin a text editor.
Uncomment the following line, specify a new range, and save the file:
switchdto implement the change:While restarting
switchd, all running ports will flap, and forwarding will be interrupted.
A bridge in VLAN-aware mode cannot have VLAN translation enabled for it. Only traditional mode bridges can utilize VLAN translation.
Converting Bridges between Supported Modes
Traditional mode bridges cannot be automatically converted to/from a VLAN-aware bridge. The original configuration must be deleted, and all member switch ports must be brought down, then a new bridge can be created.