This documentation is for an older version of the software. If you are using the current version of Cumulus Linux, this content may not be up to date. The current version of the documentation is available here. If you are redirected to the main page of the user guide, then this page may have been renamed; please search for it there.

What's New in Cumulus Linux 3.0.1

Cumulus Linux 3.0.1 includes bug fixes only.

Cumulus Linux 3.0 has a host of new features and capabilities. In addition to this chapter, please read the release notes to learn about known issues with this release.

Cumulus Linux 3.0 includes these new features and platforms:

Read on to learn about more new functionality and new behaviors.

New Behavior and Functionality

Cumulus Linux 3.0 marks a significant departure from earlier releases of the operating system. As such, some new functionality and behaviors are to be expected.

Cumulus Linux Now Based on Jessie

Cumulus Linux is now based on Debian Jessie, instead of Debian Wheezy. For a list of issues you need to be aware of, please read the Debian documentation.

Quagga Default Configuration Changes

Description

2.5.x Default Configuration

3.x Default Configuration

ospf log-adjacency-changes

Logs a single message when a peer transitions to/from FULL state

On

ospf spf timers

OSPF uses three timers (A, B, C) as an exponential backoff, to prevent consecutive SPFs from hammering the CPU.

  • A: ms from initial event until SPF runs

  • B: ms between consecutive SPF runs (the number doubles with each SPF, until it reaches the value of C)

  • C: Maximum ms between SPFs

  • A: 200

  • B: 1000

  • C: 10000

  • A: 0

  • B: 50

  • C: 5000

bgp log-neighbor-changes

Logs a single message when a peer transitions to/from Established state

On

bgp deterministic-med

Ensures path ordering no longer impactrs bestpath selection

Enabled

bgp default show-hostname

Displays the hostname in show command output.

Enabled

bgp network import-check

Enabled

bgp keepalive timers

60s

3s

bgp hold timers

180s

9s

bgp timers-connect

Controls how long Cumulus Linux waits between attempts to bring up a peer

120s

10s

Additional configuration changes:

  • BGP peer-groups restrictions have been replaced with update-groups, which dynamically examine all peers, and group them if they have the same outbound policy.

  • BGP Min Route Advertisement Interval timers for eBGP and iBGP were set to 0 seconds, rather than 30 seconds for eBGP and 5 seconds for iBGP.

  • IPv6 Route Advertisements are automatically enabled on an interface with IPv6 addresses, so the step no ipv6 nd suppress-ra is no longer needed for BGP unnumbered. The timer interval for RAs remains 600s, which may need to be adjusted to bring up peers quickly.

  • A peer needs to be attached to a peer-group only once, when it then inherits all address-families activated for that peer-group.

  • The default configuration for bgp best path as-path multipath-relax has been changed to no-as-set, as the Quagga implementation produced strange routing scenarios when allowed to create an AS_SET in some situations. An as-set configuration option has been added.

  • BGP multipath is enabled by default; the number of maximum paths defaults to 64.

  • Simplified BGP unnumbered configuration - a single command can configure a neighbor and attach to peer-group:
    neighbor <swpX> interface peer-group <group name>

PowerPC Switches Not Supported

PowerPC switches are not supported under Cumulus Linux 3.0. They are supported under Cumulus Linux 2.5 Extended Service Release (ESR). To see if your switch uses a PowerPC processor, you can either:

Default snmpd Port Binding

In previous releases of Cumulus Linux, the default port binding configuration in /etc/snmp/snmpd.conf was:

# 2.5.x default agent IP address binding (bind to all interfaces on UDP port 161)
agentAddress udp::161

This meant that the snmpd daemon listed and responded to all ports for UDP port 161.

In Cumulus Linux 3.0, the default configuration has been updated to a more secure setting:

# 3.x default agent IP address binding (bind to only loopback interface on UDP port 161)
agentAddress udp:127.0.0.1:161

This ensures that by default, the snmpd daemon will only listen on the loopback interface on UDP port 161, and will only respond to SNMP requests originating on the switch itself, rather than requests coming into the box on an interface. Since this is really only useful for testing purposes, most customers should change this to binding to a specific IP address.

iquerySecName and Rouser

In 2.5.x, default values for iquerySecName and rouser were configured in /etc/snmp/snmpd.conf as follows:

iquerySecName internalUser
rouser internalUser

In 3.x, the default configuration has been updated to a more secure setting, by commenting out the default user:

#iquerySecName internalUser
#rouser internalUser

User accounts must now be created manually for SNMP traps to function correctly.

New Bond Defaults

In order to simplify configurations, many bond settings have had their defaults changed:

Setting2.x Default3.x Default
lacp-ratenone1
miimon0100
min-links01
modenone802.3ad
use-carriernone1
xmit-hash-policynonelayer3+4

New bridge mdb Command Syntax

The syntax of the bridge mdb command has changed slightly. Instead of using vlan <vid> to specify the VLAN ID of a multicast group on a VLAN-aware bridge, Cumulus Linux uses vid <vid>. Similarly, when dumping the MDB with the bridge mdb show command, the VLAN ID, if any, is displayed following the vid keyword.

Adding Static Bridge FDB Entries

To add a static bridge FDB entry, make sure to specify static in the bridge fdb command. For example:

cumulus@switch:~$ sudo bridge fdb add 00:01:02:03:04:06 dev eth0 master static

Printing VLAN Ranges for a Bridge

In order to print a range of VLANs in a bridge, use the -c option with bridge vlan show:

cumulus@switch:~$ bridge -c vlan show

List of Ports for a VLAN No Longer Displayed

The bridge vlan show vlan <vlanid> command in the Linux 4.1 kernel no longer displays the list of ports for a VLAN, unlike in the 3.2 kernel, which did show list of ports for a VLAN.

In addition, the /sys/class/net/<portname>/brport/pvid sysfs node is no longer present in Cumulus Linux.

Expanded Reserved VLAN Range

Cumulus Linux now reserves a range of 1000 VLAN IDs, from 3000 to 3999. Previously, the range was 700 VLANs, numbered 3300 to 3999.

virtio-net Driver Changes

The default speed setting for the virtio-net driver is set to SPEED_10.

In addition, VLAN Tx offload is enabled in the virtio-net driver by default.

MLAG ad_actor_key Setting Change

In Cumulus Linux 3.0, the ad_actor_key parameter for a 10G full-duplex port is set to 13; in Cumulus Linux 2.5.x, the ad_actor_key for the same 10G speed and full-duplex port was set to 33.

New ARP Refresh Rate

For ARP timers, the default base_reachable_time_ms in Cumulus Linux 3.0 and later is 14400000 (4 hours); in Cumulus Linux 2.5.x it is 110000 (110 seconds).

switchd Doesn’t Start if License Isn’t Present

If a license is not installed on a Cumulus Linux switch, the switchd service will not start. If you install the license again, start switchd with:

cumulus@switch:~$ sudo systemctl start switchd.service

SSH to Switch as root User Disabled by Default

To improve security, the ability to use SSH to connect to a switch as the root user using a password has been disabled by default. To enable it, read User Accounts.

SSH Output No Longer Truncated

In Cumulus Linux 2.5.x, depending upon the number of peers on the network, the output of show ip bgp summary json over an SSH session might get truncated. This has been fixed in Cumulus Linux 3.0.

Not All Features Available on Mellanox Platforms

A number of features are not available or are limited on Mellanox switches at this time. These include:

  • ACLs

  • CDP

  • SPAN (however, ERSPAN is supported)

  • VRF

  • VXLAN

  • Resilient hashing

  • 64 MACs (breakout to 25G is limited)

  • sFlow

  • Specific cables supported

Supported Cables for Mellanox Switches

Cumulus Networks has tested and suggests using the following cables and transceivers with Mellanox switches at this time:

Speed: 10G (QSA Adapter Used)

Manufacturer

Label (or internal EEPROM)

Type

Form Factor

Supported Lengths

Supported Speeds

Known Issues?

Mellanox

MFM1T02A-SR

SR

SFP

X

10G

Mellanox

MFM1T02A-LR

LR

SFP

X

10G

Mellanox

MC3309130

DAC

SFP

2M, 3M, 5M, 7M

10G

Speed: 40G

Manufacturer

Label (or internal EEPROM)

Type

Form Factor

Supported Lengths

Supported Speeds

Known Issues?

Finisar

FTL410QE2C

SR

QSFP

X

40G

Mellanox

MC2210411-SR4

SR

QSFP

X

40G

Mellanox

MC2210411-LR4

LR

QSFP

X

40G

Mellanox

MC2210130

DAC

QSFP

1M, 3M, 5M

40G

Mellanox

MC2210310

AoC

QSFP

10M

40G

Ampehnol

APF14190032M3A

DAC

QSFP

3M

40G

JDSU

JQP-04SWAA1

SR

QSFP

X

40G

Speed: 100G

Manufacturer

Label (or internal EEPROM)

Type

Form Factor

Supported Lengths

Supported Speeds

Known Issues?

Mellanox

MCP1600

DAC

QSFP

2M, 3M

40G/100G

Mellanox

MMA1B00

SR

QSFP

X

40G/100G

TE Connectivity

2231368-1

DAC

QSFP

1M, 3M

40G/100G

Ampehnol

NDAAFF

DAC

QSFP

1M

40G/100G

Speed: 40G to 4x10G

Manufacturer

Label (or internal EEPROM)

Type

Form Factor

Supported Lengths

Supported Speeds

Known Issues?

Mellanox

MC2609130

DAC

QSFP to 4xSFP

1M

40G to 4x10G

Ampehnol

NDAQGF-0002 (internal)

DAC

QSFP to 4xSFP

1M, 3M

40G to 4x10G

Speed: 100G to 4x25G

Manufacturer

Label (or internal EEPROM)

Type

Form Factor

Supported Lengths

Supported Speeds

Known Issues?

Mellanox

MCP7F00-A02A

DAC

QSFP to 4xSFP

3M

100G to 4x25G

10Gtek

CAB-ZQP/4ZSP-P1M

DAC

QSFP to 4xSFP

1M

100G to 4x25G

Speed: 40G to 1x10G

Manufacturer

Label (or internal EEPROM)

Type

Form Factor

Supported Lengths

Supported Speeds

Known Issues?

Mellanox

MC2309130

DAC

QSFP to 1xSFP

3M, 5M

40G to 1x10G

Speed: 100G to 2x50G

Manufacturer

Label (or internal EEPROM)

Type

Form Factor

Supported Lengths

Supported Speeds

Known Issues?

Mellanox

MCP7H00

DAC

QSFP to 2xQSFP

1M, 3M, 5M

100G to 2x50G

Yes (with ConnectX4)

Early Access Features

The following early access features are included in Cumulus Linux 3.0:

Removed Features

  • cl-img-install. The installer has been replaced.

  • Disk image slots and /mnt/persist: For information and strategies on how to preserve your network configuration across software upgrades, read the Upgrading Cumulus Linux chapter.

  • cl-ns-mgmt: This experimental feature was introduced in Cumulus Linux 2.1.1 to help users separate their management network from the in-band network. You should use management VRF instead.

  • The following LACP bypass settings are no longer supported: priority mode, bond-lacp-bypass-period, bond-lap-bypass-priority and bond-lap-bypass-all-active .

  • The clag_enable and ad_sys_mac_addr bonding parameters.

  • cl-brctl. This utility was simply a symlink to brctl, which is what you should use to configure bridges, VLANs and the like.

  • jdoo. Use systemd and systemctl for monitoring your switches.