layout-3layout-6layout-1layout-top
layout-toplayout-6layout-1

Link on Physical Switch Port Does not Come Up

Issue

The link on a physical switch port — an interface starting with “swp” — doesn’t come up.

Environment

  • Cumulus Linux, all versions

Resolution

Here is a list of things to check:

Use Suggested Optics, Cables, Transceivers

Please read the pluggables section of the HCL for recommendations on which optics, cables and transceivers to use with your platform.

Ensure Your Cumulus Linux License Is Valid

Until you install a valid Cumulus Linux license, none of the physical switch ports will come up, so they won’t appear in the output of ip link show.

$ sudo cl-license
No license installed!

Please read this article for additional details on how to address this.

Check the output of ip link show, you should see the following:

  • Physical state: UP

  • Admin state: UP

  • LOWER_UP

    cumulus@switch$ ip link show swp1 3: swp1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP mode DEFAULT qlen 500    link/ether c4:54:44:4f:ab:00 brd ff:ff:ff:ff:ff:ff

If the physical switch port is shown as DOWN instead of UP, configure the interface to be UP with the ip link set command:

cumulus@switch$ ip link show swp3
5: swp3: <BROADCAST,MULTICAST> mtu 9000 qdisc pfifo_fast master vlan5 state DOWN mode DEFAULT qlen 500 
    link/ether c4:54:44:4f:ab:00 brd ff:ff:ff:ff:ff:ff
cumulus@switch$ sudo ip link set swp3 up  
cumulus@switch$ ip link show swp3
5: swp3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast master vlan5 state UP mode DEFAULT qlen 500 
    link/ether c4:54:44:4f:ab:00 brd ff:ff:ff:ff:ff:ff

To make this persistent after reboot so the port is up, define the swp in the /etc/network/interfaces file (using net add interface swp3 link up); you can also automate this with Mako, as documented in this article.

cumulus@switch$ net add interface swp3 link up
cumulus@switch$ net commit

Ensure Physical Connections Are Good

In the output of ip link show, if you see NO_CARRIER instead of LOWER_UP, there is no physical link.

cumulus@switch$ ip link show swp21
23: swp21: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast master vlan5 state DOWN mode DEFAULT qlen 500
    link/ether c4:54:44:4f:ab:12 brd ff:ff:ff:ff:ff:ff

Check your connections, reseat the pluggable and verify cables.

Explicitly Configure the Port Speed, Duplex Mode, Auto Negotiation Settings

You may need to explicitly configure the physical switch port attributes, particularly in scenarios where ports may allow different types of pluggables or where the attribute has been reconfigured. The following attributes may need to be explicitly configured:

  • link-speed
  • link-duplex
  • link-autoneg

For example, the following may be configured in /etc/network/interfaces:

auto swp1
iface swp1   
    address 10.1.1.1/24   
    mtu 9000  
    link-speed 10000
    link-duplex full
    link-autoneg off

Note that if the interface name starts with swp and ends with _sX, where X is a number between 0-3, then this is a 10G connection in a breakout Twinax or AOC cable.

cumulus@switch$ ip link show swp32s0
37: swp32s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 500 
    link/ether 00:e0:ec:27:ab:9b brd ff:ff:ff:ff:ff:ff
cumulus@switch$ grep swp32s0 /var/lib/cumulus/porttab 
swp32s0 xe34    0   0
cumulus@switch$ grep xe34 /etc/bcm.d/config.bcm 
port_init_speed_xe34=10000
port_init_autoneg_xe34=0
serdes_if_type_xe34=9

Please Switch Port Attributes for more details on how to configure the physical switch port attributes.

Validate Hardware Settings with ethtool

The ethtool command enables you to query or control the network driver and hardware settings. It takes the device name (like swp1) as an argument, for example:

cumulus@switch$ ethtool swp1

cumulus@switch$ sudo ethtool -m swp1

Please read  Monitoring Interfaces and Transceivers Using ethtool for more details on ethtool.

Force the Interface Configurations to Be Pushed to Hardware

Ensure the configuration in /etc/network/interfaces gets pushed to hardware. Run the following command:

cumulus@switch$ sudo ifreload -a