If you are using the current version of Cumulus Linux, the content on this page may not be up to date. The current version of the documentation is available here. If you are redirected to the main page of the user guide, then this page may have been renamed; please search for it there.

VXLAN Tunnel DSCP Operations

Cumulus Linux provides configuration options to control DSCP operations during VXLAN encapsulation and decapsulation, specifically for solutions that require end-to-end quality of service, such as RDMA over Converged Ethernet.

The configuration options propagate explicit congestion notification (ECN) between the underlay and overlay and are based on RFC 6040, which describes how to construct the IP header of an ECN field on both ingress to and egress from an IP-in-IP tunnel.

VXLAN Tunnel DSCP operations are supported on Mellanox Spectrum switches only.

Configure DSCP Operations

You can set the following DSCP operations by editing the /etc/cumulus/switchd.conf file. After you modify /etc/cumulus/switchd.conf file, you must restart switchd for the changes to take effect; run the cumulus@switch:~$ sudo systemctl restart switchd.service command.

OptionDescription
vxlan.def_encap_dscp_actionSets the VXLAN outer DSCP action during encapsulation. You can specify one of the following options:
- copy (if the inner packet is IP)
- set (to a specific value)
- derive (from the switch priority).
The default setting is derive.
vxlan.def_encap_dscp_valueIf the vxlan.def_encap_dscp_action option is set, you must specify a value.
xlan.def_decap_dscp_actionSets the VXLAN decapsulation DSCP/COS action. You can specify one of the following options:
- copy (if the inner packet is IP)
- preserve (the inner DSCP is unchanged)
- derive (from the switch priority)

The following example shows that the VXLAN outer DSCP action during encapsulation is set with a value of 16.

cumulus@switch:~$ sudo nano /etc/cumulus/switchd.conf
...
# default vxlan outer dscp action during encap
# {copy | set | derive}
# copy: only if inner packet is IP
# set: to specific value
# derive: from switch priority
vxlan.def_encap_dscp_action = set

# default vxlan encap dscp value, only applicable if action is 'set'
vxlan.def_encap_dscp_value = 16

# default vxlan decap dscp/cos action
# {copy | preserve | derive}
# copy: only if inner packet is IP
# preserve: inner dscp unchanged
# derive: from switch priority
#vxlan.def_decap_dscp_action = derive
...

You can also set the DSCP operations from the command line. Use the echo command to change the settings in the /etc/cumulus/switchd.conf file. For example, to change the encapsulation action to copy:

cumulus@switch:~$ echo "copy" > /cumulus/switchd/config/vxlan/def_encap_dscp_action

To change the VXLAN outer DSCP action during encapsulation to set with a value of 32:

cumulus@switch:~$ echo "32" > /cumulus/switchd/config/vxlan/def_encap_dscp_value
cumulus@switch:~$ echo "set" > /cumulus/switchd/config/vxlan/def_encap_dscp_action

Caveats and Errata

Cumulus Linux supports only the default global settings. Per-VXLAN and per-tunnel granularity are not supported.