Docker on Cumulus Linux

You can use Cumulus Linux to run the Docker container platform.

On a Mellanox switch, a Docker package is installed as part of the installation or ONIE upgrade process. The Docker package includes Docker Engine, and dependencies and configuration files required to run the Docker service. On a Mellanox switch that was upgraded to Cumulus Linux 4.3.0 or later with apt-upgrade, or on a Broadcom switch, you must install the Docker package manually.

To run Docker containers on the Cumulus Linux switch:

  1. Check if the Docker package is already installed on the switch with the dpkg-query -l cumulus-docker-setup command.

The following command output shows that the Docker package is installed. No further action is required. Go to the next step to enable the Docker service.

cumulus@switch:mgmt:~$ dpkg-query -l cumulus-docker-setup
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                 Version                           Architecture Description
ii  cumulus-docker-setup 1.0-cl4.3.0+u1~1607633230.afa6ce7 all          Cumulus Linux docker configuration files.

The following command output shows that the Docker package is not installed:

cumulus@switch:mgmt:~$ dpkg-query -l cumulus-docker-setup
dpkg-query: no packages found matching cumulus-docker-setup

To install the Docker package, run the following commands:

cumulus@switch:mgmt:~$ sudo -E apt-get update
cumulus@switch:mgmt:~$ sudo -E apt-get install cumulus-docker-setup
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed: docker-ce docker-ce-cli
Suggested packages:
 aufs-tools cgroupfs-mount | cgroup-lite
Recommended packages:
 apparmor docker-ce-rootless-extras libltdl7 pigz
The following NEW packages will be installed: cumulus-docker-setup docker-ce docker-ce-cli
0 upgraded, 4 newly installed, 0 to remove and 6 not upgraded.
Need to get 91.9 MB of archives.
After this operation, 420 MB of additional disk space will be used.
Do you want to continue? [Y/n]
  1. In the managment VRF, enable the Docker service. Docker pulls container images from the internet, which requires internet access through the management VRF.

    cumulus@switch:mgmt:~$ sudo systemctl enable --now docker@mgmt.service
    Created symlink /etc/systemd/system/ → /etc/systemd/system/docker@.service.
    Warning: The unit file, source configuration file or drop-ins of docker@mgmt.service changed on disk. Run 'systemctl daemon-reload' to reload units.

    The warning is a known issue, which has no functional impact.

  2. Check the status of the Docker service with the systemctl status docker@mgmt.service command:

    cumulus@switch:mgmt:~$ sudo systemctl status docker@mgmt.service
    Warning: The unit file, source configuration file or drop-ins of docker@mgmt.service changed on di
    ● docker@mgmt.service - Docker Application Container Engine
       Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
       Drop-In: /run/systemd/generator/docker@.service.d
       Active: active (running) since Tue 2020-12-15 01:02:36 UTC; 7s ago
    Main PID: 9558 (dockerd)
     Memory: 40.5M
     CGroup: /system.slice/system-docker.slice/docker@mgmt.service
           └─9558 /usr/bin/dockerd --containerd=/run/containerd/containerd.sock
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.235571032Z" level=info msg="ccReso
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.235612700Z" level=info msg="Client
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.351654900Z" level=warning msg="Una
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.352171765Z" level=info msg="Loadin
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.432399835Z" level=info msg="Defaul
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.473407023Z" level=info msg="Loadin
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.527590296Z" level=info msg="Docker
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.527846668Z" level=info msg="Daemon
    Dec 15 01:02:36 act-5812-10 systemd[1]: Started Docker Application Container Engine.
    Dec 15 01:02:36 act-5812-10 ip[9558]: time="2020-12-15T01:02:36.635997529Z" level=info msg="API li
  3. Test your installation by running the hello-world container:

    cumulus@switch:mgmt:~$ docker run hello-world
    Unable to find image 'hello-world:latest' locally
    latest: Pulling from library/hello-world
    0e03bdcc26d7: Pull complete
    Digest: sha256:1a523af650137b8accdaed439c17d684df61ee4d74feac151b5b337bd29e7eec
    Status: Downloaded newer image for hello-world:latest
    Hello from Docker!
    This message shows that your installation appears to be working correctly.
    To generate this message, Docker took the following steps:
    1. The Docker client contacted the Docker daemon.
    2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    3. The Docker daemon created a new container from that image which runs the
       executable that produces the output you are currently reading.
    4. The Docker daemon streamed that output to the Docker client, which sent it
       to your terminal.
    To try something more ambitious, you can run an Ubuntu container with:
    $ docker run -it ubuntu bash
    Share images, automate workflows, and more with a free Docker ID:
    For more examples and ideas, visit:

The Docker daemon runs in the management VRF; however, Docker containers run outside a VRF by default. To run a container process inside the management VRF on the host, run the docker run —privileged —ulimit memlock=131072 —rm —network host ip vrf exec mgmt command. For example:

cumulus@switch:mgmt:~$ sudo docker run —privileged —ulimit memlock=131072 —rm —network host debian ip vrf exec mgmt ping -c3

PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=110 time=1.21 ms
64 bytes from icmp_seq=2 ttl=110 time=1.24 ms
64 bytes from icmp_seq=3 ttl=110 time=1.26 ms ping statistics -

3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 1.212/1.237/1.262/0.045 ms

If you see the error Failed to load BPF prog: ‘Operation not permitted’, increase the memlock limit by doubling the value.

Be mindful of the types of applications you want to run in containers on a Cumulus Linux switch. Depending on the configuration of the container, DHCP servers, custom scripts, and other lightweight services run well. However, VPN, NAT and encryption-type services are CPU-intensive and might lead to undesirable effects on critical applications. Resource-intensive services are not supported.