The Cumulus RMP operating system resides on a switch as a disk image. Switches running Cumulus RMP can be configured with multiple disk images. This section discusses how to manage them.
Upgrading Cumulus RMP
If you already have Cumulus RMP installed on your switch and you are upgrading to an X.Y.Z release, like 2.5.7 from an earlier release in the same major and minor release family only (like 2.5.4 to 2.5.7), you can use
apt-get to upgrade to the new version. (If are upgrading to a major (X.0) or minor (X.Y) release, you must do a full image install, as described in Installing a New Cumulus RMP Image below.)
To upgrade to a maintenance (X.Y.Z) release using
- Reboot the switch.
Installing a New Cumulus RMP Image
Cumulus RMP comes preinstalled on your switch. However there may be instances where you need to perform a full image installation. Before you install Cumulus RMP, the switch can be in two different states:
- The switch already has Cumulus RMP installed on it (see below).
- The switch has no image on it (so the switch is only running ONIE) or a clean installation is desired. In which case, you would install Cumulus RMP in one of the following ways:
ONIE is an open source project, equivalent to PXE on servers, that allows installation of network operating systems (NOS) on bare metal switches.
Unlike Cumulus Linux, there is no license to install on a Cumulus RMP switch.
Installing a New Image when Cumulus RMP Is already Installed
Follow these upgrade steps for both major and minor releases, where:
- A major release upgrade is 2.X.X to 3.X.X (like 2.5.7 to 3.0.0)
- A minor release upgrade is X.2.X to X.3.X (like 2.5.4 to 2.5.7)
Installing a new image is a six step process:
- Installing the new image into the alternate image slot (see below).
- Backing up your configuration files into
- Selecting the alternate slot for next boot (that is, the slot you just installed into).
- Rebooting the switch.
- Copying the files from
/mnt/persistto the new slot; this happens automatically if you follow the instructions below.
/mnt/persistout so subsequent reboots don't load
Installing a new image overwrites all files — including configuration files — on the target slot. Cumulus Networks strongly recommends you create a persistent configuration to back up your important files, like your configurations; see Step 2 below.
Step 1: Installing the New Image
cl-img-install command to install a new image into the alternate image slot.
This example assumes the new image is located in the current directory (where the user is running the command from):
Step 2: Backing up Your Configuration Files into /mnt/persist
Any files that have been modified from the factory default should be backed up to
Recommended Files to Make Persistent
Cumulus Networks recommends you consider making the following files and directories part of a persistent configuration.
Network Configuration Files
|File Name and Location||Explanation||Cumulus RMP Documentation||Debian Documentation|
|/etc/network/||Network configuration files, most notably ||Configuring and Managing Network Interfaces||wiki.debian.org/NetworkConfiguration|
|/etc/resolv.conf||DNS resolution||Not unique to Cumulus RMP: wiki.debian.org/NetworkConfiguration#The_resolv.conf_configuration_file||www.debian.org/doc/manuals/debian-reference/ch05.en.html|
|/etc/hostname||Configuration file for the hostname of the switch||Quick Start Guide#ConfiguringtheHostnameandTimeZone||wiki.debian.org/HowTo/ChangeHostname|
|/etc/cumulus/ports.conf||Breakout cable configuration file||Configuring Switch Port Attributes#ConfiguringBreakoutPorts||N/A; please read the guide on breakout cables|
Additional Commonly Used Files
|File Name and Location||Explanation||Cumulus RMP Documentation||Debian Documentation|
|/etc/motd||Message of the day||Not unique to Cumulus RMP||wiki.debian.org/motd#Wheezy|
|/etc/passwd||User account information||Not unique to Cumulus RMP||www.debian.org/doc/manuals/debian-reference/ch04.en.html|
|/etc/shadow||Secure user account information||Not unique to Cumulus RMP||www.debian.org/doc/manuals/debian-reference/ch04.en.html|
|/etc/lldpd.conf||Link Layer Discover Protocol (LLDP) daemon configuration||Link Layer Discovery Protocol||packages.debian.org/wheezy/lldpd|
|/etc/lldpd.d/||Configuration directory for ||Link Layer Discovery Protocol||packages.debian.org/wheezy/lldpd|
|/etc/nsswitch.conf||Name Service Switch (NSS) configuration file||LDAP Authentication and Authorization||wiki.debian.org/LDAP/NSS|
|/etc/ssh/||SSH configuration files||SSH for Remote Access||wiki.debian.org/SSH|
|/etc/ldap/ldap.conf||Lightweight Directory Access Protocol configuration file||LDAP Authentication and Authorization||www.debian.org/doc/manuals/debian-reference/ch04.en.html|
- If you are using the root user account, consider including
- If you have custom user accounts, consider including
Simple Bash Script Example
The following script is a Bash script that can help grab all the above files and push them to
To run the script copy the above into a
.sh file (for example,
sudo nano backup.sh).
To check if the script worked use the Linux
Step 3: Selecting the Alternate Slot for Next Boot
To select the slot you just installed into, either use
cl-img-select -s to switch the primary slot to the alternate slot, or use
cl-img-select with the number of the slot you want directly (for example,
Step 4: Rebooting the Switch
Reboot the switch to boot into the new primary slot.
Step 5: Copying the Files from /mnt/persist to the New Slot
/mnt/persist automatically are rolled into the primary image slot when the switch boots. For example, in this scenario everything in
/mnt/persist gets automatically copied into slot 2 when the reboot is performed in step 4 above. The files in
/mnt/persist keep their relative path after the reboot. For example, if there was a
/mnt/persist/etc/network/interfaces, it would be copied into
Use the tree command to look at the folder structure of
So in this case
/mnt/persist/etc/network/interfaces overrides the primary slot's
/etc/network/interfaces on boot.
Step 6: Clearing /mnt/persist
/mnt/persist is not cleared out, everything in
/mnt/persist will overwrite any relative files in the primary slot whenever the switch boots. This can be a problem is a user modifies some files but forgets to also make the changes to
/mnt/persist. It is best practice to clear out
/mnt/persist so that any subsequent users can make changes and not have them overwritten the next time the switch boots.
This is an extra reminder to clear out
/mnt/persist. A future reboot will cause everything in
/mnt/persist to overwrite the current primary slot.
Full Installation of Cumulus RMP Using ONIE over USB
Follow the steps below to conduct a full installation of Cumulus RMP. This wipes out all pre-existing configuration files that may be present on the switch.
Make sure to back up any important configuration files that you may need to restore the configuration of your switch after the installation finishes.
Preparing for USB Installation
- Download the appropriate Cumulus RMP image for your x86 platform from the Cumulus Downloads page.
Prepare your flash drive by formatting in one of the supported formats: FAT32, vFAT or EXT2.Optional: Preparing a USB Drive inside Cumulus Linux
It is possible that you could severely damage your system with the following utilities, so please use caution when performing the actions below!
- Insert your flash drive into the USB port on the switch running Cumulus RMP and log in to the switch.
- Determine and note which device your flash drive can be found at using output from
sudo fdisk -l [device]. For example,
sudo fdisk -l /dev/sdb. These instructions assume your USB drive is the
/dev/sdbdevice, which is typical. Make sure to modify the commands below to use the proper device for your USB drive.
Create a new partition table on the device:
Create a new partition on the device:
Format the partition to your filesystem of choice using ONE of the examples below:
mkfs.vfat, you need to install the
dosfstoolspackage from the Debian software repositories (step 3 here shows you how to add repositories from Debian), as they are not included by default.
To continue installing Cumulus RMP, mount the USB drive in order to move files to it.
Copy the image file over to the flash drive and rename the image file to
- Insert the USB stick into the switch, then continue with the appropriate instructions below for your x86 platform.
- Prepare the switch for installation:
- If the switch is offline, connect to the console and power on the switch.
- If the switch is already online in Cumulus RMP, connect to the console and reboot the switch into the ONIE environment with the
sudo cl-img-select -icommand, followed by
sudo reboot. Then skip to step 4 below.
- If the switch is already online in ONIE, use the
SSH sessions to the switch get dropped after this step. To complete the remaining instructions, connect to the console of the switch. Cumulus RMP switches display their boot process to the console, so you need to monitor the console specifically to complete the next step.
- Monitor the console and select the ONIE option from the first GRUB screen shown below.
- Cumulus RMP uses GRUB chainloading to present a second GRUB menu specific to the ONIE partition. No action is necessary in this menu to select the default option ONIE: Install OS.
At this point, the USB drive should be automatically recognized and mounted. The image file should be located and automatic installation of Cumulus RMP should begin. Here is some sample output:
After installation completes, the switch automatically reboots into the newly installed instance of Cumulus RMP.
Understanding Image Slots
Cumulus RMP uses the concept of image slots to manage two separate Cumulus RMP images. The slots are described as follows:
- Active image slot: The currently running image slot.
- Primary image slot: The image slot that is selected for the next boot. Often this is the same as the active image slot.
- Alternate image slot: The inactive image slot, not selected for the next boot.
To identify which slot is active, which slot is the primary, and which slot is alternate use the
The above switch is currently running 2.5.7 as indicated by the active. When the switch is rebooted, it will boot into slot 1, as indicated by primary. The alternate slot is running Cumulus RMP 2.5.4 and won't be booted into unless the user selects it.
Each slot is a logical volume in the physical partition, which you can manage with LVM. When Cumulus RMP is installed on an x86 switch, the following entities are created on the disk:
- A disk partition using an ext4 file system that contains three logical volumes: two logical volumes named sysroot1 and sysroot2, and the
/mnt/persistlogical volume. The logical volumes represent the Cumulus RMP image slots, so sysroot1 is slot 1 and sysroot2 is slot 2.
/mnt/persistis where you store your persistent configuration.
- A boot partition, shared by the logical volumes. Each volume mounts this partition as
Managing Slot Sizes
As space in a slot is used, you may need to increase the size of the root filesystem by increasing the size of the corresponding logical volume. This section shows you how to check current utilization and expand the filesystem as needed.
Check utilization on the root filesystem with the
dfcommand. In the following example, filesystem utilization is 16%:
To increase available space in the root filesystem, first use the
vgscommand to check the available space in the volume group. In this example, there is 6.34 Gigabytes of free space available in the volume group CUMULUS:
Once you confirm the available space, determine the number of the currently active slot using
cl-img-selectindicates slot number 1 is active.
- Resize the slot with the
lvresizecommand. The following example increases slot size by 20 percent of total available space. Replace the "#" character in the example with the active slot number from the last step.
cumulus@switch$ sudo lvresize -l +20%FREE CUMULUS/SYSROOT# Extending logical volume SYSROOT# to 5.27 GiB Logical volume SYSROOT# successfully resizedThe use of
+is very important with the
+results in the logical volume size being set directly to the specified size, rather than extended.
Once the slot has been extended, use the
resize2fscommand to expand the filesystem to fit the new space in the slot. Again, replace the "#" character in the example with the active slot number.
Accessing the Alternate Image Slot
It may be useful to access the content of the alternate slot to retrieve configuration or logs.
cl-img-install fails while the alternate slot is mounted. It is important to unmount the alternate slot as shown in step 4 below when done.
- Determine which slot is the alternate with
cumulus@switch$ sudo cl-img-select | grep alt slot 2 (alt ): 2.5.0-199c587-201501081931-build
This output indicates slot 2 is the alternate slot.
- Create a mount point for the alternate slot:
cumulus@switch$ sudo mkdir /mnt/alt
- Mount the alternate slot to the mount point:
cumulus@switch$ sudo mount /dev/mapper/CUMULUS-SYSROOT# /mnt/alt
Where # is the number of the alternate slot.
The alternate slot is now accessible under
- Unmount the mount point
cumulus@switch$ cd / cumulus@switch$ sudo umount /mnt/alt/
Reprovisioning the System (Restart Installer)
You can reprovision the system, wiping out the contents of both image slots and
To initiate the provisioning and installation process, use
cumulus@switch:~$ sudo cl-img-select -i WARNING: WARNING: Operating System install requested. WARNING: This will wipe out all system data. WARNING: Are you sure (y/N)? y Enabling install at next reboot...done. Reboot required to take effect.
If you change your mind, you can cancel a pending reinstall operation by using
cumulus@switch:~$ sudo cl-img-select -c Cancelling pending install at next reboot...done.
Uninstalling All Images and Removing the Configuration
To remove all installed images and configurations, returning the switch to its factory defaults, use
cumulus@switch:~$ sudo cl-img-select -k WARNING: WARNING: Operating System uninstall requested. WARNING: This will wipe out all system data. WARNING: Are you sure (y/N)? y Enabling uninstall at next reboot...done. Reboot required to take effect.
If you change your mind you can cancel a pending uninstall operation by using
cumulus@switch:~$ sudo cl-img-select -c Cancelling pending uninstall at next reboot...done.
Booting into Rescue Mode
If your system becomes broken is some way, you may be able to correct things by booting into ONIE rescue mode. In rescue mode, the file systems are unmounted and you can use various Cumulus RMP utilities to try and fix the problem.
To reboot the system into the ONIE rescue mode, use
cumulus@switch:~$ sudo cl-img-select -r WARNING: WARNING: Rescue boot requested. WARNING: Are you sure (y/N)? y Enabling rescue at next reboot...done. Reboot required to take effect.
If you change your mind you can cancel a pending rescue boot operation by using
cumulus@switch:~$ sudo cl-img-select -c Cancelling pending rescue at next reboot...done.
Inspecting Image File Contents
From a running system you can display the contents of a Cumulus RMP image file using
cumulus@switch:~$ sudo cl-img-pkg -d /var/lib/cumulus/installer/onie-installer Verifying image checksum ... OK. Preparing image archive ... OK. Control File Contents ===================== Description: Cumulus RMP OS-Release: 2.1.0-0556262-201406101128-NB Architecture: amd64 Date: Tue, 10 Jun 2014 11:44:28 -0700 Installer-Version: 1.2 Platforms: im_n29xx_t40n mlx_sx1400_i73612 dell_s6000_s1220 Homepage: http://www.cumulusnetworks.com/ Data Archive Contents ===================== 128 2014-06-10 18:44:26 file.list 44 2014-06-10 18:44:27 file.list.sha1 104276331 2014-06-10 18:44:27 sysroot-internal.tar.gz 44 2014-06-10 18:44:27 sysroot-internal.tar.gz.sha1 5391348 2014-06-10 18:44:26 vmlinuz-initrd.tar.xz 44 2014-06-10 18:44:27 vmlinuz-initrd.tar.xz.sha1 cumulus@switch:~$
You can also extract the image files to the current directory with the
cumulus@switch:~$ sudo cl-img-pkg -e /var/lib/cumulus/installer/onie-installer Verifying image checksum ... OK. Preparing image archive ... OK. file.list file.list.sha1 sysroot-internal.tar.gz sysroot-internal.tar.gz.sha1 vmlinuz-initrd.tar.xz vmlinuz-initrd.tar.xz.sha1 Success: Image files extracted OK. cumulus@switch:~$ sudo ls -l total 107120 -rw-r--r-- 1 1063 3000 128 Jun 10 18:44 file.list -rw-r--r-- 1 1063 3000 44 Jun 10 18:44 file.list.sha1 -rw-r--r-- 1 1063 3000 104276331 Jun 10 18:44 sysroot-internal.tar.gz -rw-r--r-- 1 1063 3000 44 Jun 10 18:44 sysroot-internal.tar.gz.sha1 -rw-r--r-- 1 1063 3000 5391348 Jun 10 18:44 vmlinuz-initrd.tar.xz -rw-r--r-- 1 1063 3000 44 Jun 10 18:44 vmlinuz-initrd.tar.xz.sha1