Cumulus Linux supports the ability to take snapshots of the complete file system as well as the ability to roll back to a previous snapshot. Snapshots are performed automatically right before and after you upgrade Cumulus Linux and right before and after you commit a switch configuration using NCLU. In addition, you can take a snapshot at any time. You can roll back the entire file system to a specific snapshot or just retrieve specific files.
The primary snapshot components are:
- btrfs — an underlying file system in Cumulus Linux, which supports snapshots.
- snapper — a userspace utility to create and manage snapshots on demand as well as taking snapshots automatically before and after running
apt-get upgrade|install|remove|dist-upgrade. You can use
snapperto roll back to earlier snapshots, view existing snapshots, or delete one or more snapshots.
- NCLU — takes snapshots automatically before and after committing network configurations. You can use NCLU to roll back to earlier snapshots, view existing snapshots, or delete one or more snapshots.
Installing the Snapshot Package
If you're upgrading from a version of Cumulus Linux earlier than version 3.2, you need to install the
cumulus-snapshot package before you can use snapshots.
Taking and Managing Snapshots
As described above, snapshots are taken automatically:
- Before and after you update your switch configuration by running
net commit, via NCLU.
- Before and after you update Cumulus Linux by running
apt-get upgrade|install|remove|dist-upgrade, via
You can also take snapshots as needed using the
snapper utility. Run:
For more information about using
snapper --help or
Viewing Available Snapshots
You can use both NCLU and
snapper to view available snapshots on the switch.
net show commit history only displays snapshots taken when you update your switch configuration. It does not list any snapshots taken directly with
snapper. To see all the snapshots on the switch, run:
Viewing Differences between Snapshots
To see a line by line comparison of changes between two snapshots, run:
You can view the diff for a single file by specifying the name in the command:
For a higher level view, displaying the names of changed/added/deleted files only, run:
You can remove one or more snapshots using both NCLU and snapper.
Take care when deleting a snapshot, as you cannot restore it once it's been deleted.
To remove a single snapshot or a range of them created with NCLU, run:
To remove a single snapshot or a range of snapshots using
Snapshot 0 is the running configuration. You can't roll back to it or delete it. However, you can take a snapshot of it.
Snapshot 1 is the root file system.
snapper utility preserves a number of snapshots, and automatically deletes older snapshots once the limit is reached. It does this in two ways.
snapper preserves 10 snapshots that are labeled important. A snapshot is labeled important if it was created when you run
apt-get. To change this number, run:
You should always make
NUMBER_LIMIT_IMPORTANT an even number since two snapshots are always taken before and after an upgrade. This does not apply to
NUMBER_LIMIT, described next.
snapper also deletes unlabeled snapshots. The default number of snapshots
snapper preserves is 5. To change this number, run:
Also, you can prevent snapshots from being taken automatically before and running
apt-get upgrade|install|remove|dist-upgrade. Edit
/etc/cumulus/apt-snapshot.conf and set:
Rolling Back to Earlier Snapshots
If you need to restore Cumulus Linux to an earlier state, you can roll back to an older snapshot.
For a snapshot created with NCLU, you can revert to a specific snapshot listed in the output from
net show commit history, or you can revert to the previous snapshot by specifying last when you run:
For any snapshot on the switch, you can use
snapper to roll back to a specific snapshot. When running
snapper rollback, you must reboot the switch for the rollback to complete:
You can also revert to an earlier version of a specific file instead of rolling back the whole file system:
You can also copy the file directly from the snapshot directory:
Configuring Automatic Time-based Snapshots
You can configure Cumulus Linux to take hourly snapshots. You need to enable
TIMELINE_CREATE in the snapper configuration:
Caveats and Errata
root Partition Mounted Multiple Times
You may notice that the root partition gets mounted multiple times. This is due to the way the
btrfs file system handles subvolumes, mounting the root partition once for each subvolume.
btrfs keeps one subvolume for each snapshot taken, which stores the snapshot data. While all snapshots are subvolumes, not all subvolumes are snapshots.
Cumulus Linux excludes a number of directories when it takes a snapshot of the root file system (and from any rollbacks):
|/home||Excluded to avoid user data loss on rollbacks.|
Log file and Cumulus support location. Excluded from snapshots to allow post-rollback analysis.
No need to rollback temporary files.
Third-party software usually are installed in /opt. Exclude /opt to avoid re-installing these applications after rollbacks.
Contains data for HTTP and FTP servers. Excluded this directory to avoid server data loss on rollbacks.
This directory is used when installing locally built software. Exclude this directory to avoid re-installing these software after rollbacks.
Exclude this directory to avoid loss of mail after a rollback.
This is the default directory for libvirt VM images. Exclude from the snapshot. Additionally disable Copy-On-Write (COW) for this subvolume as COW and VM image I/O access patterns do not play nice.
/boot/grub/i386-pc, /boot/grub/x86_64-efi, /boot/grub/arm-uboot
The GRUB kernel modules must stay in sync with the GRUB kernel installed in the master boot record or UEFI system partition.