How to effectively snapshot and revert with Btrfs
This post will provide guidance on a few different use cases that are facilitated by Btrfs’s copy-on-write (COW) capability.
By the end of this post you should have a working knowledge of how to create snapshots on Btrfs1Alternate spellings: BTRFS and btrfs. and how to revert changes when needed. What follows are code snippets (with some commentary) as well as a summary of the relevant foundational concepts.2Btrfs subvolumes (also toplevel and default) and snapshots, and the Snapper conveniences. Those who are uninitiated and interested in the core concepts3Either out of curiosity and a desire to understand or out of a need to materially alter the code snippets. are encouraged to read the Background section before the How-Tos section.
Let’s dig in!
How-Tos
Support for reverting to filesystem snapshots varies across different operating systems (depending on installer defaults as well as OS configuration). While the Btrfs filesystem implementation is part of the Linux kernel, that by itself isn’t sufficient for some cases. The situation is summarized below.
openSUSE | Fedora | Debian and derivatives | |
---|---|---|---|
Revert Python environment | ✅ | ✅ | ✅ If $HOME is on a btrfs volume |
Revert Flatpak state | ✅ | ✅ | ✅ If $HOME (similarly, / ) is on a btrfs volume for user-level (similarly, system-level) |
Revert OS state from the GRUB menu | ✅ | ❌ | ❌ |
The necessary (but not sufficient)4 We also need to ensure that all relevant files are included and that snapshots are being taken in a timely manner. prerequisites for being able to revert on Btrfs are twofold.
- The location in question is a Btrfs subvolume.
- There exists a snapper configuration for the subvolume in question.
Convert an existing directory to a subvolume
Btrfs doesn’t provide any utilities (aka porcelain) to convert an existing directory into a subvolume. However it does provide primitives (aka plumbing) that allow us to create a subvolume where there previously existed a directory. There are two different ways.
Create subvolume and copy content over
This method may create new extents and as such isn’t recommended for large directories, since it may take longer and also use greater space.
1: # Create empty subvolume in place of directory 2: mv /parent/dir /parent/dir-old 3: sudo btrfs subvol create /parent/dir 4: # Copy contents over 5: mv /parent/dir-old/* /parent/dir-old/.* /parent/dir/ 6: # Delete what should now be an empty directory 7: rm -d /parent/dir-old
Create subvolume as snapshot and clean up as needed
The previous approach, which copied files across a subvolume boundary, can result in new file extents being created. The way to ensure that that doesn’t happen is to create a subvolume via the snapshot mechanism. This is a little more involved.
First, we create a new subvolume as a Btrfs snapshot
1: sudo btrfs subvolume snapshot /parent /parent/dir-new
Then we cleanup subvolume contents by deleting unneeded items and moving needed items to the root of the subvolume
2: # [1/3]: delete unneeded items 3: cd /parent/dir-new 4: for dir in ./*; do 5: [ "$dir" = ./"dir" ] && continue 6: rm -rf "$dir" 7: done 8: rm -rf ./.* # also the hidden files 9: # [2/3]: move items to root of subvolume 10: mv /parent/dir-new/dir /parent/dir-new/dir-ref-to-old 11: mv /parent/dir-new/dir-ref-to-old/* /parent/dir-new/dir-ref-to-old/.* /parent/dir-new/ 12: # [3/3]: cleanup: delete what should now be an empty directory 13: rm -d /parent/dir-new/dir-ref-to-old
Replace the directory with the subvolume
14: rm -rf /parent/dir && mv /parent/dir-new /parent/dir
Convert $HOME
to a subvolume
OpenSUSE documentation
has a
section on enabling Snapper in user home directories. That guide is intended
to allow individual users to manage their snapshots and rollbacks
independently. It requires that the user directories are subvolumes of /home
and accomplishes this via the
pam_snapper_homeconvert.sh
script, which is a part of the
pam_snapper
package on openSUSE. The pam_snapper_homeconvert.sh
script can
be used both to create additional users with their $HOME
as a subvolume as
well as to convert the $HOME
of an existing user into a subvolume.5See the
relevant
CONVERTUSERDIR
flag in
pam_snapper_homeconvert.sh
.
For users of
openSUSE LEAP,
the official documentation is probably the best source to consult. For users
of openSUSE Tumbleweed, the steps in
the LEAP documentation can probably be followed with minimal changes. For
users of other operating systems, the code of
the pam_snapper_homeconvert.sh
script is worth perusing.
Now that we’ve covered some ways of creating subvolumes, let’s consider some practical situations where this knowledge can be put to use.
Revert Python environment
Only relevant if you use a user-level package manager for Python such as Conda, Mamba, or pip.
Python package installations aren’t reproducible in general, unless you’re using something like Nix. Thus, you may encounter situations6While Conda and Mamba take care to guard against the environment having conflicting dependencies, the environment may still break after an upgrade, in non-obvious ways, when the package dependencies don’t accurately reflect reality. Meanwhile, Pip will only inform you that an environment has conflicting packages after the fact. where the only reasonable way to fix a broken environment is by reverting to a previously-working state. For this to work we need to:
- ensure that the Python environment(s) is in a Btrfs subvolume
- have an associated snapper configuration
- ensure that we take timely snapshots
Have Python environments in a Btrfs subvolume
There are several ways of creating and managing Python virtual environments. Conda and Mamba are two notable options.7 Conda is a cross-platform package and environment management system. Mamba adds features on top of Conda offering “higher speed and more reliable environment solutions”.
If Conda (or Mamba) is already installed, follow the steps to convert an existing directory to a subvolume to convert the root prefix into a subvolume.
Otherwise, if neither is installed on the system, the easiest way is to:
Create a subvolume to host the virtual environments (assuming the current user is
user
).1: sudo btrfs subvolume create /home/user/miniconda3 2: sudo chown user:user /home/user/miniconda3
- Then
install
conda
using it as the installation prefix. Finally (optionally) install
mamba
in the base environment.4: conda install conda-forge::mamba -n base
Create a Snapper configuration
Assuming that snapper
is installed,8Snapper binaries for different
operating systems can be downloaded from the
openSUSE website. the Python environment root prefix is
/home/user/miniconda3
, and the user is user
:
We first create a snapper configuration.
1: sudo snapper -c home_user_conda create-config /home/user/miniconda3
By default snapper requires root access to take and list snapshots. Assuming that the snapper configurations are stored in
/etc/snapper/configs/
, we take steps to grant access to non-root users.2: sudo sed -i -e "s/ALLOW_USERS=\"\"/ALLOW_USERS=\"user\"/g" /etc/snapper/configs/home_user_conda 3: # allow user to browse .snapshots without 'sudo' 4: # NOTE: it is necessary that the owner of the snapshot directory remain root so 5: # we alter the group to be a group that includes the user 6: sudo chmod a+rx ~/miniconda3/.snapshots 7: sudo chown root:user ~/miniconda3/.snapshots
Ensure that timely Snapper snapshots are taken
The default Snapper configuration is configured to take hourly snapshots. While
this is a reasonable default for the $HOME
directory, we can do better for the
Python environment.
First we alter the configuration created in the previous step to disable hourly snapshots from being created.
1: sudo sed -i -e "s/TIMELINE_CREATE=\"yes\"/TIMELINE_CREATE=\"no\"/g" /etc/snapper/configs/home_user_conda
Then we create wrappers9In addition to delimiting a command with a pre and post snapshot, one can also create standalone snapshots. as desired, and put these in our
~/.bashrc
or equivalent. For instance,conda_update() { snapper -c home_user_conda create -p -d "${CONDA_DEFAULT_ENV}: Update" -c number \ --command "mamba update --all" } conda_install() { snapper -c home_user_conda create -p -d "${CONDA_DEFAULT_ENV}: Installing [${@}]" -c number \ --command "mamba install ""$@" }
Revert using snapper undochange
One can list existing snapper snapshots via
1: snapper -c home_user_conda list
If one has been taking snapshots in pre/post pairs diligently, one can also do
1: snapper -c home_user_conda list -t pre-post
One can see a file-level summary of changes between snapshots via
1: # Format: snapper -c <config> status <start>..<end> 2: snapper -c home_user_conda status 1..2 # changes between 1 and 2 3: snapper -c home_user_conda status 2..0 # changes since 2 (till present)
Diffs can be reviewed between snapshots via
1: # Format: snapper -c <config> diff <start>..<end> [files] 2: snapper -c home_user_conda diff 1..2 /home/user/miniconda3/pkgs/urls.txt
Changes can be reverted via the
undochange
command. For instance in order to restore snapshot version 2, we undo the changes since 2.1: # Format: snapper -c <config> undochange <start>..<end> [files] 2: snapper -c home_user_conda undochange 2..0
Revert Flatpak state10Flatpak is a package manager which allows users to install and run applications independently of the system libraries provided by the OS distribution.
While it’s possible to
install an older build of a Flatpak application, there is no simple
way11Cf. snap revert
. to revert to the state prior to a flatpak
update
. However, if our flatpak installation is in a Btrfs subvolume, we can
use an associated snapper
configuration to allow us to checkpoint on every
update.12Similar to how we revert Python environments. Having created these
checkpoints, we can then revert to them13Since Flatpak stores all its data
(runtimes, applications and configuration) nested within a single directory, it
is easy to see that we can meet the conditions for being able to revert on
Btrfs. as and when needed.
- First we convert
/var/lib/flatpak
14For system-wide usage. For per-user usage, convert$HOME/.local/share/flatpak
instead. into a subvolume using either of the approaches discussed previously. Then we create a snapper configuration similarly to what we did for Python environments.
1: # assuming /var/lib is on a Btrfs partition 2: sudo snapper -c var_lib_flatpak create-config /var/lib/flatpak 3: sudo sed -i -e "s/ALLOW_USERS=\"\"/ALLOW_USERS=\"user\"/g" /etc/snapper/configs/var_lib_flatpak 4: # allow user to browse .snapshots without 'sudo' 5: # NOTE: it is necessary that the owner of the snapshot directory remain root so 6: # we alter the group to be a group that includes the user 7: sudo chmod a+rx /var/lib/flatpak/.snapshots 8: sudo chown root:user /var/lib/flatpak/.snapshots 9: sudo sed -i -e "s/TIMELINE_CREATE=\"yes\"/TIMELINE_CREATE=\"no\"/g" /etc/snapper/configs/var_lib_flatpak
Finally we create some wrappers as desired. Such as,
flatpak_update() { snapper -c var_lib_flatpak create -p -d "flatpak/update" -c number \ --command "sudo flatpak update" } flatpak_install() { snapper -c var_lib_flatpak create -p -d "flatpak/install: ${@}" -c number \ --command "flatpak install ""$@" }
- And then, when needed, we can revert to prior versions
using
snapper undochange
.
Revert OS state from the GRUB menu
In order to be able to boot (from the GRUB menu) into a read-only snapshot and revert to it, openSUSE has implemented some custom functionality:
- Patches to GRUB so that paths15For loading the OS, are resolved wrt the default subvolume as opposed to the toplevel subvolume.16The situation is described in some detail in this mailing list post.
- Hooks, that are invoked after updating system packages, to take snapshots via snapper and then update the default subvolume.17As such, the default subvolume is ephemeral and is subject to change. Specifically, the default subvolume changes at the next system update or system rollback.
Having done so, rolling back the system to a previous snapshot is as simple as:
- Booting into the read-only snapshot from GRUB.
- Verifying that system is working as expected.
Rolling back.
1: sudo snapper rollback 2: systemctl reboot
However, in such systems, care needs to be taken when
creating subvolumes at the root-level.18Because, in these systems, /
happens to refer to the default subvolume, which as noted before is ephemeral,
as opposed to the toplevel subvolume. Specifically, creating a subvolume
located at /
19I.e., creating a subvolume directly at, say, /dirname
. is
to be avoided, since doing so violates the assumptions needed to boot into an
older OS snapshot from GRUB and rolling back via Snapper’s rollback
mechanism. These assumptions require that subvolumes located at /
are robust
to changes in the default subvolume.20For reasons
that are too long for this margin to contain. Let’s consider an instance where
this arises naturally.
Shield Nix 21Nix package manager is a user-level package manager which provides reproducible builds and the ability to roll back to previous versions. package installation when reverting OS state
This is only relevant on platforms where one has the ability to revert OS state (i.e., openSUSE).
We want to ensure that when we revert the root filesystem, that we also don’t
revert the Nix store. Ensuring that the Nix store is within a separate subvolume
is necessary for this purpose. Since Nix uses /nix
as its store location, for
reasons mentioned before, we want to ensure that /nix
is robust to changes
in the default subvolume.22Which is what gets mounted at /
.
We achieve this by following the convention used for other subvolumes mounted
under23As opposed to “mounted at”. /
, and make nix
a subvolume of @
24In openSUSE, the toplevel subvolume contains a single subvolume called
@
. All other subvolumes are descendents of @
. and mount it as /nix
.
Below, code Listing 1 assumes that Nix isn’t yet installed.25If it already is installed, we’d need to follow steps to convert an existing directory to a subvolume.
1: # Mount the top-level subvolume explicitly 2: # mount -o <subvol> <partition> <mount point> 3: # e.g., mount -o subvolid=0 /dev/sda2 /mnt 4: # when using luks encryption, on openSUSE, the decrypted partition is at 5: # /dev/system/root 6: sudo mount -o subvolid=0 /dev/system/root /mnt 7: # On openSUSE, the toplevel subvolume, has a single subvolume within it called 8: # '@'. the '@' subvolume acts as the ancestor of all other subvolumes and 9: # represents a "stable" target 10: sudo btrfs subvol create /mnt/@/nix 11: sudo umount /mnt
12: sudo sh -c 'echo "/dev/system/root /nix btrfs subvol=/@/nix 0 0" >> /etc/fstab' 13: sudo mkdir /nix # make the mount target 14: sudo mount -a # mount /nix (and test the fstab config)
And now we can follow the steps to install Nix.
Background
Btrfs
Btrfs26For the curious, a somewhat dated, but still relevant, article: “A short history of btrfs”, LWN.is an Extent-based, copy-on-write (COW) filesystem27Other notable examples supporting copy-on-write are ZFS and NTFS. for Linux.28That is, not only for GNU/Linux, but even non-GNU distributions such as Alpine Linux. Copy-on-write means that making copies or taking snapshots is extremely light on resources.29I.e., efficient.
Btrfs subvolume30 From the Btrfs documentation on Subvolumes.
A Btrfs subvolume is a part of [the] filesystem with its own independent file/directory hierarchy and inode number namespace. Subvolumes can share file extents. … A subvolume has always inode number 256.31Except in the case of the toplevel subvolume.
A subvolume looks like a normal directory, with some additional operations.… Subvolumes can be renamed or moved, nesting subvolumes is not restricted but has some implications regarding snapshotting. The numeric id (called subvolid or rootid) of the subvolume is persistent and cannot be changed.
Default subvolume
The default subvolume is what gets mounted via mount
unless a subvolid
or
subvol
is specified.32See
Btrfs-specific mount options.
The default subvolume can be queried via
1: sudo btrfs subvolume get-default /
- The default subvolume can be set via
the
set-default
command.
Toplevel subvolume30 From the Btrfs documentation on Subvolumes.
A freshly created [Btrfs] filesystem is also a subvolume, called top-level, internally has [sic] an id [of] 5.33
subvolid=5
, which corresponds to theBtrfs_FS_TREE_OBJECTID
. This subvolume cannot be removed or replaced by another subvolume. This is also the subvolume that will be mounted by default, unless the default subvolume has been changed …
In addition, when mounting, subvolid=0
is an alias for the toplevel
subvolume.34Since
Linux kernel version 2.6.34.
Btrfs snapshot
A snapshot is a subvolume like any other, with given initial content. By default, snapshots are created read-write. File modifications in a snapshot do not affect the files in the original subvolume.35 Btrfs subvolume and snapshot.
[…]
[S]napshotting is not recursive, so a subvolume or a snapshot is effectively a barrier and no files in the nested appear in the snapshot. Instead there’s a stub subvolume (also sometimes empty subvolume with the same name as original subvolume, with inode number 2). This can be used intentionally but could be confusing in case of nested layouts.36Btrfs Nested subvolumes.
Snapper37Not to be confused with snap
and snapcraft
.
Snapper is a utility that allows one to
create and compare snapshots on Btrfs (and
LVM) and revert changes. Some notable subcommands that snapper
provides
are:
create-config
anddelete-config
-
rollback
to revert OS state 38Snapper’srollback
is only relevant for reverting changes to the default subvolume. For reverting changes to other subvolumes, useundochange
. -
undochange
to revert subvolume state -
create
with--command
to create a pre and post snapshot and run command in between
Snapper stores all snapshots in the .snapshots
directory inside the subvolume
in question.39From the
Snapper documentation.
Comments
Comments can be left on twitter, mastodon, as well as below, so have at it.
New post!
— The Weary Travelers blog (@wearyTravlrsBlg) May 21, 2023
Have you ever wanted to set up revertable snapshots using BTRFS? Find out how:https://t.co/sGWuEhszbm
Reply here if you have comments.
Footnotes:
Alternate spellings: BTRFS and btrfs.
Btrfs subvolumes (also toplevel and default) and snapshots, and the Snapper conveniences.
Either out of curiosity and a desire to understand or out of a need to materially alter the code snippets.
We also need to ensure that all relevant files are included and that snapshots are being taken in a timely manner.
See the
relevant
CONVERTUSERDIR
flag in
pam_snapper_homeconvert.sh
.
While Conda and Mamba take care to guard against the environment having conflicting dependencies, the environment may still break after an upgrade, in non-obvious ways, when the package dependencies don’t accurately reflect reality. Meanwhile, Pip will only inform you that an environment has conflicting packages after the fact.
Conda is a cross-platform package and environment management system. Mamba adds features on top of Conda offering “higher speed and more reliable environment solutions”.
Snapper binaries for different operating systems can be downloaded from the openSUSE website.
In addition to delimiting a command with a pre and post snapshot, one can also create standalone snapshots.
Flatpak is a package manager which allows users to install and run applications independently of the system libraries provided by the OS distribution.
Cf. snap revert
.
Similar to how we revert Python environments.
Since Flatpak stores all its data (runtimes, applications and configuration) nested within a single directory, it is easy to see that we can meet the conditions for being able to revert on Btrfs.
For
system-wide usage. For per-user usage, convert
$HOME/.local/share/flatpak
instead.
For loading the OS,
The situation is described in some detail in this mailing list post.
As such, the default subvolume is ephemeral and is subject to change. Specifically, the default subvolume changes at the next system update or system rollback.
Because, in these systems, /
happens to refer to the default subvolume, which as noted before is ephemeral,
as opposed to the toplevel subvolume.
I.e., creating a subvolume directly at, say, /dirname
.
For reasons that are too long for this margin to contain.
Nix package manager is a user-level package manager which provides reproducible builds and the ability to roll back to previous versions.
Which is what gets mounted at /
.
As opposed to “mounted at”.
In openSUSE, the toplevel subvolume contains a single subvolume called
@
. All other subvolumes are descendents of @
.
If it already is installed, we’d need to follow steps to convert an existing directory to a subvolume.
For the curious, a somewhat dated, but still relevant, article: “A short history of btrfs”, LWN.
That is, not only for GNU/Linux, but even non-GNU distributions such as Alpine Linux.
I.e., efficient.
Except in the case of the toplevel subvolume.
subvolid=5
, which corresponds to the
Btrfs_FS_TREE_OBJECTID
.
Since Linux kernel version 2.6.34.
Btrfs Nested subvolumes.
Snapper’s rollback
is only relevant for reverting changes to the default subvolume. For
reverting changes to other subvolumes, use undochange
.