Backup of dom0

[tlaurion]

Your current backup strategy of dom0,/pushing for shutdown scripts should also be pushed forward, enforcing /boot backup into dom0 for restoration should be implemented

[tlaurion]

Encompasses #3

[tasket]

@tlaurion Here is what my current dom0 backup preparation looks like:

rm -v /var/lib/qubes/updates/rpm/*
rsync -av --delete /boot /boot-bak 

And then the 'root' volume is backed up with Wyng.

This is not as complete as it could be, as one could add the output of sfdisk -d and cryptsetup and lvs (and perhaps also the contents of /etc) to the /boot-bak dir for a more thorough description of the system.

As for what can be done with this data during a restore operation, that is a big question. There are probably boot configuration management tools out there that could do a better job than me of reconstructing a system at this level. I think it calls for the involvement of upstream Qubes devs as well, and there may be some strategy I'm unaware of that has to take things like saltstack into account.

For now, as an enabling feature, I would consider adding a --volume option to wyng-util-qubes so that a volume like root could be included in a backup along with the named VMs without having to run Wyng directly to backup such a volume.

[tlaurion]

For now, as an enabling feature, I would consider adding a --volume option to wyng-util-qubes so that a volume like root could be included in a backup along with the named VMs without having to run Wyng directly to backup such a volume.

Looking for it!
With /boot backuped by default and optionally restored.

[Eric678]

I am pretty sure I have this working. I was pushed here by a bad dom0 update and no way to roll it back.
One can save /boot and /boot/efi using --import-other-from along with root in a single wyng session. I have had so many crashes since that bad update I am pretty sure one can safely snap a running root (without having to shut it down) and be sure it will recover fine. Best to do it without any qubes running.
Then to get it back, run up the rescue OS on the qubes install iso, mount the device that has the wyng scripts and wyng archive available, create a .repo file and yum.repos.d directory (I will put up an enhancement request in qubes issues to have that included) as the full repo is already mounted in /run/insfall/repo then "dnf install python3-pycryptodomex" will pull in 160 packages and wyng runs just fine - can full arch-check and take a snap of root (if you wish to get a clean properly shut down copy). I notice that python zlib is pulled in but not zstd and wyng does not complain, so I presume zstd is in the zlib package. One has to cryptsetup the device with qubes_dom0 as well of course.
Noticing the the Qubes device attach tool picks up root.snap and one can attach dom0 to any other qube (yes it is ro) but perhaps it should be smarter, or is there something that can be done to mask it in LV attributes?
If everything looks OK, I wonder if it is quicker(and safer) to just move the root LV and rename root.snap that wyng created, make it rw and run with that?

[kennethrrosen]

I am pretty sure I have this working.

@Eric678 would you be willing to write up a more detailed how-to? And would the process be similar if the user has installed Qubes with BTRFS?

[Eric678]

@kennethrrosen I will post up the scripts that do it all - only there is a problem at the very last step - restoring boot and efi back onto a block device. I guess it has never been tested...

  File "/usr/local/bin/wyng", line 4344, in receive_volume
    punch_hole = save_storage.block_discard_chunk
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'block_discard_chunk'

wyng-backup is just not quite ready for prime time yet - still beta.

[tasket]

For people preferring Btrfs:

Backing up the dom0 root is not going to be as easy as it is with LVM. I am using a small script to put the contents of /home and /etc into a non-networked VM, and including that VM in backups, but I realize that's not what you're aiming for.

Another option is to do a Btrfs snapshot of dom0 root, then btrfs-send that to a file which could be imported in Wyng backups. (Probably best to put /var/lib/qubes in a separate subvol in that case.)

Yet another option, which may be simplest, is to standardize your installation process on using LVM for dom0 root, leaving most drive space free. Then creating a Btrfs Qubes pool with any desired custom fs options is easy. With this hybrid config, you can snapshot the LVM dom0 and easily include it with your Btrfs volumes when backing up (wyng-util-qubes will tell Wyng where to find each storage system automatically).

[tlaurion]

Qubesos needs advice to change btrfs install option from installer. Qubesos would need to deploy /var/lib/qubes as rootfs subvolume so that dom0 can be backup alongside /boot and everything else to have synced backup/restore of whole system.

thinlvm is then superior for backup/restoration use case, until btrfs default installation option changes to do the right thing. There is no way to support this hybrid lvm+btrfs partition scheme for end user repro/programmed reproduction. If the default is bad, then the default should be changed.

[tasket]

ThinLVM might seem better only for the root fs, which is a fairly small thing. Small but important.

Another alternative would be to put dom0 root in a disk image (on Btrfs), which makes it like any other VM volume.

But the simplest way to get from A to B is to have the Qubes installer do its automatic default for LVM, but make the PV (volgroup partition) small at around 30GB. From there, you can decide whether to automate the Btrfs partition setup from the installer, or from a post-install script in dom0 such as the one that sets up VM templates.

---

There are 1,000 ways to configure Linux (and even Qubes) boot. If wyng-util-qubes or its GUI successor is going to handle boot+root restoration directly, it will have to do a lot of sanity checks so it can abort when there is an unfamiliar factor – like, say, dom0 root isn't even on the boot drive, or the root fs isn't ext4, or the fstab has extra entries in /mnt. It will have to cope with disk GUIDs referenced by Grub, fstab, crypttab, etc. And if its acting as a distribution mechanism, then the GUIDs will have to be created anew and replaced in the right spots. So this aspect is much harder than having Time Machine restore MacOS where Apple permits very little variation in the boot config and the archive has a singular purpose (restore a pc).