I have been using Proxmox as a server OS for >5 years now and have had a very robust and tested backup procedure for all of my VMs. The problem is that though I have a good setup for the VMs I have a very poor system for backing up my Proxmox host.
The solution to this is DD (Data Description) which is a very powerful and dangerous tools for data manipulation. It can handle wiping drives by writing “0” to the entire disk or pull data between sectors on a damaged disk. In my scenario I use it to create a disk image that is a 100% copy of my host.
sudo dd if=/dev/sdX conv=sync,noerror status=progress bs=64K | gzip -c > /media/XXX
To do this I boot into a virtually mounted Xubuntu iso (Thank you Dell IDrac for virtual media). I locate my Proxmox host disk and run the command. It pipes the host disk right into gzip to compress the file and then saves it on the onboard USB.
The dd command paired with gzip makes it an exceedingly attractive backup solution as it is a full bare metal backup that is compressed so the empty space doesn’t fill up the backup.
There are concerns with using dd as a backup, namely with bit rot where if 1 bit is flipped it can mess everything up but to remedy this I am taking multiple backups and I am hosting this in a RaidZ ZFS pool that can catch bit rot.
Most importantly of all, “You don’t have backups unless you tested them” and I was able to successfully recover using this method making it my go to for low level backups.