[virt-tools-list] pre/post backup script for KVM guests

[Daniel Berteaud]

Le mercredi 21 octobre 2009 à 11:56 +0300, Nikolai K. Bochev a écrit :
> Hello Daniel.

Hi

> 
> I have one question - does BackupPC backup sparse files correctly ? 

Well, I'm not sure, I never tried. But before BackupPC (which is quite
independant from this script, even if I wrote it with BackupPC in mind),
dd need to handle sparse file (I don't know if it does):
- the script uses dd to dump the block devices or files representing the
virtual disks (and optionnaly compress it)
- then BackupPC (or what ever backup tool you want to use) will backup
these dumps

But usually, even if sparse file are not supported and are treated like
normal files, if you compress it (on the fly), you'll get about the same
size. I use pbzip2, which gives quite a high compression ration, and can
use all the cores of my host's CPU, and doesn't slow down the process
too much (Xeon 5520). If pbzip2 is too slow, you can try lzop, which
should just as fast as your disk.


> I am working on a python script to automate the whole process ( something among the lines of what your script does ) i.e. :
> 
> 1. Taking a lvm snapshot
> 2. mounting the snapshot
> 3. backing up the kvm images ( selectively  - storebackup supports conditions - i.e. i wouldn't want to backup the iso files as i did above )
> 4. unmounting and destroying the lvm snapshot.

My script can use LVM snapshots only if virtual disk are directly a LV.
It doesn't use snapshots if you have a file system on a LV where you
store images files(.raw, .qcow2 etc...)

Regards.

> 
> I could scrap the above for your script, but the disk usage of backups is critical for me.
> 
> ----- Original Message -----
> From: "Daniel Berteaud" <daniel at firewall-services.com>
> To: virt-tools-list at redhat.com, backuppc-users at lists.sourceforge.net
> Cc: tech at firewall-services.com
> Sent: Monday, October 19, 2009 4:02:58 PM
> Subject: [virt-tools-list] pre/post backup script for KVM guests
> 
> Hi everyone.
> 
> I've written a script to backup virtual machines managed by libvirt
> (only tested with KVM guests, but should works for Xen too, maybe others
> as well).
> 
> It's called virt-backup.pl
> 
> I've written it for integration with BackupPC:
> - take a dump of a VM in using the pre-backup facility
> - backup the dumps using BackupPC
> - cleanup the dumps in the post-backup phase
> 
> This script can be used outside of BackupPC as it's quite generic.
> 
> There are two main mode for this script:
> --pre: take the backup of the VM. This is called --pre because I use it
> as a pre-backup script for BackupPC
> --post: cleanup the dumps
> 
> Here're some functionalities:
> 
> - no configuration file needed, everything can be passed as command line
> arg
> - can take snapshots of virtual disks if they are on LVM Logical volume
> (the default is to try LVM in any case, if it's not possible, then, just
> dump the block device/file)
> - Supports backup of running VM with minimal downtime (if each virtual
> disks can be snapshoted, just resume the VM immediately resulting in
> just a few seconds of downtime, then, dump the snapshots. If snapshots
> are not available, the guest is suspended during the dump)
> - can save the state of running vm (equivalent of virsh save/virsh
> restore). This is optional because it's still not very reliable, and
> sometimes the restoration fails, leaving a crashed qemu process running
> and eating CPU cycles.
> - Can compress on-the-fly the virtual disks dumps
> (gzip,bzip2,pbzip2,lzop,xz)
> - Support virtual disks exclusions (if you want to backup the system
> disk of a VM, but not the data one for example)
> - Can work on installations where virtual disks are stored on one hosts,
> and guests runs on another on (NFS, iscsi etc... in any case, the script
> must be run on the host which holds the virtual disks)
> - Can backup as many guests as you want in one run (they'll be dumped
> sequentially)
> - Backups are run with low priority (nice and ionice), so it should'nt
> slow down too much your system.
> 
> Here are the dependencies for this script to work
> - Sys::Virt perl module
> - XML::Simple per module
> - Getopt::Long perl module
> - lvm2
> - gzip (optional)
> - bzip2 (optional)
> - pbzip2 (optional)
> - lzop (optional)
> - xz (optional)
> 
> You can run this script without argument to see the help. Edit it if you
> want more informations (there are some examples on how to use it at the
> beginning of the script).
> 
> 
> The script can be found here:
> 
> http://repo.firewall-services.com/misc/virt-backup.pl
> 
> Regards, Daniel
> 
-- 
Daniel Berteaud
FIREWALL-SERVICES SARL.
Société de Services en Logiciels Libres
Technopôle Montesquieu
33650 MARTILLAC
Tel : 05 56 64 15 32
Fax : 05 56 64 15 32
Mail: daniel at firewall-services.com
Web : http://www.firewall-services.com