Mar 20, 2011

Taking snapshots on KVM with libvirt

There is an interesting feature related to the KVM virtualization, the snapshots, which take the disk, memory and device state of a concrete domain at a specific moment. This can have many use cases, such as saving a copy of a virtual machine, preserving a domain's state before a potentially dangerous operation, etc. Snapshots are identified with an unique name and can be taken with the virtual machine turned on.

This feature can only be used with the virsh command and not with the graphical tool (the version of Virtual Machine Manager included on Kubuntu 10.10 is the 0.8.4).

So as to create a snapshot for a particular domain, we must set a XML file with at least a name and description for the snapshot.

javi@javi-kubuntu:~$ cat UbuntuServer_10.10-ss.xml
   <description>Snapshot of OS install and updates</description>

javi@javi-kubuntu:~$ virsh snapshot-create UbuntuServer_10.10 UbuntuServer_10.10-ss.xml
error: Requested operation is not valid: Disk '/var/lib/libvirt/images/UbuntuServer_10.10.img' does not support snapshotting

It is possible that as in the previous output, we get an error message about the virtual disk does not support snapshotting. What is the problem in this case?

The sort of the disk image is raw (default format when you create a virtual machine), which is very simple and does not allow snapshotting. In contradistinction to raw, qcow2 (QEMU image format) supports to have smaller images, encryption, compression, snapshots and so on.

In order to make a qcow2 virtual disk, we can run the following command. In this case, we have reserved all the space for the virtual disk. If we do not want to preallocate the image with metadata, we must take away the "preallocation" option.

javi@javi-kubuntu:~$ sudo qemu-img create -f qcow2 -o preallocation=metadata /var/lib/libvirt/images/UbuntuServer_10.10.qcow2 8G

javi@javi-kubuntu:~$ sudo chown libvirt-qemu:kvm /var/lib/libvirt/images/UbuntuServer_10.10.qcow2

If we want to convert an existing image (for example raw to qcow2), we can use the next order.

javi@javi-kubuntu:/var/lib/libvirt/images$ sudo qemu-img convert -f raw -O qcow2 -o preallocation=metadata UbuntuServer_10.10.raw UbuntuServer_10.10.qcow2

javi@javi-kubuntu:/var/lib/libvirt/images$ sudo chown libvirt-qemu:kvm UbuntuServer_10.10.qcow2

In this moment, we will be able to set a snapshot up with the virsh snapshot-create command. This operation will generate a new XML file.

javi@javi-kubuntu:~$ ls -l /var/lib/libvirt/qemu/snapshot/UbuntuServer_10.10/UbuntuServer_10.10-16032011.xml
-rw------- 1 root root 307 2011-03-16 11:40 /var/lib/libvirt/qemu/snapshot/UbuntuServer_10.10/UbuntuServer_10.10-16032011.xml

javi@javi-kubuntu:~$ virsh snapshot-dumpxml UbuntuServer_10.10 UbuntuServer_10.10-16032011
   <description>Snapshot of OS install and updates</description>

So as to list all the available snapshots for a certain domain, we can type the following order.

javi@javi-kubuntu:~$ virsh snapshot-list UbuntuServer_10.10
Name                        Creation time             Status
UbuntuServer_10.10-16032011 2011-03-16 11:40:22 +0100 shutoff

If at any point we want to bring back a snapshot, we must run the next command.

javi@javi-kubuntu:~$ virsh snapshot-revert UbuntuServer_10.10 UbuntuServer_10.10-16032011

And finally, we also have the option to remove a snapshot.

javi@javi-kubuntu:~$ virsh snapshot-delete UbuntuServer_10.10 UbuntuServer_10.10-16032011


  1. The option -o preallocation=metadata doesn't preallocate the whole disk, though it looks like. With qemu-image info you'll see that. It just makes disk access faster later on for the guest; see for a nice article about this.

  2. Under my personal opinion, I'm afraid that you're wrong, and I'm going to explain you my reasons.

    First of all, I've taken a look at the qemu-img info (on my kubuntu 10.10) and I haven't seen anything about the preallocation option doesn't preallocate the whole disk.

    Then I've revised the article and I've read: "Some time, preallocation does not fill the target file with zeros, but only prepare some its internal data structure"

    What does it mean? "Some time", that is to say, there are times that works ok and other no ok? I cannot understand this point.

    Second, surfing by the net I haven't been able to find any patch or fix related to the problem commented in the article. Could you show me that patch? In short, I mean that the article tells an afirmation but it doesn't demonstrate it.

    And finally, I only base on the facts: when I ask the operating system about the image size, it returns me exactly its real size.

    # qemu-img create -f qcow2 -o preallocation=metadata preallocation.qcow2 1G
    Formatting 'preallocation.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=0 preallocation='metadata'

    # qemu-img create -f qcow2 no_preallocation.qcow2 1G
    Formatting 'no_preallocation.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=0

    # stat preallocation.qcow2
    File: «preallocation.qcow2»
    Size: 1074135040 Blocks: 544 IO Block: 4096 archivo regular
    Device: 806h/2054d Inode: 263099 Links: 1

    # stat no_preallocation.qcow2
    File: «no_preallocation.qcow2»
    Size: 262144 Blocks: 272 IO Block: 4096 archivo regular
    Device: 806h/2054d Inode: 270966 Links: 1

    One more thing, if you open a preallocated qcow2 file with an hex editor, you will be able to see that it's completely filled (you can check it for example with a small file).

  3. Are there any performance implications from using qcow2 as opposed to raw?

  4. Depends on the preallocation. If you previously preallocate all the space for the qcow2 file, the performance is similar to raw. On the contrary, the qcow2 performance is really poor because the hypervisor has to be constantly requesting free space to the operating system.

  5. Javier, i think Jürgen is right and you are wrong. I read it in every place i looked, but after your post i thought i should test it myself:

    $ qemu-img create -f qcow2 pre.qcow2 -o size=10G,preallocation=metadata
    Formatting 'pre.qcow2', fmt=qcow2 size=10737418240 encryption=off cluster_size=0 preallocation='metadata'
    $ qemu-img create -f qcow2 nopre.qcow2 -o size=10G,preallocation=off
    Formatting 'nopre.qcow2', fmt=qcow2 size=10737418240 encryption=off cluster_size=0 preallocation='off'
    $ qemu-img info pre.qcow2
    image: pre.qcow2
    file format: qcow2
    virtual size: 10G (10737418240 bytes)
    disk size: 1.7M
    cluster_size: 65536
    $ qemu-img info nopre.qcow2
    image: nopre.qcow2
    file format: qcow2
    virtual size: 10G (10737418240 bytes)
    disk size: 136K
    cluster_size: 65536

    You see that the preallocated metadata uses around 1.7M for a 10G image, not the full size.

    Your assumption comes from the output stat thats exactly like the one from ls:
    $ ls -l
    insgesamt 1876
    -rw-r--r-- 1 gunni gunni 197120 2011-05-17 15:07 nopre.qcow2
    -rw-r--r-- 1 gunni gunni 10739318784 2011-05-17 15:07 pre.qcow2

    But using du shows you the usage:
    $ du -h *
    136K nopre.qcow2
    1,8M pre.qcow2

    stat and ls show you the total size of a file, even it is a sparse file. I would say with preallocation the file gets created as a sparse file so it uses not the whole image size but only some space for the metadata. I dont know how much this depends on the underlying filesystem that may need sparse file support.

  6. Yes, you are right, thanks!

    This article shows a clear explanation:

    Also say that I have seen a new patch for the preallocation parameter: full. In this way the file is filled with zeros.

  7. I can't seem to get this command to work in CentOS 6, maybe you know why I receive this error...

    error: internal error unable to execute QEMU command 'savevm': The command savevm has not been found

    I've noticed that virt-manager creates that basic XML file if using the "Save" feature in GUI.

    - Trey

  8. It's a little bit weird because I've been checking it out on the Internet and I've not seen any error reported for RHEL. I've seen a thread which deals this issue on Fedora:

    First of all, make sure that your installation is correct. I normally use the documentation provided by Red Hat:

    Maybe, there is a bug in this version of libvirt (0.8.1), so I think that it'd be better that you install the lastest version of libvirt directly from its source code, by compiling it on CentOS.

    There is another way to take snapshots, by means of LVM. I published an article about it:

    And finally, a little tip: for many years I've prefered to use CentOS for production environments,but lately, I realize that Ubuntu Server is much better: most updated packages, repositories with more software and as stable as CentOS.

  9. I was able to get snapshot's was a but that's resolved in CentOS 6 CR from upstream RHEL 6.1. Am now able to use libvirt-0.8.7 which works as expected.

  10. Where are the snapshots stored once they are created?

  11. With reference to the last comment, the snapshots are stored inside the qcow2 file. How are they stored? Does a copy of what is in memory also get put there, and does this mean that the qcow2 file can grow far beyond the size that the virtual machine sees as it is storing much more information?
    Thanks for any clarification.

  12. When you take a snapshot, you get a read-only inmutable copy of the original image state, and then, the original image remains writable.

    In the next article you can see how snapshots works:

  13. Good evening.

    First of all thanks for this tutorial. Second: is a good practice to keep snapshots? I can't see a clear way to 'merge' all of them together.


  14. Hi,

    The main problem of a snapshot is that it takes up much disk space, but if you do not have problem with the space, a snapshot is a good way to make a full backup from time to time.

    In my personal opinion, the best utility of a snapshot is for example when you have to install or upgrade an application, update the operating system, or carry out any critical task on the virtual machine. It is a fast method to recover the previous state in case of failure.

  15. This is 2014, and now a Snapshots command is available in GUI. View -->Snapshots.
    To create a new snapshot, click on the + button. I hope this helps if anyone was still looking for a GUI way.