1 Basics

Status: Supported
Architecture: x86
Component: Toolstack

2 Overview

Migration is a mechanism to move a virtual machine while the VM is running. Live migration moves a running virtual machine between two physical servers, but the same mechanism can be used for non-live migration (pause and copy) and suspend/resume from disk.

3 User details

No hardware requirements, although hypervisor logdirty support is required for live migration.

From the command line, xl migrate/save/restore are the top level interactions. e.g.

xl create my-vm.cfg
xl migrate my-vm localhost

or

xl create my-vm.cfg
xl save my-vm /path/to/save/file
xl restore /path/to/save/file

Xen 4.6 sees the introduction of Migration v2. There is no change for people using xl, although the libxl API has had an extension.

4 Technical details

Migration is formed of several layers. libxc is responsible for the contents of the VM (ram, vcpus, etc) and the live migration loop, while libxl is responsible for items such as emulator state.

The format of the migration v2 stream is specified in two documents, and is architecture neutral. Compatibility with legacy streams is maintained via the convert-legacy-stream script which transforms a legacy stream into a migration v2 stream.

4.1 libxl

With migration v2 support, LIBXL_HAVE_SRM_V2 and LIBXL_HAVE_SRM_V1 are introduced to indicate support. domain_restore_params gains a new parameter, stream_version, which is used to distinguish between legacy and v2 migration streams, and hence whether legacy conversion is required.

5 Limitations

Hypervisor logdirty support is incompatible with hardware passthrough, as IOMMU faults cannot be used to track writes.

While not a bug in migration specifically, VMs are very sensitive to changes in cpuid information, and cpuid levelling support currently has its issues. Extreme care should be taken when migrating VMs between non-identical CPUs until the cpuid levelling improvements are complete.

6 Testing

Changes in libxc should be tested with every guest type (32bit PV, 64bit PV, HVM), while changes in libxl should test HVM guests with both qemu-traditional and qemu-upstream.

In general, testing can be done on a single host using xl save/restore or xl migrate $VM localhost.

Any changes to the conversion script should be tested in all upgrade scenarios, which will involve starting with VMs from Xen 4.5

7 Areas for improvement

8 Known issues

9 References

Xen Developer Summit 2015 Presentation video and slides for Migration v2

10 History

Date Revision Version Notes
2015-10-24 1 Xen 4.6 Document written