1 Dom0less

“Dom0less” is a set of Xen features that enable the deployment of a Xen system without an control domain (often referred to as “dom0”). Each feature can be used independently from the others, unless otherwise stated.

1.1 Booting Multiple Domains from Device Tree

This feature enables Xen to create a set of DomUs at boot time. Information about the DomUs to be created by Xen is passed to the hypervisor via Device Tree. Specifically, the existing Device Tree based Multiboot specification has been extended to allow for multiple domains to be passed to Xen. See docs/misc/arm/device-tree/booting.txt for more information about the Multiboot specification and how to use it.

Currently, a control domain (“dom0”) is still required to manage the DomU domains, but the system can start also without dom0 if the Device Tree doesn’t specify the dom0 kernel and it declares one or more domUs. Instead of waiting for the control domain (when declared) to be fully booted and the Xen tools to become available, domains created by Xen this way are started right away in parallel. Hence, their boot time is typically much shorter.

1.2 Configuration

1.2.1 Loading binaries into memory

U-Boot needs to load not just Xen, the device tree binary, the dom0 kernel and ramdisk. It also needs to load the kernel and ramdisk of any additional domains to boot. For example if this is the bootcmd for Xen and Dom0:

tftpb 0x1280000 xen.dtb
tftpb 0x0x80000 xen-Image
tftpb 0x1400000 xen.ub
tftpb 0x9000000 xen-rootfs.cpio.gz.u-boot

bootm 0x1400000 0x9000000 0x1280000

If we want to add one DomU with Image-DomU as the DomU kernel and ramdisk-DomU as DomU ramdisk:

tftpb 0x1280000 xen.dtb
tftpb 0x80000 xen-Image
tftpb 0x1400000 xen.ub
tftpb 0x9000000 xen-rootfs.cpio.gz.u-boot

tftpb 0x2000000 Image-DomU
tftpb 0x3000000 ramdisk-DomU

bootm 0x1400000 0x9000000 0x1280000

1.2.2 Device Tree configuration

In addition to loading the necessary binaries, we also need to advertise the presence of the additional VM and its configuration. It is done via device tree adding a node under /chosen as follows:

domU1 {
    #address-cells = <1>;
    #size-cells = <1>;
    compatible = "xen,domain";
    memory = <0 0x20000>;
    cpus = <1>;
    vpl011;

    module@2000000 {
        compatible = "multiboot,kernel", "multiboot,module";
        reg = <0x2000000 0xffffff>;
        bootargs = "console=ttyAMA0";
    };

    module@30000000 {
        compatible = "multiboot,ramdisk", "multiboot,module";
        reg = <0x3000000 0xffffff>;
    };
};

Where memory is the memory of the VM in KBs, cpus is the number of cpus. module@2000000 and module@3000000 advertise where the kernel and ramdisk are in memory.

Note: the size specified should exactly match the size of the Kernel/initramfs. Otherwise, they may be unusable in Xen (for instance if they are compressed).

See docs/misc/arm/device-tree/booting.txt for more information.

1.3 PV Drivers

It is possible to use PV drivers with dom0less guests with some restrictions:

After the execution of init-dom0less, it is possible to use “xl” to hotplug PV drivers to dom0less guests. E.g. xl network-attach domU.

The implementation works as follows: - Xen allocates the xenstore event channel for each dom0less domU that has the “xen,enhanced” property, and sets HVM_PARAM_STORE_EVTCHN - Xen does not allocate the xenstore page and sets HVM_PARAM_STORE_PFN to ~0ULL (invalid) - Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to invalid - Old kernels will continue without xenstore support (Note: some old buggy kernels might crash because they don’t check the validity of HVM_PARAM_STORE_PFN before using it! Disable “xen,enhanced” in those cases) - New kernels will wait for a notification on the xenstore event channel (HVM_PARAM_STORE_EVTCHN) before continuing with the initialization - Once dom0 is booted, init-dom0less is executed: - it allocates the xenstore shared page and sets HVM_PARAM_STORE_PFN - it calls xs_introduce_domain - Xenstored notices the new domain, initializes interfaces as usual, and sends an event channel notification to the domain using the xenstore event channel (HVM_PARAM_STORE_EVTCHN) - The Linux domU kernel receives the event channel notification, checks HVM_PARAM_STORE_PFN again and continue with the initialization

1.4 Limitations

Domains started by Xen at boot time currently have the following limitations:

1.5 Notes