Xen Test Framework
Xen Test Framework

Introduction

This is both a framework for creating microkernel-based tests, and a suite of tests built using the framework. (This is not the Xen Project's CI / Continuous Integration / automated push gate system. For that, see OSSTest.)

The microkernel boots, runs a set of related tests, and report the overall status on the configured consoles. Automating a suite of tests simply involves booting each microkernel and checking the final printed line on the guest's console.

The build system and library abstractions are specifically designed to make it easy to write code once and compile it for multiple different environments (virtual machines).

The current environments supported are:

Environment Guest Width Paging
pv32pae PV 32bit PAE
pv64 PV 64bit Long mode
hvm32 HVM 32bit None
hvm32pse HVM 32bit PSE
hvm32pae HVM 32bit PAE
hvm64 HVM 64bit Long mode

Getting Started

Requirements:

  • Xen 4.8 or later
  • GNU Make >= 3.81
  • Python 2.7 or later

For x86:

  • GNU compatible 32 and 64-bit toolchain, capable of -std=gnu99, -m64, and -m32.
    • Clang may be used, via CC="clang".
  • Optionally, a toolchain with x32 support.
    • hvm64 tests would prefer to use the elf32-x86-64 format, so they both load and disassemble correctly. In the absence of x32 support, elf32-i386 will be used which will load correctly, but disassemble incorrectly.

To obtain and build:

$ git clone git://xenbits.xen.org/xtf.git
$ cd xtf
$ make -j4

To run tests on a Xen host: (see Errata first)

# ./xtf-runner test-pv64-example
Executing 'xl create -p tests/example/test-pv64-example.cfg'
Parsing config from tests/example/test-pv64-example.cfg
Executing 'xl console test-pv64-example'
Executing 'xl unpause test-pv64-example'
--- Xen Test Framework ---
Environment: PV 64bit (Long mode 4 levels)
Hello World
Test result: SUCCESS

Combined test results:
test-pv64-example                        SUCCESS

Errata

  • For the paths of binaries, xl accepts either an absolute path, or certain relative paths (/etc/xen/ or $CWD for kernel=, $libdir/xen/boot for firmware_override=). The default xtfdir= is configured correctly for running the tests out of the build working tree. If the tests are running elsewhere, use make install DESTDIR=$X xtfdir=$Y to configure absolute paths appropriately for the test system.
  • Some tests require the Xen Force Emulation Prefix to function. FEP deliberately opens the x86 instruction emulator to arbitrary testing, and should not be enabled in production. These tests must be run on a hypervisor configured with CONFIG_HVM_FEP and booted with ‘'hvm_fep’` to enable the runtime use of FEP. These tests will detect if FEP is unavailable and exit with an error.
  • Binutils 2.41 contains a regression causing a build failure:
    Error: operand size mismatch for push.
    
    A fix 7fe76f02413f was backported into Bintuils 2.41 stable branch.

Further Information

Introduction to tests

Introduction to tests

Test Index

Test index

History

The Xen Test Framework grew out of the work done to debug XSA-106. As reported, Xen's instruction emulator failed to perform dpl checks for instructions generating software exceptions, which allowed guest userspace to bypass a security check set up by the guest kernel. Further investigation showed that the exception frames which resulted from emulation were incorrect; specifically that the reported %eip was usually wrong.

A test kernel was written to compare the exception frames of faults and traps when executed by hardware, and when emulated by the instruction emulator. On doing this, it was observed that different hardware was behaving differently (depending on the degree of hardware assistance available), and some CPUs were not behaving as described in the manual.

Once the embargo on XSA-106 lifted, changesets 7dfa94c6212b, ecf5678200ad and 36ebf14ebe60 were the eventual bugfixes which caused Xen to inject software exceptions correctly.

The test was deemed, in principle, to be a good project to invest some time in and to clean up for more general use. Naturally therefore, the cleanup progressed only in the authors copious free time, when more important tasks were not being done.

Moving forward by a year, the author was dismayed to discover that the XSA-156 release contained a regression (causing infinite loops inside guests which used hardware debugging facilities, fixed in 0747bc8b4d85) which would have been caught by the original test for XSA-106, had that test been in a usable state.

It now is.

Licence

The Xen Test Framework is licenced under the BSD 2-clause license
Copyright (c) 2014-2024 Citrix Systems Ltd.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.