Chapter 133: First DomU Linux¶

What: add the Xen toolstack to Dom0, build a tiny DomU kernel and initramfs, write an xl domain config, and start a second Linux kernel under Xen.

Why: Xen is not interesting until a guest exists. This chapter produces the first real “two Linux systems are alive” result.

Focus: Dom0 controls the platform. DomU receives only the memory, vCPU, console, and devices Xen gives it. xl: Xen’s command-line tool for creating, listing, controlling, and destroying domains. xenstore: Xen’s shared configuration/state database used by domains and tools. event channel: Xen’s lightweight notification mechanism between domains and Xen.

133.1 Boot path¶

We extend Chapter 132:

QEMU -> Xen -> Dom0 Linux -> xl create -> DomU Linux

The new moving parts are:

a Dom0 rootfs with Xen tools,
a DomU kernel,
a DomU initramfs,
an xl config file.

133.2 Why the Chapter 132 Dom0 is too small¶

Chapter 132 used a tiny BusyBox initramfs. It was perfect for proving:

Xen -> Dom0 -> shell

But xl needs more user space:

libraries,
xenstore daemon,
xenfs,
event channel device,
control scripts or minimal replacements.

For this chapter, use a small Buildroot Dom0. That is still buildable, but less fragile than hand-copying every toolstack dependency.

133.3 Buildroot Dom0 shape¶

Create a Buildroot config for ARMv7 with:

Target architecture: ARM little endian
Target ABI: EABIhf
Target CPU: cortex-A15 for QEMU
Init system: BusyBox or systemd, BusyBox is smaller
Root filesystem: cpio gzip

Packages:

Xen tools
dtc, optional
busybox shell utilities

Buildroot symbol names vary. Search:

$ make menuconfig

Then use / search for:

xen

Enable the Xen toolstack package if available in your Buildroot release.

The output should include a rootfs cpio archive and /usr/sbin/xl or /usr/bin/xl inside it.

133.4 Dom0 kernel config¶

Dom0 Linux needs:

CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_HVC_XEN=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y

For the first DomU, block/net backends are optional if the guest uses only initramfs and console. Enable them anyway so Chapter 135 has a smoother path.

133.5 Dom0 startup script¶

Dom0 must mount xenfs and start xenstore if your rootfs does not do it automatically.

Minimal init sequence:

mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs none /dev
mkdir -p /proc/xen
mount -t xenfs none /proc/xen

# Depending on packaging:
xenstored --pid-file /run/xenstored.pid &
sleep 1

Some distributions start xenstored through init scripts. Buildroot packaging may provide a service. The test is not “did a service file exist”, it is:

# xl info

133.6 Dom0 checks before DomU¶

Boot Xen + Dom0 as in Chapter 132, but with the richer Dom0 initramfs.

Inside Dom0:

# which xl
# mount | grep xen
# ls /proc/xen
# ls /dev/xen 2>/dev/null || true
# xl info
# xl list

Expected xl list:

Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0   ...     1     r-----     ...

If xl info fails, stop. DomU cannot work until Dom0 tooling works.

133.7 Build DomU Linux¶

Use a separate kernel build directory:

$ cd ~/imx6ull/xen-qemu/src/linux
$ make ARCH=arm O=~/imx6ull/xen-qemu/build/linux-domu multi_v7_defconfig

Enable:

$ scripts/config --file ~/imx6ull/xen-qemu/build/linux-domu/.config \
    -e XEN \
    -e HVC_XEN \
    -e XEN_DEV_EVTCHN \
    -e XENFS \
    -e BLK_DEV_INITRD \
    -e DEVTMPFS \
    -e DEVTMPFS_MOUNT

$ make ARCH=arm O=~/imx6ull/xen-qemu/build/linux-domu olddefconfig

Build:

$ make ARCH=arm O=~/imx6ull/xen-qemu/build/linux-domu \
    CROSS_COMPILE=$CROSS_COMPILE -j$(nproc) zImage

Copy into Dom0 rootfs or a location Dom0 can read:

/boot/domu-zImage

133.8 Build DomU initramfs¶

DomU /init:

#!/bin/sh
mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs none /dev

echo
echo "================"
echo " DomU is alive"
echo "================"
echo

uname -a
cat /proc/cmdline
head -5 /proc/meminfo

exec /bin/sh

Pack:

$ find . -print0 | cpio --null -ov --format=newc | gzip -9 > domu-initramfs.cpio.gz

Copy into Dom0:

/boot/domu-initramfs.cpio.gz

133.9 Domain config¶

Create /etc/xen/domu-tiny.cfg inside Dom0:

name = "domu-tiny"
kernel = "/boot/domu-zImage"
ramdisk = "/boot/domu-initramfs.cpio.gz"
memory = 64
vcpus = 1
extra = "console=hvc0 rdinit=/init"
on_crash = "preserve"

Line by line:

Line	Meaning
`name`	domain name shown by `xl list`
`kernel`	path inside Dom0 rootfs
`ramdisk`	path inside Dom0 rootfs
`memory`	MiB assigned to DomU
`vcpus`	virtual CPUs assigned to DomU
`extra`	DomU kernel command line
`on_crash`	keep crashed domain for inspection

Start with 64 MiB. It is enough for an initramfs shell and small enough to keep memory pressure visible.

133.10 Start DomU¶

Inside Dom0:

# xl create -c /etc/xen/domu-tiny.cfg

Expected:

DomU is alive
/ #

Detach:

Ctrl-]

List:

# xl list

Attach again:

# xl console domu-tiny

133.11 Prove there are two Linux kernels¶

In Dom0:

# uname -a
# cat /proc/cmdline
# xl list

In DomU:

# uname -a
# cat /proc/cmdline
# head -5 /proc/meminfo

The two command lines should differ:

Dom0 has Xen Dom0 bootargs.
DomU has console=hvc0 rdinit=/init.

The memory sizes should differ too.

133.12 Shut down and restart¶

Try graceful shutdown:

# xl shutdown domu-tiny
# xl list

If the tiny DomU lacks enough user-space support for graceful shutdown:

# xl destroy domu-tiny

Start it again:

# xl create -c /etc/xen/domu-tiny.cfg

Lifecycle matters. A guest that can boot once but not restart is not understood yet.

133.13 Prove failure isolation¶

Inside DomU:

# kill -SEGV 1

If the guest remains alive, try:

# echo c > /proc/sysrq-trigger

This requires sysrq support and may not be enabled. The goal is simple: break DomU.

Back in Dom0:

# xl list
# dmesg | tail -40

Dom0 should survive. Xen should survive. QEMU should keep running.

This is the first concrete hypervisor payoff.

133.14 Save logs¶

Capture:

# xl info > /root/xl-info.txt
# xl list > /root/xl-list-with-domu.txt
# dmesg > /root/dom0-dmesg-after-domu.txt

On the host, save the whole QEMU serial run:

$ qemu-system-arm ... 2>&1 | tee logs/xen-qemu-domu.log

133.15 Failure cases in guest creation¶

Missing kernel files or an impossible memory allocation can stop domain creation. A malformed initramfs can fail later, after the guest kernel has begun executing. A wrong console can conceal that progress without being the cause of a stopped kernel.

These cases belong to different owners: toolstack artifact access, hypervisor resource allocation and guest runtime setup. Treating them all as a failed xl command discards useful diagnostic information.

133.16 Key Notes¶

A DomU is another kernel with assigned memory, virtual CPUs and interfaces. It is not an ordinary process in Dom0’s PID namespace.
Dom0’s toolstack and service state own domain management. A minimal proof-of-boot initramfs does not necessarily contain those management facilities.
Kernel and rootfs paths in a domain configuration are resolved in Dom0, not on the development host.
Guest restart, normal shutdown and forced destruction have different persistence consequences. Lifecycle policy should distinguish them.
Containing one guest failure is evidence for that failure path. It does not establish containment of Dom0, shared hardware or every DMA-capable device.

133.17 Pitfalls¶

Using the tiny Chapter 132 Dom0. It probably lacks xl.
No xenfs. Mount /proc/xen.
xenstored not running. xl info will fail.
Wrong DomU console. Start with console=hvc0.
Too much DomU memory. Use 64 MiB first.
Confusing host files and Dom0 files. kernel = "/boot/domu-zImage" is a path inside Dom0.
Skipping restart. Boot once, destroy, boot again.

133.18 Going deeper¶

Xen xl documentation.
Xen xl.cfg documentation.
Xen event channels.
Xen xenstore documentation.
Chapter 135 for DomU on real i.MX6ULL.