Chapter 129: Build and boot tiny Linux in QEMU¶

What: build a 32-bit ARM Linux kernel, build a static BusyBox initramfs, generate a QEMU Device Tree, and boot to a shell in QEMU.

Why: before Xen, Dom0, DomU, or Jailhouse enter the story, the reader must understand a plain ARM Linux boot with no hidden board support package. Kernel, DTB, initramfs, command line, console. Those are the pieces.

Focus: kernel, Device Tree and initramfs are separate boot inputs with different owners. QEMU makes their interfaces easier to inspect. A failure in one input can produce symptoms in another layer, so the boot contract matters more than the command sequence. QEMU: a machine emulator. Here it creates a fake ARM board with RAM, CPU, UART, timer, interrupt controller, and virtio devices. DTB: Device Tree Blob, the binary hardware description passed to the Linux kernel. initramfs: a cpio archive the kernel unpacks into RAM and runs as the first root filesystem.

129.1 What we are about to build¶

The boot path is:

qemu-system-arm
  -> creates an ARMv7-A "virt" board
  -> loads zImage into guest RAM
  -> loads qemu-virt-arm.dtb into guest RAM
  -> loads initramfs.cpio.gz into guest RAM
  -> starts the kernel
Linux
  -> reads the DTB
  -> initializes the PL011 UART
  -> unpacks the initramfs
  -> runs /init
/init
  -> mounts /proc, /sys, /dev
  -> gives us a BusyBox shell

No U-Boot. No Xen. No Dom0. No virtual disks.

We remove those layers on purpose. If this chapter fails, the failure is in one of four inputs:

kernel image,
DTB,
initramfs,
command line.

That is the mental model we want before Chapter 130 adds U-Boot and Chapter 132 adds Xen.

129.2 Host packages¶

On Ubuntu or Debian:

$ sudo apt update
$ sudo apt install build-essential git bc bison flex libssl-dev \
    gcc-arm-linux-gnueabihf qemu-system-arm device-tree-compiler \
    cpio gzip xz-utils rsync file wget

Check the tools:

$ arm-linux-gnueabihf-gcc --version | head -1
$ qemu-system-arm --version | head -1
$ dtc --version

This chapter uses Debian/Ubuntu’s arm-linux-gnueabihf- toolchain name. If you prefer the book’s project-local Arm GNU toolchain, set:

$ export CROSS_COMPILE=arm-none-linux-gnueabihf-

Otherwise:

$ export CROSS_COMPILE=arm-linux-gnueabihf-

Everything below uses $CROSS_COMPILE.

129.3 Workspace¶

$ mkdir -p ~/imx6ull/virt-lab/{src,build,rootfs,boot,logs}
$ cd ~/imx6ull/virt-lab

By the end, the important files will be:

boot/
  zImage
  qemu-virt-arm.dtb
  qemu-virt-arm.dts
  initramfs.cpio.gz
logs/
  qemu-linux-direct.log
  qemu-linux-wrong-console.log
  qemu-linux-no-initramfs.log
  qemu-linux-no-init.log

129.4 Build BusyBox first¶

We need a user space. The smallest useful one is a static BusyBox.

Download:

$ cd ~/imx6ull/virt-lab/src
$ wget https://busybox.net/downloads/busybox-1.36.1.tar.bz2
$ tar xf busybox-1.36.1.tar.bz2
$ cd busybox-1.36.1

Configure:

$ make ARCH=arm CROSS_COMPILE=$CROSS_COMPILE defconfig

Enable static linking:

$ scripts/config -e STATIC
$ make ARCH=arm CROSS_COMPILE=$CROSS_COMPILE olddefconfig

Build:

$ make ARCH=arm CROSS_COMPILE=$CROSS_COMPILE -j$(nproc)

Inspect the result:

$ file busybox
busybox: ELF 32-bit LSB executable, ARM, EABI5, statically linked, ...

$ ls -lh busybox

The important words are ARM and statically linked. If file says x86-64, you built for the host by accident. If it says dynamically linked, the initramfs will need shared libraries, which we are deliberately avoiding.

129.5 Install BusyBox into the rootfs tree¶

Use BusyBox’s installer. Do not create every applet symlink by hand.

$ rm -rf ~/imx6ull/virt-lab/rootfs/*
$ make ARCH=arm CROSS_COMPILE=$CROSS_COMPILE \
    CONFIG_PREFIX=~/imx6ull/virt-lab/rootfs install

Check:

$ cd ~/imx6ull/virt-lab/rootfs
$ ls
bin  linuxrc  sbin  usr

$ file bin/busybox
bin/busybox: ELF 32-bit LSB executable, ARM, EABI5, statically linked, ...

$ ls -l bin/sh

BusyBox should have installed bin/sh as a symlink to busybox.

Now add the directories Linux user space expects:

$ mkdir -p proc sys dev tmp run mnt root etc
$ chmod 1777 tmp

129.6 Write `/init`¶

The kernel’s first user-space process will be /init because we will pass rdinit=/init.

Create it:

$ cat > init <<'EOF'
#!/bin/sh

mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs none /dev

echo
echo "================================="
echo " QEMU tiny Linux is alive"
echo "================================="
echo

echo "[init] kernel:"
uname -a

echo
echo "[init] command line:"
cat /proc/cmdline

echo
echo "[init] CPU:"
grep -E 'Processor|model name|Hardware|Features' /proc/cpuinfo || true

echo
echo "[init] memory:"
head -5 /proc/meminfo

echo
echo "[init] device-tree compatible:"
tr '\0' '\n' < /sys/firmware/devicetree/base/compatible 2>/dev/null || true

echo
echo "[init] starting shell"
exec /bin/sh
EOF

$ chmod +x init

This script does three important mounts:

/proc: kernel process and system information,
/sys: device model and sysfs,
/dev: device nodes from devtmpfs.

Without /dev, a real system quickly becomes annoying. Without /proc and /sys, you are blind.

129.7 Pack the initramfs¶

From inside the rootfs directory:

$ cd ~/imx6ull/virt-lab/rootfs
$ find . -print0 | cpio --null -ov --format=newc | gzip -9 > ../boot/initramfs.cpio.gz

Inspect the archive without unpacking it:

$ cd ~/imx6ull/virt-lab
$ ls -lh boot/initramfs.cpio.gz
$ gzip -dc boot/initramfs.cpio.gz | cpio -it | head -30

You must see:

.
bin
bin/busybox
bin/sh
init
proc
sys
dev

If /init is missing, the kernel cannot run it. If bin/sh is missing, /init will fail at the final exec.

129.8 Build Linux for QEMU ARM `virt`¶

Get Linux:

$ cd ~/imx6ull/virt-lab/src
$ git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
$ cd linux
$ git checkout v6.6

Configure:

$ make ARCH=arm O=~/imx6ull/virt-lab/build/linux-qemu multi_v7_defconfig

Enable the pieces this boot needs:

$ cd ~/imx6ull/virt-lab/src/linux
$ scripts/config --file ~/imx6ull/virt-lab/build/linux-qemu/.config \
    -e DEVTMPFS \
    -e DEVTMPFS_MOUNT \
    -e BLK_DEV_INITRD \
    -e SERIAL_AMBA_PL011 \
    -e SERIAL_AMBA_PL011_CONSOLE \
    -e VIRTIO \
    -e VIRTIO_MMIO

$ make ARCH=arm O=~/imx6ull/virt-lab/build/linux-qemu olddefconfig

Build:

$ make ARCH=arm O=~/imx6ull/virt-lab/build/linux-qemu \
    CROSS_COMPILE=$CROSS_COMPILE -j$(nproc) zImage

Copy:

$ cp ~/imx6ull/virt-lab/build/linux-qemu/arch/arm/boot/zImage \
    ~/imx6ull/virt-lab/boot/zImage

Inspect:

$ file ~/imx6ull/virt-lab/boot/zImage
$ ls -lh ~/imx6ull/virt-lab/boot/zImage

zImage is a compressed ARM kernel image. It is not an ELF file you can run on the host.

129.9 Get QEMU’s Device Tree¶

The virt machine generates its own hardware description. Dump it:

$ cd ~/imx6ull/virt-lab
$ qemu-system-arm \
    -M virt,dumpdtb=boot/qemu-virt-arm.dtb \
    -cpu cortex-a15 \
    -m 256M \
    -nographic

Depending on QEMU version, it may exit immediately after dumping the DTB or sit with no useful boot. If it sits, quit:

Ctrl-a x

Convert the DTB to readable DTS:

$ dtc -I dtb -O dts -o boot/qemu-virt-arm.dts boot/qemu-virt-arm.dtb

Now inspect the hardware Linux will see:

$ grep -n "compatible" boot/qemu-virt-arm.dts | head
$ grep -n "pl011" boot/qemu-virt-arm.dts
$ grep -n "memory@" boot/qemu-virt-arm.dts
$ grep -n "interrupt-controller" boot/qemu-virt-arm.dts | head

You should find a PL011 UART. That is why the Linux console will be ttyAMA0.

129.10 Boot Linux directly¶

Run:

$ cd ~/imx6ull/virt-lab
$ qemu-system-arm \
    -M virt \
    -cpu cortex-a15 \
    -m 256M \
    -nographic \
    -kernel boot/zImage \
    -dtb boot/qemu-virt-arm.dtb \
    -initrd boot/initramfs.cpio.gz \
    -append "console=ttyAMA0 root=/dev/ram0 rdinit=/init"

Expected output near the end:

Run /init as init process

=================================
 QEMU tiny Linux is alive
=================================

[init] kernel:
Linux (none) ...

[init] command line:
console=ttyAMA0 root=/dev/ram0 rdinit=/init

[init] starting shell
/ #

Save a clean log:

$ qemu-system-arm \
    -M virt \
    -cpu cortex-a15 \
    -m 256M \
    -nographic \
    -kernel boot/zImage \
    -dtb boot/qemu-virt-arm.dtb \
    -initrd boot/initramfs.cpio.gz \
    -append "console=ttyAMA0 root=/dev/ram0 rdinit=/init" \
    2>&1 | tee logs/qemu-linux-direct.log

Inside the guest:

/ # cat /proc/iomem
/ # ls /sys/firmware/devicetree/base
/ # tr '\0' '\n' < /sys/firmware/devicetree/base/compatible
/ # mount
/ # poweroff -f

If poweroff -f does not exit QEMU, use:

Ctrl-a x

129.11 Every QEMU option explained¶

Option	Meaning
`-M virt`	create QEMU’s generic ARM virtual board
`-cpu cortex-a15`	emulate an ARMv7-A CPU suitable for this example
`-m 256M`	give the guest 256 MiB RAM
`-nographic`	connect guest serial to the terminal
`-kernel boot/zImage`	load the Linux kernel directly
`-dtb boot/qemu-virt-arm.dtb`	pass this Device Tree to the kernel
`-initrd boot/initramfs.cpio.gz`	load this cpio archive as initramfs
`-append "..."`	pass the kernel command line

The command line is the virtual hardware bench. Change one piece at a time.

129.12 What `rdinit=/init` did¶

The kernel unpacks the initramfs into a RAM filesystem and looks for a first program.

With:

rdinit=/init

the kernel runs /init from the initramfs.

Without rdinit=, the kernel tries the usual paths:

/sbin/init
/etc/init
/bin/init
/bin/sh

Our initramfs has /init, so we tell the kernel exactly what to run.

129.13 Failure cases in the boot contract¶

Mismatch	Consequence	Boundary
Console does not name the configured UART	Useful output can disappear despite continued execution	Observation path
No usable root filesystem or initramfs	The kernel cannot start initial user space	Kernel-to-user-space handoff
`/init` lacks executable permission or a usable interpreter	Finding the file does not make it executable	Filesystem and exec contract
`/init` is built for another architecture	Execution can fail with a format error	Target ABI
DTB does not describe the modeled machine	Device discovery or startup can fail	Hardware description

QEMU can generate a DTB in some direct-boot paths, so omitting an explicit DTB argument is not by itself a universal failure. What matters is whether the kernel receives a description consistent with the machine. These are diagnostic relationships, not required fault-injection tasks.

129.14 Clean rebuild script¶

Create run-direct.sh:

#!/bin/sh
set -eu

qemu-system-arm \
    -M virt \
    -cpu cortex-a15 \
    -m 256M \
    -nographic \
    -kernel boot/zImage \
    -dtb boot/qemu-virt-arm.dtb \
    -initrd boot/initramfs.cpio.gz \
    -append "console=ttyAMA0 root=/dev/ram0 rdinit=/init"

Make it executable:

$ chmod +x run-direct.sh
$ ./run-direct.sh

This script is the input to Chapter 130. U-Boot will replace QEMU’s direct loader, but the artifacts stay the same.

129.15 Key Notes¶

The kernel, Device Tree and initramfs are separate boot inputs. They provide executable kernel code, a hardware description and the initial user-space tree respectively.
PID 1 is a runtime role with specific failure consequences, not just an arbitrary shell. The /init path must contain an executable compatible with the target and its available interpreter or libraries.
Console routing determines where evidence is visible. An invisible console and an unbooted kernel are different failure cases.
A static BusyBox reduces initramfs dependencies at a footprint cost. Dynamic linking instead requires a consistent loader and library set.
QEMU demonstrates this boot contract, not the physical MINI’s hardware support.

129.16 Pitfalls¶

Building BusyBox for the host. Check with file bin/busybox.
Dynamic BusyBox. Static is simpler for initramfs.
Missing /init executable bit. The kernel cannot execute it.
Wrong console=. QEMU virt uses PL011, which appears as ttyAMA0.
Confusing qemu-system-arm with qemu-arm. qemu-system-arm emulates a whole machine. qemu-arm runs one ARM user-space program.
Assuming QEMU virt equals i.MX6ULL. It does not. It teaches the boot contract.
Changing kernel, DTB, and initramfs together. Change one input at a time.

129.17 Going deeper¶

QEMU Arm virt machine documentation.
Linux ARM boot protocol.
Linux initramfs documentation.
BusyBox build documentation.
Chapter 130: U-Boot in QEMU.