Chapter 138: Bare-metal or Zephyr inmate cell¶
What: build a tiny non-Linux workload and run it as a Jailhouse inmate cell.
Why: the most useful Jailhouse product pattern is often Linux plus one small deterministic workload.
Focus: start with bare metal, then move to Zephyr only after the cell contract is clear.
138.1 Why not start with Zephyr¶
Zephyr is not the hard part.
The hard part is the contract:
where does the inmate start?
where is its RAM?
which CPU runs it?
which MMIO regions can it touch?
which interrupt lines can reach it?
how does it talk to Linux?
If you start with Zephyr and it fails, you have too many possible causes:
wrong Jailhouse cell config
wrong link address
wrong Zephyr board target
wrong devicetree
wrong console driver
wrong interrupt setup
wrong shared memory setup
So we start with a tiny bare-metal inmate. It is not a product. It is a flashlight.
Once the flashlight works, Zephyr becomes much less mysterious.
138.2 The inmate is not a Linux program¶
A normal Linux program starts because the kernel does a lot of work:
loads ELF segments
sets up virtual memory
maps shared libraries
builds a stack
passes argc and argv
jumps to user mode
A Jailhouse inmate does not get that environment.
For a simple bare-metal inmate, you provide:
entry point
link address
stack
exception vector choice if needed
MMIO access code
loop or shutdown path
That is why this command is wrong:
# jailhouse cell load inmate-demo /bin/ls
/bin/ls expects Linux. An inmate binary expects the cell environment.
138.3 The three addresses¶
Keep these three addresses separate:
Address |
Meaning |
|---|---|
Cell memory physical address |
The RAM range assigned in the inmate cell config. |
Inmate link address |
The address the compiler and linker assume at build time. |
Load address |
The address where |
For the simplest bare-metal inmate, make all three match:
cell memory starts at 0x7c000000
binary is linked at 0x7c000000
binary is loaded at 0x7c000000
If the cell config gives the inmate RAM at one address but the binary is linked for another, the first branch or global variable access may go into nonsense.
138.4 Workspace¶
Continue from the previous chapter:
$ mkdir -p ~/imx6ull/jailhouse-lab/inmates/baremetal
$ cd ~/imx6ull/jailhouse-lab/inmates/baremetal
We will create:
start.S
main.c
inmate.ld
Makefile
README-notes.txt
This chapter shows the shape. You will fill in the exact base addresses from your QEMU ARM64 inmate config.
138.5 Read the demo inmate config first¶
Open the demo inmate config:
$ cd ~/imx6ull/jailhouse-lab/src/jailhouse
$ less configs/arm64/qemu-arm64-inmate-demo.c
Find the RAM region assigned to the inmate.
Write it down:
inmate RAM physical start:
inmate RAM size:
inmate load address:
inmate CPU:
inmate console type:
inmate console MMIO base:
Do not write code until those lines are filled.
138.6 The smallest useful startup code¶
Create start.S:
.section .text.start
.global _start
_start:
ldr x0, =_stack_top
mov sp, x0
bl inmate_main
1:
wfi
b 1b
What this does:
set stack pointer
call C code
wait forever when C returns
There is no C runtime here. No main(argc, argv). No libc. No exit syscall.
138.7 The linker script¶
Create inmate.ld.
Replace 0x7c000000 with the inmate RAM start from your cell config:
ENTRY(_start)
MEMORY
{
RAM (rwx) : ORIGIN = 0x7c000000, LENGTH = 16M
}
SECTIONS
{
. = ORIGIN(RAM);
.text : {
*(.text.start)
*(.text*)
} > RAM
.rodata : {
*(.rodata*)
} > RAM
.data : {
*(.data*)
} > RAM
.bss : {
__bss_start = .;
*(.bss*)
*(COMMON)
__bss_end = .;
} > RAM
. = ALIGN(16);
. = ORIGIN(RAM) + LENGTH(RAM);
_stack_top = .;
}
This script says:
put code at the start of inmate RAM
put read only data after code
put initialized data after that
put zero data after that
place stack at the top of inmate RAM
For a real inmate, you may reserve a fixed stack size and detect overflow. For the initial example, this is enough.
138.8 A C file that does not need libc¶
This first version uses a PL011 UART because it is easy to understand. Treat it as a template, not as a guaranteed QEMU ARM64 Jailhouse demo binary.
Create main.c:
#include <stdint.h>
#define UART_BASE 0x09000000UL
#define UART_DR (*(volatile uint32_t *)(UART_BASE + 0x00))
#define UART_FR (*(volatile uint32_t *)(UART_BASE + 0x18))
#define UART_FR_TXFF (1U << 5)
static void uart_putc(char c)
{
while (UART_FR & UART_FR_TXFF) {
}
UART_DR = (uint32_t)c;
}
static void uart_puts(const char *s)
{
while (*s != '\0') {
if (*s == '\n') {
uart_putc('\r');
}
uart_putc(*s);
s++;
}
}
void inmate_main(void)
{
uart_puts("hello from bare-metal Jailhouse inmate\n");
for (;;) {
__asm__ volatile("wfi");
}
}
This assumes a PL011 UART at 0x09000000, which is the QEMU virt UART base used by many ARM examples.
But do not trust the example. Check the inmate config:
If the UART is not assigned to the inmate, this code is wrong.
If Linux still owns that UART, this code is wrong.
If the console uses a virtual Jailhouse console instead, this code is not the right output path.
One device has one owner. That rule is the whole game.
The stock Jailhouse gic-demo.bin is a better first proof because its source and cell config are already matched by the Jailhouse project. Your own PL011 inmate is the next experiment after you have identified which console or MMIO region the inmate really owns.
138.9 Build without libc¶
Create Makefile:
CROSS_COMPILE ?= aarch64-linux-gnu-
CC := $(CROSS_COMPILE)gcc
OBJCOPY := $(CROSS_COMPILE)objcopy
CFLAGS := -Wall -Wextra -ffreestanding -fno-stack-protector -fno-pic \
-mgeneral-regs-only -O2
LDFLAGS := -nostdlib -Wl,-T,inmate.ld -Wl,-Map,inmate.map
all: inmate.bin
inmate.elf: start.S main.c inmate.ld
$(CC) $(CFLAGS) $(LDFLAGS) start.S main.c -o $@
inmate.bin: inmate.elf
$(OBJCOPY) -O binary $< $@
clean:
rm -f inmate.elf inmate.bin inmate.map
Build:
$ make
$ file inmate.elf inmate.bin
$ aarch64-linux-gnu-objdump -h inmate.elf
Check the map:
$ grep -E "_start|_stack_top|\\.text" inmate.map
Expected idea:
_start is at the inmate link address
stack top is inside the inmate RAM region
138.10 Load your inmate¶
Boot the QEMU ARM64 Jailhouse image from Chapter 137.
Copy your binary into the guest. Use the easiest path available:
$ scp inmate.bin root@QEMU_IP:/root/
If networking is not available, use a shared folder, rebuild the image, or copy through the QEMU disk image. The transport is not the lesson.
Inside the QEMU guest, first prove the stock demo still works:
# modprobe jailhouse
# jailhouse enable /path/to/qemu-arm64.cell
# jailhouse cell create /path/to/qemu-arm64-inmate-demo.cell
# jailhouse cell load inmate-demo /path/to/gic-demo.bin
# jailhouse cell start inmate-demo
Watch the console or the output path used by the demo image.
Expected result:
the stock inmate produces timer or interrupt output
Linux root cell stays alive
Only after that proof should you shut down the cell and load your own binary:
# jailhouse cell shutdown inmate-demo
# jailhouse cell load inmate-demo /root/inmate.bin
# jailhouse cell start inmate-demo
Expected result if the PL011 ownership assumption is correct:
hello from bare-metal Jailhouse inmate
If you do not see output, first check whether the inmate is running:
# jailhouse cell list
A silent inmate is not automatically a dead inmate.
138.11 If the UART path is wrong¶
Many first attempts fail because the console assumption is wrong.
There are three common output paths:
real or emulated UART MMIO
Jailhouse virtual debug console
shared memory buffer
If your config does not assign the PL011 UART to the inmate, do not keep poking 0x09000000.
Instead, use the demo binary and config as the reference:
# jailhouse cell load inmate-demo /path/to/gic-demo.bin
# jailhouse cell start inmate-demo
If the stock demo prints but your binary does not, compare:
entry address
link address
console path
binary format
CPU mode expectation
If neither prints, debug the cell config and QEMU console setup.
138.14 Reload loop¶
During development you need a repeatable loop:
# jailhouse cell shutdown inmate-demo
# jailhouse cell destroy inmate-demo
# jailhouse cell create /path/to/qemu-arm64-inmate-demo.cell
# jailhouse cell load inmate-demo /root/inmate.bin
# jailhouse cell start inmate-demo
If shutdown is not implemented, destroy may be the only practical development path.
Record what your inmate supports:
clean shutdown: yes or no
reload without disabling Jailhouse: yes or no
requires full QEMU restart: yes or no
Production systems need a defined stop and recovery path. An illustrative startup sequence does not establish that interrupts, buffers and device state are safe to reuse after stopping the inmate.
138.15 Failure case: inconsistent addresses¶
The linker, cell memory map and loader must describe the same runtime location. When the binary assumes a different origin, its references or entry conditions can target unassigned memory and fail.
A raw binary does not carry a relocation solution merely because its bytes were accepted by the loader.
138.16 Failure case: unassigned MMIO¶
An inmate can only rely on devices whose assigned MMIO and supporting resources match its configuration. An incorrect UART base may cause an access fault, no output or a wait that never completes.
Silence is ambiguous: it does not distinguish a missing device, a protection violation and a wrong console path by itself.
138.17 Failure case: invalid runtime state¶
A C call needs a valid stack in writable assigned memory. If the stack pointer lies outside that region, the first spill or call can fail before any useful diagnostic output.
Small bare-metal code still has runtime dependencies. Removing libc does not remove stack, alignment or initialized-state requirements.
138.18 Move to Zephyr¶
Now Zephyr is reasonable.
Zephyr gives you:
scheduler
drivers
devicetree
logging
timers
IPC libraries
build system
But Zephyr still must obey the same Jailhouse contract:
link address matches cell RAM
drivers match assigned devices
interrupts match assigned IRQs
shared memory matches both sides
console path is real
The cell config does not become optional because Zephyr is present.
138.19 Install Zephyr tools¶
Use a Python virtual environment:
$ sudo apt install python3-venv python3-pip
$ mkdir -p ~/imx6ull/zephyr-lab
$ cd ~/imx6ull/zephyr-lab
$ python3 -m venv .venv
$ . .venv/bin/activate
$ pip install west
Fetch Zephyr:
$ west init zephyrproject
$ cd zephyrproject
$ west update
$ west zephyr-export
$ pip install -r zephyr/scripts/requirements.txt
Record versions:
$ west --version | tee ~/imx6ull/jailhouse-lab/logs/west-version.txt
$ git -C zephyr rev-parse HEAD | tee ~/imx6ull/jailhouse-lab/logs/zephyr-commit.txt
138.20 Build a normal Zephyr QEMU sample first¶
Before Jailhouse, prove Zephyr builds:
$ cd ~/imx6ull/zephyr-lab/zephyrproject/zephyr
$ west boards | grep -i qemu
$ west build -b qemu_cortex_a53/qemu_cortex_a53 samples/hello_world
If your Zephyr release uses the older short board name, use:
$ west build -b qemu_cortex_a53 samples/hello_world
Use the exact target printed by west boards. The important part is that this is the ARM64 QEMU Cortex-A53 board, not a Cortex-M board.
Run the normal Zephyr QEMU target if supported by your setup:
$ west build -t run
This test is not Jailhouse. It proves the Zephyr toolchain and board support before you mix in cells.
138.21 What must change for Zephyr as an inmate¶
A normal Zephyr QEMU board assumes it owns the machine.
A Zephyr inmate owns only a partition.
You must align:
Zephyr link address
Zephyr RAM size
Zephyr device tree
Zephyr console
Zephyr interrupt controller view
Jailhouse inmate cell config
Start from the smallest sample:
samples/hello_world
Avoid networking, storage, and complex drivers until hello-world prints from the inmate cell.
138.22 Zephyr resource-table path¶
For Linux plus a remote processor, Zephyr has OpenAMP resource-table samples. That is especially relevant on STM32MP1 in Chapter 139.
For Jailhouse, the resource table is not the root mechanism. Jailhouse uses cell configs.
So separate the two paths:
Jailhouse inmate on A-core:
Jailhouse cell config is the hardware ownership contract.
STM32MP1 M4 firmware:
remoteproc resource table and RPMsg describe communication resources.
This distinction prevents a lot of confusion.
138.23 Key Notes¶
Cell configuration, link address, entry point and stack form one runtime contract. They must agree before a C function can execute.
A private console and shared-memory heartbeat are different evidence paths. Neither should assume ownership of hardware still used by root Linux.
Shared memory needs a protocol for ownership, ordering, validity and peer restart. A visible counter alone does not specify reliable communication.
Loading, stopping and reloading a cell must account for stale state and supporting resources, not merely replace instruction bytes.
Moving to Zephyr adds board definitions, drivers and scheduling assumptions. A normal QEMU Zephyr build does not establish compatibility as a Jailhouse inmate.
138.24 The Zephyr adaptation boundary¶
Zephyr’s configured RAM base and size, console device, CPU assumptions and shared-memory description must match the cell’s assigned resources. Its startup and interrupt model also have to work under that entry contract.
A resource table used by a remoteproc integration is not automatically a Jailhouse cell description. Those mechanisms have different loaders and ownership rules, so copying metadata between them does not establish an adaptation.
138.25 Product lesson¶
The useful product pattern is not:
we ran an RTOS under a hypervisor
The useful product pattern is:
Linux owns complex services
the inmate owns one bounded real-time responsibility
the two sides communicate through a small protocol
the hardware ownership table is reviewed like safety code
That is why the bare-metal inmate matters. It shows the boundary without hiding it inside an RTOS.
138.26 Troubleshooting¶
The inmate does not print
Check:
is the cell running?
is the console assigned?
does the UART base match the assigned MMIO?
does the inmate use the right output path?
The inmate crashes immediately
Check:
link address
load address
stack address
binary format
cell RAM flags
The root cell freezes
Check:
memory overlap
device ownership overlap
wrong interrupt assignment
bad cell config
Zephyr builds but does not run as inmate
Return to the bare-metal inmate.
Then compare:
entry point
linker map
devicetree memory
console device
interrupt controller setup
138.27 What comes next¶
Jailhouse taught us static partitioning.
Chapter 139 studies a different and very practical production pattern: Linux on Cortex-A7 plus RTOS firmware on a Cortex-M4 companion core. That is not a hypervisor lab, but for STM32MP1 it is often the better architecture.