Download PDF

Chapter 30A: Kernel lifecycle: mainline, stable, LTS, vendor BSPs

ABI: Application Binary Interface: the calling convention, register use, binary format, and library contract that let separately built code run together.

What: a decision framework for one of the most important architectural choices in any Linux-based product: which kernel release do we ship? Mainline tip, the latest stable, an LTS, a vendor BSP frozen years ago, or something curated between them? BSP: Board Support Package: vendor patches, configs, bootloader files, and scripts needed to boot one board.

Why: the choice determines your security-fix cadence, your driver-update cost, your hardware-support range, and your migration burden for the next five-to-ten years. If you choose well, updates ship easily for years. If you choose poorly, three years from now you are backporting six years of CVEs onto a fork no one upstream maintains.

Focus: maintenance economics, not features. Whatever you pick, you commit to maintaining the gap between it and what the world ships next. If you cannot maintain it yourself, you pay someone who will.

30A.1 The six release tracks

The kernel has more than one “version”. When someone says “use Linux 6.6”, they could mean any of six related things:

Track

What it is

Maintainer

Release cadence

Lifetime

Mainline

Linus’s tree at git.kernel.org/torvalds/linux.git

Linus Torvalds

One “x.y” release every ~9 weeks

“Tip” only; superseded by next release

Stable

Per-release backport tree

Greg Kroah-Hartman

Daily/weekly through ~6 weeks after release

~6 weeks per minor

LTS

Selected mainline releases extended

Greg KH + Sasha Levin

Weekly tagged releases

2 years (regular) or 6 years (extended)

Vendor BSP

Forked mainline + thousands of vendor patches

NXP / TI / ST / etc.

Vendor-internal cadence

Until the vendor abandons the SoC

Yocto / Buildroot kernel

A curated combination of an LTS + a small layer of fixes

Yocto / Buildroot maintainers

Aligned with Yocto release cadence

Tied to the Yocto release’s lifetime

Distribution kernel

Debian, Ubuntu, Fedora kernel packages

Distro maintainers

Distro release cadence

Tied to distro release lifetime

For embedded systems, the three that matter are Mainline, LTS, and Vendor BSP. The other three are either too volatile (mainline tip), too short-lived (stable), or too desktop-oriented (distro) for most embedded products.

30A.2 Mainline

git.kernel.org/torvalds/linux.git, the canonical tree. Every kernel feature, fix, and driver ultimately lives here.

Releases: the merge window opens 1 week after each release, accepts new features for 2 weeks, then there are 6-8 weekly -rcN candidates before the next x.y final. The cycle is roughly:

6.5 release  ─────────►  6.6-rc1 (window closes)
                 (1 week)   │
                            └─ 8 × rc cycles, ~1 week each ─►  6.6 release  ─►  6.7-rc1 ...

In 2025, mainline runs at roughly 6.6 → 6.7 → 6.8 → 6.9 across a calendar year, with each minor having ~9-week cadence.

Lifetime: none. The day 6.6 ships, 6.7-rc1 is already absorbing the next merge window. There is no concept of “support” for plain mainline. If you ship 6.6.0 and there’s a CVE in 6.6.1, you must either (a) take 6.6.1 (the stable update), (b) cherry-pick the fix yourself, or (c) decline the fix.

When to choose mainline:

  • Development. You’re building the next product. You want every recent driver fix and every new feature.

  • Pre-production hardware bring-up. Mainline is where the i.MX6ULL gained DT bindings, gained mainline EVK support, gained subsequent fixes. Tracking mainline means you can be confident about what works.

  • Demo / proof-of-concept devices. Lifetime doesn’t matter. Recent features do.

When not to choose mainline:

  • Shipping product. A device in a customer’s hands needs an upstream commitment longer than 9 weeks. Mainline gives you no such commitment.

30A.3 Stable

git.kernel.org/stable/linux.git, branches linux-6.6.y, linux-6.5.y, etc.

Greg KH manages this tree. Stable releases backport bug fixes and security patches only from mainline to a specific x.y. So linux-6.6.y accumulates fixes from mainline that apply cleanly (or with light tweaks) to 6.6’s codebase.

Releases come out daily-to-weekly with version tags like v6.6.1, v6.6.2, …, v6.6.42.

Lifetime: historically ~6 weeks per minor (one cycle of mainline). The current minor’s stable tree continues until the next minor’s stable tree starts. The older one is then EOL’d.

Exception: selected stable trees are promoted to LTS (next section) and continue for years.

When to choose stable (non-LTS):

  • You want the latest known-good kernel, with all minor-version fixes, and you’re going to upgrade to the next minor every 9 weeks anyway. Mostly: development boards, where you regularly rebase.

When not to choose stable:

  • Shipping product, again. You’ll lose support in 6 weeks.

30A.4 Long-Term Support (LTS)

Selected mainline releases are designated LTS and get stable-tree backports for years instead of weeks. The current list changes, so always check kernel.org before choosing for a product. As of June 2026, kernel.org lists:

LTS

Released

Projected EOL

Notes

6.18

Nov 2025

Dec 2028

Newest longterm line at this revision.

6.12

Nov 2024

Dec 2028

Strong default for new products if your BSP and drivers are ready.

6.6

Oct 2023

Dec 2027

Good teaching baseline and still common in embedded.

6.1

Dec 2022

Dec 2027

Heavily used by Android and several distros.

5.15

Oct 2021

Dec 2026

Still common, but aging.

5.10

Dec 2020

Dec 2026

Old baseline, mostly for legacy products.

LTS lifetime is the lifetime of the product, in practice. A product shipping in 2026 should normally start from the newest longterm line that supports the hardware and vendor stack. If you ship on an older LTS, write the migration date into the product plan before release.

The “extended LTS” track (sometimes called “Civil Infrastructure Platform” or CIP) backports security fixes for 6+ years on selected LTS releases, funded by industrial users (Toshiba, Siemens, others). Less drama-prone than mainline LTS but slower-moving.

When to choose LTS:

  • Any shipping product with a multi-year field life. This is the default answer.

  • Cost-sensitive products that can’t afford a dedicated kernel maintainer. LTS gives you free upstream backports.

  • Products that need security-fix updates after end-of-development.

When not to choose LTS:

  • You need a brand-new feature only in mainline that hasn’t been backported.

  • Your hardware vendor’s BSP is pinned to a non-LTS release and you can’t afford to forward-port their patches.

30A.5 Vendor BSPs

Every silicon vendor (NXP, TI, ST, Rockchip, Broadcom, …) maintains a downstream fork of mainline with their own additions. For i.MX6ULL, that’s NXP’s linux-imx at github.com/nxp-imx/linux-imx.

A vendor BSP typically contains:

  1. A pinned mainline base. NXP’s recent linux-imx branches are based on 5.15.71 or 6.6.23, not on mainline tip.

  2. Vendor patches. Thousands. Drivers for proprietary silicon, performance tweaks, customer-requested features, security fixes the vendor hasn’t yet upstreamed.

  3. Vendor BSP-only drivers. Drivers that exist only in the BSP, never in mainline. Frequently for GPU, video codec, ISP, or other IP blocks the vendor considers a trade secret.

The vendor releases new tags periodically (NXP about quarterly). Old tags get little attention.

The upside. Every peripheral on the vendor’s reference board has a working driver. The vendor’s tool, Yocto layer (meta-imx), demo image, evaluation kit, assumes their BSP.

Yocto: a metadata-driven build system for producing custom Linux distributions.

The downside. Three things you must factor in:

  • Security-fix latency. The vendor’s quarterly cadence means CVEs in your BSP base are typically 1-3 months behind mainline. Some critical fixes never get backported to the vendor’s older releases.

  • Mainline drift. Every quarter, the vendor’s tree drifts further from mainline. Any patch you write against the BSP needs porting before it works against mainline. The longer you wait, the harder that port becomes.

  • Vendor abandonment risk. When NXP stops supporting i.MX6ULL (they will, eventually), linux-imx will stop getting updates for that SoC. You’re then maintaining the BSP yourself or porting to mainline.

When to choose a vendor BSP:

  • You need a vendor driver that isn’t in mainline. Common for GPUs, hardware codecs, vendor-specific ISP/camera pipelines.

  • Time-to-market is paramount and the product lifecycle is < 3 years. Vendor BSPs ship fast. If you don’t need to maintain it for 10 years, the migration cost may never arrive.

  • You’re under contractual pressure to use the vendor’s reference design. This happens.

When not to choose a vendor BSP:

  • Multi-year product life and security matters. The BSP is a long-term liability.

  • Mainline already supports your hardware completely. For i.MX6ULL with stock peripherals, mainline is fully sufficient. Going vendor adds future cost without benefit.

30A.6 The decision framework

A flowchart-as-text:

Question 1: Does your hardware work on mainline?
   ├─ Yes  ─► Question 2
   └─ No   ─► Vendor BSP is forced.  Plan a migration to mainline within 1-2 years.
                  See Chapter 122A for the playbook.

Question 2: What's the product's expected field life?
   ├─ < 1 year         ─► Mainline tip is fine; rebase to next LTS before manufacturing.
   ├─ 1-3 years        ─► Current LTS.
   ├─ 3-7 years        ─► Current LTS, with explicit budget for one LTS migration mid-lifecycle.
   └─ 7+ years         ─► Extended LTS (CIP) or self-maintained LTS branch.

Question 3: Do you have a kernel maintainer on the team?
   ├─ Yes              ─► You can run mainline if Q1 says you can.
   └─ No               ─► Stick with LTS.  Don't try to maintain a fork.

For our i.MX6ULL on Point Atom MINI:

  • Q1: Does it work on mainline? Yes. Full support since v4.10.

  • Q2: Product life? Assume 5 years for an industrial product.

  • Q3: Maintainer? This book exists to teach you to be one.

→ Current LTS, with a mid-lifecycle migration plan. As of this revision, that usually means 6.12 or 6.18 for a new product, unless your vendor stack requires 6.6. If you choose 6.6, plan the next LTS migration before manufacturing.

30A.7 The 4.1.15 trap

Here is a common scenario. You inherit a vendor BSP pinned to Linux 4.1.15 (NXP’s i.MX BSP from 2017). The hardware works and the board boots, so you are tempted to ship it.

What you would actually be shipping:

  • A kernel from 2017. Five-plus years of CVE backlog. Even cherry-picking critical fixes is a part-time job.

  • A driver model from 2017. Many subsystems (DT bindings YAML, modern clk framework, regmap-everywhere, devm-_* helpers, etc.) have evolved. Code you write against 4.1.15 doesn’t transfer to mainline without rewrite.

MCU bridge: Think of regmap like a typed wrapper around your read_reg() and write_reg() helpers, with caching, locking, and bus differences handled centrally. regmap: a kernel helper that wraps register reads and writes over I2C, SPI, or MMIO.

  • Toolchain constraints. 4.1.15 doesn’t build with modern gcc (>= 11) without patches. You’re committing to an old gcc too.

  • No upstream support. Linux 4.1 has been EOL for years. Issues you file get closed as “fixed in 5.x”.

The 4.1.15 trap is real. It works today. It costs increasingly more every year. If you find yourself inheriting one, treat the migration to mainline as a P1 work item, not a P3 “someday”. Chapter 122A walks the migration.

The same applies to any sufficiently-old vendor BSP: 4.9, 4.14, 4.19, 5.4, 5.10. Each year past the LTS EOL the situation worsens.

30A.8 Worked example: shipping decisions

Three scenarios:

Scenario A, Consumer IoT smart-light, 3-year product life

  • Need: low-power Wi-Fi, BLE, a custom LED-strip driver, OTA updates.

  • All hardware supported in mainline (the Wi-Fi via standard mainline drivers, BLE via Bluetooth subsystem, LED strip via SPI).

  • Decision: current LTS. Pick the newest longterm line that supports the Wi-Fi and BLE stack. Set up an OTA pipeline (Ch 63) so kernel upgrades reach the field.

Scenario B, Industrial PLC, 10-year product life, IEC 61131 certified

  • Need: hard real-time (< 100 µs jitter), CAN bus, EtherCAT master, vendor-specific ISP for camera.

  • ISP driver only in NXP’s BSP. Never upstreamed.

  • Decision: Vendor BSP, for now, with mandatory migration plan. Track NXP’s LTS-aligned branch (6.6-based). Allocate budget in year 3 to begin upstreaming the ISP driver. Aim to be on mainline LTS by year 5. Otherwise: pay an external Linux company to extended-LTS your kernel for the remaining lifespan.

Scenario C, Hobbyist single-board computer kit

  • Need: just works out of the box. Users will run their own kernels too.

  • Decision: Mainline tip, refresh per Yocto release. Hobbyists expect to be current. Nobody’s running this kit for 5 years.

The pattern across all three: the product’s field life and the security level required together determine the kernel track.

30A.9 Key Notes

  • A release tag fixes source identity, while a maintained branch supplies an ongoing stream of changes. A teaching baseline and a supported shipping baseline serve different purposes.

  • Stable and longterm maintenance primarily carry fixes, not every new feature. Needed hardware support must be assessed in the selected tree rather than inferred from mainline development.

  • The product team owns qualification and delivery of updates even when upstream provides backports. A support window does not automatically deliver fixes to deployed devices.

  • A vendor BSP can meet a hardware dependency while adding downstream maintenance obligations. Custom drivers, bindings and toolchain constraints contribute to the cost of moving that dependency later.

  • Field life can outlast a kernel’s maintenance window. Migration or extended support is therefore a lifecycle resource decision, not only a version-number choice.

  • The worked shipping cases illustrate different constraints, not fixed recommendations for every product. Dated support tables and MINI hardware claims need current evidence before a release decision.

  • Build coverage across source tracks can reveal configuration or API drift, but cannot establish runtime compatibility, security-update safety or board support by itself.

30A.10 Pitfalls

  • “Just stay on 5.4 forever.” 5.4 is already outside the current kernel.org longterm list. That is not forever.

  • “We’ll upgrade the kernel later.” “Later” never comes if your code is so deeply intertwined with vendor BSP internals that migration is a major rewrite. Budget the migration now, even if you delay execution.

  • “LTS = no breaking changes.” LTS gets bug fixes and security patches. Behavior fixes can change semantics. Test before deploying.

  • “Mainline is unstable.” Modern mainline is very stable. What’s unstable is mainline tip between releases. Pick a tagged release (v6.6 not master) and you have what was tested.

  • “We don’t need security fixes. The device isn’t on the internet.” This is less and less true. Even isolated devices receive USB sticks. Even airgapped devices face supply-chain attacks. Apply the fixes.

  • “Our customer requires the vendor BSP.” Sometimes true (compliance, IP). Often a habit. Push back on the contractual requirement when the technical justification is weak.

30A.11 Going deeper

  • kernel.org/category/releases.html: current state of every kernel release.

  • lwn.net/Articles/...: weekly Linux Weekly News articles, the best source for keeping current.

  • The linux-cip (Civil Infrastructure Platform) project: extended-LTS for industrial applications.

  • Bootlin’s “Embedded Linux from Scratch” training (free online materials), covers a similar decision framework.

  • Documentation/process/stable-kernel-rules.rst: what does and doesn’t get backported to stable trees.

  • drm/kernel-doc-rst discussions on linux-rt-users mailing list for PREEMPT_RT-specific lifecycle considerations.

PREEMPT_RT: the Linux real-time patch set that makes more kernel paths preemptible and reduces latency.

Next chapter: Chapter 30B: Product kernel configs. Before leaving the kernel, we split development, production, and recovery configs so kernel configuration becomes a maintained product asset. Buildroot: a configuration-driven build system that produces a complete root filesystem and related images.