diff --git a/docs/Development/RFCs/0789-cosi-extension-images.md b/docs/Development/RFCs/0789-cosi-extension-images.md new file mode 100644 index 0000000000..d40c072455 --- /dev/null +++ b/docs/Development/RFCs/0789-cosi-extension-images.md @@ -0,0 +1,347 @@ +# 0789 COSI Extension Images + +- Date: 2026-09-08 +- RFC PR: [microsoft/trident#789](https://github.com/microsoft/trident/pull/789) + +## Summary + +Carry sysext and confext images as compressed members of a COSI tar. An optional +`extensions` section lists each [`ImageFile`](../../Reference/Composable-OS-Image.md#imagefile-object) +and, when placement is prescribed, its destination `path`. Trident can deploy +extensions with the OS image instead of fetching them separately. + +## Motivation and Goals + +Host Configuration extensions need separate hosting, downloads and updates. +Bundling gives OS-coupled extensions the COSI's integrity chain and the same +A/B rollout, reboot and rollback as the OS. Host Configuration extensions remain +available for content released independently. + +## Scope + +### Requirements + +- COSI revision 1.3 adds optional `extensions.sysexts` and + `extensions.confexts` arrays, each entry `{ id, image, path? }`. +- Clean Install and A/B Update deploy the effective set from COSI and Host + Configuration; collisions fail. +- An extension-only change can use Runtime Update without rewriting partitions + or rebooting. + +### Out of Scope + +Changing the Host Configuration API, DDIs, SELinux support or signing; +partition-less COSI files; and bundled extensions under +[disk streaming](../../Explanation/Disk-Streaming.md), which must be refused +until streaming can enable the merge units. + +### Exit Criteria + +Publish the 1.3 schema and samples. Deploy sysexts and confexts on Clean Install +and A/B Update with slot-correct rollback; apply a verified extension-only +change by Runtime Update. Reject collisions and unsupported disk streaming. + +## Dependencies + +[Image Customizer](https://github.com/microsoft/azure-linux-image-tools), or +another writer, must emit the section. Extension-only updates also need a +COSI-to-COSI writer that copies region images verbatim and an explicit +Runtime Update request (including its precedence over `forceAbUpdate`). + +Fix the existing `set_up_extensions` replacement bug first: with the same ID +and destination but different content, it installs the new file then deletes +that path as the old file. Runtime stage/finalize also needs durable payloads; +finalize and rollback construct their context with `image: None`. + +## Implementation + +### Tar Layout + +Payloads are ZSTD-compressed DDIs under `images/extensions/`; this preserves +`ImageFile.path`'s `^images/.+` constraint. The primary GPT image must +immediately follow `metadata.json`. Write extension members after the +disk-region images and include their window sizes in `compression.maxWindowLog`. +Older readers ignore the additional tar members. + +### Metadata Schema + +The two arrays mirror Host Configuration and determine the kind; each entry +contains an `id`, an `image` and optional `path`. This is the proposed addition to +the [1.2 schema](../../Reference/Composable-OS-Image/cosi-metadata-v1.2.schema.json): + +```json +{ + "properties": { + "extensions": { "$ref": "#/$defs/Extensions" } + }, + "$defs": { + "Extensions": { + "type": "object", + "properties": { + "sysexts": { "type": "array", "items": { "$ref": "#/$defs/ExtensionImage" } }, + "confexts": { "type": "array", "items": { "$ref": "#/$defs/ExtensionImage" } } + } + }, + "ExtensionImage": { + "type": "object", + "required": ["id", "image"], + "properties": { + "id": { "type": "string", "minLength": 1 }, + "image": { "$ref": "#/$defs/ImageFile" }, + "path": { "type": "string", "pattern": "^/.+\\.raw$" } + } + } + } +} +``` + +Example (`extensions` fragment of `metadata.json`): + +```json +{ + "sysexts": [{ + "id": "gpu-driver", + "image": { + "path": "images/extensions/gpu-driver.rawzst", + "compressedSize": 41943040, + "uncompressedSize": 134217728, + "sha384": "3a1f9c0d4e5b6a7c8d9e0f1a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d5e" + } + }], + "confexts": [{ + "id": "fleet-config", + "image": { + "path": "images/extensions/fleet-config.rawzst", + "compressedSize": 262144, + "uncompressedSize": 1048576, + "sha384": "c3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f60718293a4b5c6d7e8f90a1b2" + }, + "path": "/var/lib/confexts/fleet-config.raw" + }] +} +``` + +`id` is the extension's stable logical identity, not the COSI root `id`, the +host OS's `ID=`, or the image filename. For a sysext it must equal `SYSEXT_ID=` inside +`/usr/lib/extension-release.d/extension-release.{name}`; for a confext it +must equal `CONFEXT_ID=` inside +`/etc/extension-release.d/extension-release.{name}`. Trident already requires +those fields in DDIs. `(kind, id)` identifies an extension across COSI versions; +its compressed hash triggers replacement. IDs must be unique within each +array. Metadata can now match old and new extensions before reading their +payloads, but Trident must still verify the declared ID against the DDI. + +`ImageFile` supplies the tar path, compressed size for ranged reads, +uncompressed size for output bounds, and SHA-384 of the compressed member. +Trident checks both sizes and the hash against the bytes read and written. +It does not reuse Host Configuration's `Extension.sha384`, which hashes the +uncompressed DDI; a source switch must replace the image, not compare those +two hashes. Recompression may cause an unnecessary replacement, never a missed +one. + +`path` is deliberately optional, **not** a missing piece of metadata. Omission +leaves placement to Trident and may be the usual case for fully +Trident-managed extensions. Today Trident derives `{name}.raw` from +`extension-release.{name}` and uses `/var/lib/extensions/` or +`/var/lib/confexts/`. An explicit `path` must use a permitted directory and +match that name. [RFC 0838](https://github.com/microsoft/trident/pull/838) +proposes fully Trident-managed, slotted sysext storage without `path`; it +handles `/etc` layers separately. How bundled extensions select such a policy +is unresolved; omitting `path` alone must not opt in. + +There is no `url` (the tar is the source), `kind` (the array supplies it), or +separate name, enable flag or compatibility fields (the DDI supplies them, or +systemd merges all discovered images). The name determines the required +`{name}.raw` filename and can affect systemd's merge precedence; it is not +Trident's update identity. + +### Versioning + +COSI revision 1.3 adds the optional section; older COSIs behave unchanged. +Existing Trident accepts any `1.x` metadata and ignores unknown fields, so it +silently skips bundled extensions. That omission does not self-heal after a +Trident upgrade because the image is already recorded as deployed. Require a +1.3-aware reader, warn on unknown minor versions in new readers, and use a +[health check](../../Reference/Host-Configuration/API-Reference/Health.md) for +required extensions when reader versions cannot be controlled. + +### Trident-Side Consumption + +Compute an effective set from COSI and Host Configuration entries, retaining +each entry's source. Match old and new entries by `(kind, id)`; replace when +content changes, and reconcile changes to placement or filename. The extensions +subsystem reads bundled members instead of downloading them; +`update_host_configuration` writes resolved paths back only for Host +Configuration entries. `osconfig` enables the sysext/confext merge +units for the effective set. Move the SELinux check into validation that also +runs for Runtime Updates; its current `configure` hook does not. + +[Disk streaming](../../Explanation/Disk-Streaming.md) must reject bundled +extensions: `osconfig` skips stream-image configuration, leaving the merge +units disabled. + +#### Deployed Extension Inventory + +Host Configuration does not record bundled extensions. Persist an inventory +(kind, ID, name, resolved placement, source and source hash) in Host Status so +Trident can compare old/new sets and resume separate stage/finalize calls. +Retain deployed and pending inventories and image summaries alongside +`spec_old` and `spec`. Promote after successful Runtime Update, or after +post-boot commit for A/B; discard pending data on rollback. Extend +`ManualRollbackChainItem` to carry historical inventory and summary. + +Manual runtime rollback also needs the old DDI bytes. Either retain them while +the rollback chain references them or refuse rollback across a bundled +extension change; metadata alone cannot restore them. + +#### Destination Validation + +Existing A/B location validation reads only Host Configuration entries before +loading COSI. Validate the *resolved* effective set after reading the image: +every destination must be writable and, when A/B is configured, slot-scoped. +A/B volumes qualify; the root-verity `/etc` overlay qualifies during +provisioning if its backing volume is A/B. Slotted storage from +[RFC 0838](https://github.com/microsoft/trident/pull/838) qualifies only with +its loader capability; `path` omission alone must not bypass the A/B check. + +#### Where Images Are Written + +Clean Install and A/B Update place files in the target root (the inactive slot +for A/B). Runtime Update places them in the running root without provisioning +partitions. Stage temporary files on the destination filesystem for atomic +placement; the current fixed `/var/lib/extensions/.staging` falls back to a +non-atomic cross-filesystem copy. + +#### Rollback + +An A/B rollback restores the previous extension set only when placement is +slot-scoped. Writing a bundled image to a shared volume would change the running +slot and defeat rollback; see [Destination Validation](#destination-validation). + +#### Extension-Only Updates + +Today any change to the COSI metadata hash forces A/B servicing, including an +extension-only change. A partition-less COSI is not an option: `os.image` +describes a complete OS. Compare the new COSI against a stored summary of the +deployed OS instead. + +##### Deployed Image Summary + +Store each region's hash, uncompressed size and identity (partition number, +mount point, `fsType`, `fsUuid`, `partType`, verity root hash), plus disk geometry, +`osArch`, `osRelease` and `bootloader`. Exclude extensions, packages, compression +settings and `id`: none changes the deployed OS. A structured summary identifies +which region differs. If missing or of an unknown summary version, fall back to +the existing metadata-hash comparison. + +Equal summaries and equal extension sets require no servicing; equal summaries +with changed extensions permit Runtime Update; a changed summary requires A/B. +Other subsystems can still require A/B. A no-op needs a metadata-only Host Status +update, since `update` currently returns without recording the accepted image. + +##### Constructing an Extension-Only Update + +Copy an existing COSI's region images and metadata entries verbatim; replace +only the extension members and metadata (and `compression.maxWindowLog` or `id` +if needed). Rebuilding filesystems changes bytes and hashes, even for the same +files, and safely selects A/B instead of skipping an OS change. + +##### Requested and Verified, Never Inferred + +Runtime Update requires an explicit request, matching OS summaries, a +committed/active source slot, and no A/B requirement from any other subsystem. +If any condition fails, report an error; do not partially apply changes or +silently switch to A/B. + +##### Rollback on This Path + +Runtime Update changes the running slot. Retain the old DDI until finalize and +auto-rollback complete; both can run with `image: None`. Historical manual +rollback needs longer retention or must be refused. + +#### Interaction With Host Configuration `sysexts` and `confexts` + +Merge both sources. Reject duplicate extension IDs per kind or destination +collisions after resolving placement, including omitted `path`. Silent +precedence could run an unvalidated combination; identical entries still +collide. COSI ID duplicates can be rejected from metadata; Host Configuration +IDs still require reading their DDIs. Enforce the Host Configuration's +documented ID-uniqueness rule too. + +### Validation + +In `cosi/validation.rs`, add `V1_3` error kinds for invalid explicit paths +(absolute, `.raw`, allowed directory), duplicate `(kind, id)` pairs, duplicate +explicit destinations, reused tar members and collisions with disk-region +images. Reuse +`Extension::validate_sysext`/`validate_confext` path rules. A missing member is +detected when read: `Cosi::new` only scans as far as `metadata.json`. + +After decompressing and mounting the DDI, check actual compressed and +uncompressed sizes, the compressed SHA-384, exactly one `extension-release` +whose `SYSEXT_ID` or `CONFEXT_ID` equals the metadata `id`, the `{name}.raw` +destination suffix, resolved placement, and duplicate destinations/IDs across +both sources. Omitted paths cannot be collision-checked from metadata alone. + +Warn on definite OS `ID` mismatch with COSI `osRelease`, not `VERSION_ID`: +systemd may accept a version mismatch via `SYSEXT_LEVEL` or `CONFEXT_LEVEL`. +Do not warn on `ID=_any`; it declares compatibility, not delivery method. + +### SELinux + +Still unsupported in enforcing mode on systemd 255. Check the effective set in +validation for every servicing type, not only the Host Configuration lists in +`configure`, which skips Runtime Updates. + +## Testing and Metrics + +- v1.3 schema and valid/invalid samples, including missing or empty IDs; + semantic path and duplicate-ID rules in Rust tests. +- Clean Install and A/B Update with both kinds, merge-unit activation and + rollback (including absent `path` and slotted placement once supported). +- Runtime Update with identical region images, distinct stage/finalize and + auto-rollback; same-ID/same-path replacement and historical rollback refusal. +- Reject mixed-source collisions, invalid destinations, enforcing SELinux, + mismatched metadata/DDI IDs, changed OS summaries under requested Runtime + Update, and disk streaming. + +## Implementation Plan + +1. Fix [prerequisites](#dependencies); publish COSI 1.3 schema and samples. +2. Read, validate and install bundled extensions; persist the effective set. +3. Add the image summary, copy-based writer and verified Runtime Update path. + +Step 2 can ship before step 3, but the RFC is not complete without both. + +## Counter-Arguments + +### Drawbacks + +Bundling still ties distribution to a new COSI, increases its size and gives +hosts no per-extension opt-out. Use Host Configuration for extensions that must +move independently. Older readers silently omit bundled extensions. + +### Alternatives + +Embedding `.raw` files in a partition image hides them from COSI metadata, +fails when the destination is a separately created filesystem, and changes +verity hashes. A separate Host Configuration URL remains preferable for +independent release cycles. + +Flatcar's sysext-bakery pairs pre-built DDIs with +[`systemd-sysupdate`](https://www.freedesktop.org/software/systemd/man/latest/systemd-sysupdate.html), +hosting artefacts over HTTP with a `SHA256SUMS` manifest. Like `os.sysexts`, it +optimises for extensions that move independently of the OS, though without an +OCI transport. Bundling optimises for extensions that must move with it. + +## Open Questions + +- **Slotted placement:** how does a COSI entry request the policy in + [RFC 0838](https://github.com/microsoft/trident/pull/838)? Omitting `path` + cannot silently change an existing image's placement or imply loader support. +- **Runtime request:** where is the explicit request recorded across + stage/finalize, and how does it interact with `forceAbUpdate`? +- **Manual runtime rollback:** keep historical DDIs as long as the rollback + chain needs them, or refuse rollback across bundled extension changes? +- **Operator override:** retain the hard error on bundled/Host Configuration + collisions, or permit an explicit override when a hotfix is needed?