Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions tools/tailor/COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,6 @@ improving them:
- **Human-readable stdout/stderr.** Log lines, progress output, and diagnostic
wording are for people, not parsing. A stable machine-readable output mode is
future work; until it ships, do not screen-scrape tailor's output.
- **Lockfile format.** `tailor.lock` is generated and consumed by tailor; its
on-disk layout may change. Regenerate it with `tailor lock` / `tailor update`
rather than editing or parsing it.
- **Minimum Supported Rust Version (MSRV).** The Rust toolchain used to build
tailor may move at any time.
- **Deprecation policy.** There is not yet a formal warn-before-remove cadence.
Expand Down
41 changes: 13 additions & 28 deletions tools/tailor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,14 @@ The nuance lives in a handful of small, reviewable files instead of N copy-paste

You describe *what* you want; tailor handles the *how*. For each cell it assembles and runs the complete IC invocation through the Docker API (no shelling out):

- **Base resolution** — a local file, an `oci:` reference, or `azureLinux:` (MCR) shorthand, resolved to a digest and pinned in `tailor.lock`.
- **Base resolution** — a local file, an `oci:` reference, or `azureLinux:` (MCR) shorthand. Registry inputs resolve to digests for each run.
- **The privileged run** — `--privileged`, a read-only workspace mount plus narrow writable carve-outs, the correct `--platform linux/<arch>`, and **every host path rewritten** into `/host` (an easy-to-get-wrong step you'd otherwise script by hand).
- **Output & cleanup** — per-format `--output-image-format`, streamed logs, exit/output verification, and **sudo-free cleanup** of the root-owned files IC leaves behind.
- **Reproducibility** — `tailor build --locked` pins the IC container and base digests; `tailor build --dry-run` prints the exact `docker run …` first.
- **Iteration** — incremental build stamps avoid unchanged cells; `tailor build --dry-run` previews the container invocation offline. Registry tags can change between builds.

### Just an `image.yaml`

You don't need a workspace to start. This standalone definition — no `tailor.yaml`, no toolchain to pin — is a complete, buildable image:
You don't need a workspace to start. This standalone definition — no `tailor.yaml`, no configured toolchain — is a complete, buildable image:

```yaml
# image.yaml
Expand Down Expand Up @@ -82,27 +82,12 @@ Build it (add `--dry-run` to preview without launching the container):
tailor build
```

From that definition tailor resolves the base image, assembles the full privileged Image Customizer invocation — host paths rewritten into the `/host` mount, the platform set, the output format applied — and runs it:

```text
docker run \
--rm \
--privileged \
--platform linux/amd64 \
-v /home/you/hello:/host/home/you/hello:ro \
-v /home/you/hello/.tailor/cache:/host/home/you/hello/.tailor/cache:rw \
-v /home/you/hello/artifacts:/host/home/you/hello/artifacts:rw \
-v /dev:/dev \
mcr.microsoft.com/azurelinux/imagecustomizer:latest \
customize \
--config-file /host/home/you/hello/.tailor-render.hello_amd64_cosi.ic.yaml \
--build-dir /tmp \
--image oci:mcr.microsoft.com/azurelinux/3.0/image/minimal-os \
--output-image-format cosi \
--output-image-file /host/home/you/hello/artifacts/hello_amd64_cosi.cosi
```

The result is `artifacts/hello_amd64_cosi.cosi`. You wrote no `outputs:` (so it defaulted to `cosi`) and pinned no toolchain (so it used the latest Image Customizer). Grow into a workspace — a shared toolchain, defaults, and many images — only when you need it.
From that definition tailor resolves the base, chooses `linux/amd64`, rewrites host paths under
`/host`, and runs IC with `--output-image-format cosi`. The result is
`artifacts/hello_amd64_cosi.cosi`. You wrote no `outputs:` (so it defaulted to `cosi`) and
configured no toolchain (so it used the `latest` Image Customizer tag). Run `tailor build --dry-run` to see the
actual invocation for your environment. Grow into a workspace — a shared toolchain, defaults, and
many images — only when you need it.

## Installation

Expand Down Expand Up @@ -133,8 +118,7 @@ The `advanced` scaffold creates a workspace `tailor.yaml`, a `myimage/image.yaml
## Features

- Workspace and standalone image definitions.
- Multiple Image Customizer toolchains, with a `tailor.lock` lockfile (`lock` to freeze, `update` to
refresh) and `--locked` reproducible builds.
- Multiple Image Customizer toolchains with configurable versions, tags, and pull policies.
- Matrix expansion over user-defined axes, with `arch` as a typed, reserved axis.
- Per-axis fragments in `by-<axis>/<value>.yaml` and feature fragments in `by-feature/<name>.yaml`.
- Deterministic merge model: maps deep-merge, lists append, scalar conflicts require `$set`.
Expand All @@ -143,13 +127,14 @@ The `advanced` scaffold creates a workspace `tailor.yaml`, a `myimage/image.yaml
`inputs:`.
- Managed tools-dir (`--tools-dir`) for sealed images, with a safety guard that keeps builds off the
host root.
- Multiple output formats, `convert` for one-off format changes, and optional post-build `zstd`
compression.
- Multiple output formats, workspace-free `convert`, optional post-build `zstd` compression, and
extracted-artifact staging policies.
- Preview-gated Secure Boot signing (`previewFeatures: [signing]`).
- `export` rendered configs for a pipeline, third-party license notices via `tailor notice`, and a
stable exit-code taxonomy (`0`/`1`/`2`/`130`).
- Dry runs, validation, rendered config snapshots, selectors, exact cell slugs, and portable static
builds.
- Order-aware inter-image builds, per-output-directory build locks, and guarded scratch/mount paths.

Run `tailor --version` to see the version of your build; the current version line is `1.x` (see the
[compatibility policy](COMPATIBILITY.md)).
Expand Down
29 changes: 15 additions & 14 deletions tools/tailor/docs/explanation/base-images.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,38 @@
# Base images

A base image is the OS image Image Customizer modifies. tailor offers four ways to declare one, but
they split into two fundamentally different acquisition models — and the choice matters more for
reproducibility than for convenience.
A base image is the OS image Image Customizer modifies. tailor supports five sources. Registry
images resolve to digests for a run; file and workspace-image sources consume local artifacts.

## Registry bases vs. file bases

| Kind | Who fetches | Build input | Reproducible everywhere |
| Kind | Who supplies it | Build input | Portability |
| --- | --- | --- | --- |
| `oci`, `azureLinux` | Image Customizer, at build time | a registry digest (`--image`) | needs the `input-image-oci` preview + a cache dir |
| `path` | you | a local file (`--image-file`) | yes, but the path is repeated per image |
| `ref` (catalogue) | `tailor bases download` **or** a CI feed | a local file (`--image-file`) | yes, file-based, CI-parity |

The `oci`/`azureLinux` kinds are great for a quick dev build straight off a registry. But they pull
*at build time*, behind an IC preview feature, into a writable cache — exactly what a locked-down
pipeline cannot do, and not pinned to one verified artifact. A `path` base avoids that, but every
| `oci`, `azureLinux` | Image Customizer, at build time | resolved registry digest (`--image`) | requires IC's `input-image-oci` preview flag and a cache dir |
| `path` | you | local file (`--image-file`) | requires the same file at the declared path on each machine |
| `ref` (catalogue) | `tailor bases download` **or** a CI feed | local file (`--image-file`) | requires the same file in the named slot |
| `image` | another workspace image | paired producer artifact (`--image-file`) | requires the producer build; see [dependent images](../how-to/build-dependent-images.md) |

The `oci`/`azureLinux` kinds are useful for dev builds straight from a registry, but may require
registry access during build and can resolve a newer digest on a later run. A `path` base avoids
that, but every
image repeats the same `../../../artifacts/<name>.vhdx`, and the `../` count breaks when the layout
moves.

## Change detection

For local file bases (`path` and catalogue `ref` slots), tailor computes an **XXH3-128** content
hash plus the file size before planning. This hash is only for rebuild/change detection; it is not a
security or provenance digest, and local files are not pinned in `tailor.lock`.
security or provenance digest. Supply identical bytes separately on each builder; a matching
path alone is no provenance guarantee.

Large bases are cached on the build path. After the first hash, `tailor build` stores a versioned
entry under `<output>/.tailor/base-hashes/` keyed by the base path and guarded by the stored absolute
path, size, and mtime. If the size and mtime still match, the cached XXH3-128 value is reused and
the base file is not re-read (run with `-v` to see a `reusing cached hash, skipping read` line); if
anything is missing, malformed, unwritable, or changed, tailor just hashes the file again.

`tailor build --force` deliberately ignores all incremental checks, so it re-hashes the base every
time — omit `--force` for fast back-to-back iterations on an unchanged base.
`tailor build --force` rebuilds even when a stamp says the artifact is current; it does **not**
disable the base hash cache. An unchanged base may still reuse its cached hash.

## The catalogue model

Expand Down
2 changes: 1 addition & 1 deletion tools/tailor/docs/explanation/crate-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ graph TD
| Crate | Responsibility |
| --- | --- |
| `tailor-config` | Parse `tailor.yaml`/`image.yaml`, expand matrices, merge fragments, interpolate params, render cells. |
| `tailor-core` | Domain model, build plans, lockfile/stamp logic, orchestration, and port traits. |
| `tailor-core` | Domain model, build plans, incremental stamps, orchestration, and port traits. |
| `tailor-resolve` | Resolve toolchain and base image digests/hashes. |
| `tailor-exec` | Docker/Bollard execution adapter, IC arg construction, path translation, cleanup. |
| `tailor-sign` | Host-side signing backends for the `Signer` port (openssl/`sbsign`); preview-gated. |
Expand Down
2 changes: 1 addition & 1 deletion tools/tailor/docs/explanation/design-rationale.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Image Customizer already owns image semantics: storage, packages, services, scri
- loading workspace and image definitions,
- merging fragments,
- expanding matrices,
- resolving/pinning toolchains and base images,
- resolving toolchains and base images for each build,
- building the IC command line,
- running the IC container per cell.

Expand Down
17 changes: 10 additions & 7 deletions tools/tailor/docs/explanation/merge-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@ matrix:
Here `by-edition/...` applies before `by-channel/...`. This is intentional: adding a directory or changing alphabetical order cannot silently change precedence. Authors control precedence where they declare the axes.

A fragment can target a **combination** of axes (`by-edition+arch/pro+arm64.yaml`, a conjunction) or
**several values of one axis** (`by-channel/stable+edge.yaml`, a disjunction). Such fragments sort after the
single-axis ones they refine (more axes = more specific = later), and a narrower single-value fragment wins
over a broader disjunction on the same axis. `tailor explain <image> --cell <slug>` prints the exact order;
see [image.yaml](../reference/image-yaml.md) and [Merge directives](../reference/directives.md).
**several values of one axis** (`by-channel/stable+edge.yaml`, a disjunction). Its optional inline
`match:` further restricts the path match with nested `all`/`any`/`not` predicates; it does not
alter precedence. Composite fragments sort after single-axis ones, while a narrower single-value
fragment wins over a broader disjunction on the same axis. `tailor explain <image> --cell <slug>`
prints the applicable order; see [Fragments](../reference/image-yaml.md#fragments) and
[Field-specific merging](../reference/directives.md#field-specific-merging).

## Fragment sort order

Expand All @@ -36,7 +38,7 @@ For a given cell, the fragments that apply are sorted by these keys, in order, a
(`by-edition/pro.yaml`) applies before a multi-axis one (`by-edition+arch/pro+arm64.yaml`), so the
more specific composite wins.
2. **Axis declaration order** — among fragments of equal arity, the axis declared earlier in `matrix:`
applies first (later-declared axes win later conflicts).
applies first. Feature fragments follow matrix-axis fragments of equal arity.
3. **Breadth** — a broader disjunction (`by-channel/stable+edge.yaml`) applies before a narrower
single value (`by-channel/edge.yaml`) on the same axis, so the single value wins.

Expand Down Expand Up @@ -64,8 +66,9 @@ previous):
- Differing scalar assignments conflict unless the later assignment uses `$set`.
- Set a key's value to the bare token `$unset` to remove the inherited key entirely.

This makes accidental double ownership loud while keeping additive IC lists easy. See
`docs/reference/directives.md` for every directive.
These rules apply to merged `config`, `base`, and `outputs`; `params` conflict on different
values, `rpmSources`/`extraParams` concatenate, and `skip` is last-wins. See
[Field-specific merging](../reference/directives.md#field-specific-merging) for the exact surface.

## Why no IC-aware merge?

Expand Down
13 changes: 5 additions & 8 deletions tools/tailor/docs/explanation/threat-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,15 +57,12 @@ load-bearing — treat a regression in them as a serious bug:
paths a build produced (the build directory's children, the image cache, the
per-cell log), never arbitrary locations.

## Supply-chain pinning
## Registry input identity

Because a moving container tag is a reproducibility hazard (today's `:latest` is
not tomorrow's), tailor resolves and pins the digests of the images it runs —
the IC toolchain, tools-directory sources, base images, and the janitor image —
into `tailor.lock`. Pinning by digest is a **consistency and reproducibility**
measure under the trusted-host model, not a defense against a compromised
registry account; run `tailor lock` to freeze digests and `tailor update` to
refresh them deliberately.
Registry inputs resolve to digests for a build invocation, but a moving tag can
select different bytes next time. Digest resolution is not a defense against a
compromised registry account and does not make a build offline. Build only from
trusted registries and images within the trusted-host boundary above.

## Reporting

Expand Down
17 changes: 12 additions & 5 deletions tools/tailor/docs/how-to/README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# How-to guides

How-to guides are task-oriented recipes. Most assume a **workspace** (`tailor.yaml` plus one or
more `image.yaml` files); the execution verbs (`build`, `render` with a real run) additionally need a
container **engine**, and some steps need a **registry** login or a base-image **catalogue**. Each
guide notes anything extra it needs.
more `image.yaml` files); actual `build` and `convert` runs need a container **engine**, while
`render`, `validate`, and `build --dry-run` do not. Some steps need a **registry** login or a
base-image **catalogue**. Each guide notes anything extra it needs.

## Author a workspace

- [Add an image](add-an-image.md)
- [Add an axis](add-an-axis.md)
- [Share config with `$include`](share-config-with-include.md)
- [Override a base per axis](override-a-base-per-axis.md)
- [Build dependent images](build-dependent-images.md)
- [Track local build inputs](track-build-inputs.md)

## Choose a base image

Expand All @@ -26,19 +28,24 @@ guide notes anything extra it needs.
- [Change an output format](change-an-output-format.md)
- [Convert an image format](convert-an-image-format.md)
- [Set the build directory](set-the-build-directory.md)
- [Clean and rebuild a cell](clean-and-rebuild.md)
- [Manage extracted Image Customizer artifacts](manage-extracted-artifacts.md)
- [Skip experimental cells in bulk runs](skip-experimental-cells.md)

## Pin and reproduce
## Toolchains

- [Pin the Image Customizer version](pin-the-ic-version.md)
- [Select an Image Customizer tag](pin-the-ic-version.md)
- [Use a locally-built Image Customizer image](use-a-local-ic-image.md)

## Environment and engine

- [Select a container engine (Docker or Podman)](select-a-container-engine.md)
- [Capture Image Customizer logs](capture-build-logs.md)

## Pipelines and CI

- [Export configs for a pipeline](export-configs-for-a-pipeline.md)
- [Generate a CI matrix](generate-a-ci-matrix.md)
- [Handle exit codes in scripts](handle-exit-codes.md)

## Signing and preview features
Expand Down
63 changes: 63 additions & 0 deletions tools/tailor/docs/how-to/build-dependent-images.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Build dependent images

Use a workspace when one image consumes another's artifact. Define both images as members; tailor
builds producer cells before their consumers, including when you name only the consumer.

For a producer with a single `raw` output:

```yaml
# base-os/image.yaml
name: base-os
base: { path: ../bases/input.img }
matrix:
arch: [amd64, arm64]
outputs:
- format: raw
config:
os:
hostname: base-os
```

Select its output as another image's base:

```yaml
# appliance/image.yaml
name: appliance
base: { image: base-os, output: raw }
matrix:
arch: [amd64, arm64]
config:
os:
hostname: appliance
```

```bash
tailor validate appliance
tailor build appliance --dry-run
tailor build appliance --arch amd64
```

The shared `arch` pairs each appliance cell with its producer cell. If the producer has multiple
outputs, specify `output` by **format** (not file extension). For producer-only axes, pin a
coordinate with `cell: { flavor: minimal }` on the `base:` reference. Missing or ambiguous pairs
fail validation; dependency cycles fail before building.

To refer to an artifact *inside* the IC config instead of using it as the base, declare an input:

```yaml
inputs:
- name: payload
image: base-os
output: raw
config:
os:
additionalFiles:
- source: "${inputs.payload}"
destination: /opt/payload.raw
```

Tailor substitutes the paired artifact path, hashes its contents for incremental rebuilds, and
builds its producer first. Use `dependsOn: [base-os]` only when you need build ordering **without**
consuming its bytes; order-only edges do not change the consumer's fingerprint.
Comment on lines +60 to +61

See [Inter-image dependencies](../reference/image-yaml.md#inter-image-dependencies).
6 changes: 3 additions & 3 deletions tools/tailor/docs/how-to/build-with-an-azure-linux-base.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,12 @@ Pass `-vv` to also stream the Image Customizer logs.

## What tailor handles for you

- **Digest pinning.** tailor resolves the base to a digest and passes IC a reproducible
`--image oci:<repo>@sha256:…`. Run `tailor lock` to record it in `tailor.lock`.
- **Digest resolution.** tailor resolves the base to a digest and passes IC
`--image oci:<repo>@sha256:…` for that run. A tag may resolve to a different digest later.
- **Image cache.** `oci`/`azureLinux` bases need a cache directory; tailor defaults
`runtime.imageCacheDir` to `<workspace>/.tailor/cache`.
- **Ownership.** IC writes outputs as root; the sudo-free janitor normalizes them (and the cache) back
to you, so artifacts are yours to read and delete. tailor defaults `runtime.janitorImage` to
`mcr.microsoft.com/azurelinux/base/core:3.0`.

Set any of these explicitly under `runtime:` in `tailor.yaml` to override the defaults.
Set `imageCacheDir` or `janitorImage` under `runtime:` in `tailor.yaml` to override those defaults.
26 changes: 26 additions & 0 deletions tools/tailor/docs/how-to/capture-build-logs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Capture Image Customizer logs

Live IC logs stream during a build. To retain a full per-cell debug log:

```bash
tailor --log-dir ./build-logs build appliance
```

Logs are written as `<log-dir>/<slug>.log`; clones use `<slug>_clone<n>.log`. The opt-in order
is `--log-dir` → `TAILOR_LOG_DIR` → `runtime.logDir` (off by default). Relative flag and
environment paths resolve from the current directory; a manifest path resolves from the workspace
root:

```yaml
# tailor.yaml
runtime:
logDir: build-logs
logLevel: info
```

IC uses `debug` by default; `--ic-log-level warn` overrides `runtime.logLevel` for a run.
`-v`/`-q` affect tailor's own verbosity, **not** IC's log level.
`--timestamps elapsed|time|off` controls live status/log-line prefixes, not the IC log format.
To disable terminal colors set `NO_COLOR=1`; set `CLICOLOR_FORCE=1` to preserve colors in CI.

See [CLI reference](../reference/cli.md).
Loading
Loading