provider services take part in the image phase, with image distribution on demand by ndeloof · Pull Request #14252 · docker/compose

ndeloof · 2026-09-22T07:49:17Z

What this PR does, in one sentence

Provider-backed services take part in the image phase: compose hands the provider the identity of the service image and — on request — streams it the image bytes, so a build: or image: declared on a provider service actually reaches the provider's runtime.

Context

A provider service can declare build: today: compose builds the image… into the local daemon, where the provider never sees it. Providers whose runtime lives elsewhere (a sandbox VM with its own daemon, a cloud backend) have no way to receive a locally-built image, and no say in the image phase at all — docker compose pull skips them entirely. The only workarounds are out-of-band: push to a registry the provider can reach, or manual docker save | load plumbing.

What the PR brings

Providers opt in by declaring a pull block in their metadata output — the presence of the command block is the declaration of support, the same convention stop already uses (now documented as the general rule). For opted-in providers, compose invokes <provider> compose pull during the image phase of up (after any build) and on docker compose pull, passing:

the image identity (--image) and the state of the local daemon cache (--digest/--created, when present) — facts to persist as bookkeeping keys, not instructions: digest as identity test, created as the ordering fallback for backends that cannot preserve digests (a rebuilt local image must be detectable as newer even by a backend that transforms what it ingests);
two verdicts compose alone can compute, so providers never re-implement its arbitration: --source (local: the daemon's image is the desired state — build-only service, pull_policy: build, or just built by this run; registry: resolve upstream, which covers the build-as-CI-recipe workflow where consumers never build) and --policy (missing on up: a usable version suffices; always on pull: ensure freshness).

When the provider needs the local bytes, it sends a get-image request and compose answers on its stdin with one JSON line then the image tar as an HTTP/1.1 chunked body (RFC 9112): length-prefixed blocks need no in-band delimiter in binary data, the zero-length chunk marks a complete transfer (a truncated stream must be discarded), and any language's stock chunked reader consumes it. Export failures are announced in-band so the provider is never left waiting.

Guardrails: providers without the metadata block see zero behavior change; the wire format is locked by unit tests (byte-identical round-trip on delimiter-hostile payloads, error announce, end-to-end against a fake provider process); an e2e scenario drives the full path through the example provider (up builds and streams under the local verdict, compose pull re-invokes under the freshness contract). The PR also fixes a pre-existing gap the scenario exposed: a project made only of provider services failed the start phase with "no container to start" — softened for provider services only, the endpoint-relay still going through the regular start path.

Why this is the right next brick

This is deliberately the provider-initiated design: the image phase is where compose already makes images available to where services run, and the digest/created negotiation gives idempotence for free (no re-streaming on every up). A side-channel transport for very large images and the pull-fails→build fallback chain can layer on later without breaking the announced-answer protocol (encoding field).

🤖 Generated with Claude Code

docker-agent

Assessment: 🟡 NEEDS ATTENTION

Lower-confidence findings (not posted inline)

[medium→low] pkg/compose/plugins.go:395 — get-image request handled for any plugin command without command-context guard (confidence: weak 52/100)

codecov · 2026-09-22T09:15:37Z

Codecov Report

❌ Patch coverage is 80.66667% with 29 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/compose/plugins.go	76.59%	14 Missing and 8 partials ⚠️
pkg/compose/provider_images.go	89.74%	3 Missing and 1 partial ⚠️
pkg/compose/build.go	80.00%	1 Missing and 1 partial ⚠️
pkg/compose/run.go	0.00%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

docker-agent

This review covers only the commits pushed since d7fc6a1 (the previous completed review).

Assessment: 🟢 APPROVE

The incremental diff (92 lines across 3 files) makes two targeted fixes:

plugins.go — handlePluginMessage gains a command parameter so get-image requests are rejected with a protocol error when sent outside the pull command, preventing image-export stalls in up/down/stop. The command string is passed by value and is thread-safe.
pull.go — ensureProviderImages now runs unconditionally before the err != nil check (fixing the previously reported finding): a regular-service pull failure no longer silently skips provider image distribution. When both fail, errors.Join(err, providerErr) surfaces both. Under IgnoreFailures=true, providerErr is suppressed symmetrically with regular service pull errors — consistent with the semantics documented in the prior review thread resolution.
provider_images_test.go — New unit test TestExecutePlugin_GetImageRefusedOutsidePull validates the protocol restriction. The assert.ErrorContains substring ("get-image is only supported during the pull command") correctly matches the actual error message from plugins.go ("invalid message from plugin: get-image is only supported during the pull command").

No new bugs found in the + lines.

docker-agent

Assessment: 🟡 NEEDS ATTENTION

docker-agent

Assessment: 🟡 NEEDS ATTENTION

Provider image-phase integration is well-structured overall and the chunked-streaming protocol is sound. Two correctness issues were found in the new code.

docker-agent

Assessment: 🟡 NEEDS ATTENTION

…ution Providers opt into image distribution by declaring a pull block in their metadata — the presence of the command block is the declaration of support, generalizing the stop precedent. During its pull command a provider can request the service image from the local daemon with a get-image message (optionally platform-narrowed); compose answers on stdin with one image-stream JSON line then the tar as HTTP/1.1 chunked data (RFC 9112): length-prefixed blocks need no in-band delimiter in binary data, the zero-length chunk marks a COMPLETE transfer, and any stock chunked reader consumes it. There is deliberately no trailer nor final CRLF after the zero chunk — a stock reader stops there without consuming further bytes, so the next stdin answer starts clean for a provider requesting several images. An export failure is announced in-band through the error field so the provider is never left waiting; a failure after the announce closes the answer channel so the truncation is observable as EOF mid-chunk instead of a stream nobody will finish. The stream is exclusive on stdin for its whole duration — the framing invariant behind the lock. get-image is scoped to the pull command: emitted during up, down or stop it is a protocol error rather than served, so an image export can never stall another lifecycle command. Message dispatch moves out of executePlugin into handlePluginMessage, which also keeps the function under the complexity threshold. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

ensureImagesExists and docker compose pull now give provider-backed services their turn: compose invokes the provider's pull command with the image identity and the state of the local daemon cache (--image/--digest/--created), plus two verdicts compose alone can compute so providers never re-implement that arbitration: - --source: the authority for this invocation — local (the daemon's image is the desired state: build-only service, pull_policy build, or an image the current run just built) or registry (resolve the reference upstream — including the CI workflow where build is only the recipe used to publish the image consumers pull); - --policy: missing on the up path (a usable version suffices) or always on compose pull (ensure freshness of the authority). digest/created describe the cache, they are not instructions: the provider persists them as the bookkeeping keys of what it ingested — digest as identity test, created as the ordering fallback for backends that cannot preserve digests. Providers without the metadata block are skipped: behavior unchanged. Providers are independent, so their pulls run without fail-fast cancellation — the first failure no longer kills its siblings mid-run and mangles their reports into context-canceled noise; every provider runs to completion and every failure is reported, prefixed by service. On the pull path the provider phase runs unconditionally — skipping it because an unrelated service failed to pull would silently leave provider runtimes stale — and its error is a per-service pull error, following their exact regime: reported with them, suppressed by IgnoreFailures. A fatal errgroup error is reported joined with the accumulated per-service context instead of dropping it. The provider runs under the root context: the errgroup context is canceled once Wait returns and would kill the provider process on the spot. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

startService errored with 'no container to start' whenever the project-wide container listing came back empty, before the per-service filtering that lets provider services pass through. A project whose services are all provider-backed legitimately reaches the start phase with zero containers. The empty-list error is now softened for provider services only — a relay container deployed for published endpoints still goes through the regular start path. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

docker-agent

Assessment: 🟢 APPROVE

extension.md gains the Image distribution section: invocation contract (--image/--digest/--created/--source/--policy), the get-image request and its chunked image-stream answer (zero-chunk terminated, stdin exclusivity and drain-before-next-answer contract), the two-branch decision guidance (digest as identity, created as ordering fallback, reproducible builds caveat), and the support-by-presence metadata convention is now stated as the general rule. The example provider implements pull end to end — request, stock chunked reader, hash recorded at PROVIDER_PULL_MARKER — and the e2e scenario locks the whole path: up builds the service image and streams it to the provider under the local verdict, compose pull re-invokes it under the freshness contract. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

docker-agent

Assessment: 🟢 APPROVE

The provider image distribution implementation is solid. The chunked-transfer protocol, mutex discipline, and error propagation are all well-handled. The two drafter hypotheses about a trailing CRLF mismatch between ChunkedWriter.Close() and the documented protocol were investigated and found to be false positives: Go's httputil.ChunkedWriter.Close() writes only "0\r\n" (not "0\r\n\r\n") — confirmed from the Go 1.26.3 source — and the code comment and docs/extension.md are correct. ChunkedReader returns EOF when it reads the zero-length chunk without consuming any additional bytes, so the wire format is internally consistent and the test assertion ("no byte may follow the terminating zero chunk") is valid.