Skip to content

Opt in to upstream image source labels via frontend input - #1241

Merged
cpuguy83 merged 5 commits into
project-dalec:mainfrom
djsly:djsly-upstream-source-labels
Sep 17, 2026
Merged

cpuguy83 merged 5 commits into
project-dalec:mainfrom
djsly:djsly-upstream-source-labels

Conversation

@djsly

@djsly djsly commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Add opt-in upstream component source-label inference for container outputs. This supersedes the original automatic-default proposal: inference and inherited-provenance cleanup are disabled unless the caller supplies the raw frontend input dalec.image-source-label=true, not a build argument.

  • Infer config.Labels["org.opencontainers.image.source"] from exactly one top-level Git source after argument substitution, per target/platform. Support HTTP(S), including SourceHut ~user paths; retain .git; omit credentials, query strings and fragments; reject local/private-address and non-canonical numeric IPv4 forms. Never infer from website, archives, build definitions or base images.
  • With the input absent or false, neither infer nor remove inherited provenance. Existing explicit global/target label merging is unchanged. A same-named build argument cannot enable inference.
  • When enabled, remove inherited OCI source/revision and legacy vcs-url/vcs-ref before applying component metadata. This avoids legacy fallback to the base repository or pairing a base commit with the component source. Preserve unrelated base labels and explicit provenance; do not infer a revision.
  • Explicit global/target OCI or legacy source keys override inference, including empty values; target labels win. An empty OCI override suppresses inherited legacy fallback only when enabled. Explicit legacy labels remain intentional and must separately be emptied to suppress all discovery.
  • Share configuration across Linux normal/minimal and Windows container outputs. RPM container/depsonly outputs do not infer the application's source, because the application is not included. They preserve explicit image settings and apply the same enabled-only inherited-provenance cleanup. Package-only outputs remain unchanged.
  • HTTP(S) does not imply a public repository: opt in only when its URL is appropriate to disclose.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #1240

Special notes for your reviewer:

Latest review fixes: 921fcb3231c6eecf8874beb01265ba75122ccb45. Both new findings were reproduced before fixing them: SourceHut URL/config tests rejected the valid URL, and the corrected deps-only integration test reported the erroneous CoreDNS source.

Latest local gates, Linux Go 1.26.5:

  • go test -p 2 -count=1 --test.short --timeout=10m ./... passed.
  • Focused race tests for URL/source configuration and frontend-input parsing passed. Custom lint, go generate ./..., and byte comparisons of all generated files passed. CI-pinned golangci-lint 2.12.2 reported 0 issues.
  • 65 source-label integration cases passed: 13 each for Bookworm normal/minimal, Azlinux3 normal/deps-only, and Windows cross-container metadata. Covers absent/false, inert build argument, invalid input, single/ambiguous/no Git source, explicit global/target OCI and legacy labels, empty overrides and explicit revisions. Windows local validation is cross-build metadata, not Windows runtime execution.
  • The deps-only full-spec regression checks actual curl installation, absence of the application artifact, and absence of inferred application provenance with a CoreDNS Git source.
  • Both DEB/RPM package-only controls passed: absent/enabled configurations match except the existing per-solve creation timestamp.

Exact successful integration selectors (with go test -p 2 -count=1 -timeout=59m -v ./test -run=...):

^Test(Bookworm|Azlinux3)$/(^container$|^minimal_container$)/^image_configs$/^source_labels$
^Test(Bookworm|Azlinux3|Windows)$/(^container$|^minimal_container$|^depsonly$|^test_image_configs$|^package_source_labels$)/(^image_configs$/^source_labels$|^source_labels$|^full_spec$)
^Test(Bookworm|Azlinux3)$/^package_source_labels$

The second selector exercised Windows/deps-only matrices and the deps-only contents case; the first and third separately cover normal/minimal matrices and package-only controls. Counts were checked from the actual test output.

Rebuilt the frontend from the repository Dockerfile (sha256:5eb7415a92b3a20b1438f9577837590b5437a0d50d1cebb5403c12716d561704) and exported actual OCI images through it. Against a labeled Azure Linux base, all four azlinux3/container/depsonly exports contain curl and no CoreDNS executable:

Input / override Exported provenance Actual Renovate 44.90.2 sourceUrl
true All four inherited source/revision labels removed; no application inference absent
absent All four inherited provenance labels preserved https://example.com/base
false All four inherited provenance labels preserved https://example.com/base
true + explicit target source/revision Explicit target values preserved https://example.com/explicit-target

Unrelated base labels survived every case. Enabled deps-only manifest: sha256:aa3bc4a2acb179f72aecd5819e9baf5ce89f50d2cb39052c1acf96b6c8d84aa0.

Re-exported the representative CoreDNS v1.12.0 normal container with this frontend, using the previously validated compilation/package inputs. Unmodified Renovate again discovered https://github.com/coredns/coredns.git from its actual config blob. Manifest: sha256:4cad3043e5362e19649d92e1ca2c269b10ff72f1305ca5a1e466746aa8bd4fbc.

Prior redesign gates also validated normal-container absent/false/empty-label controls and ran the compiled CoreDNS binary read-only without network. A separate consumer follow-up used released Renovate 44.90.2 code, real OCI bytes and captured public GitHub responses: lookupUpdates selected 1.11.0-1 to 1.12.0-1, and the actual PR changelog/body functions rendered the CoreDNS v1.12.0 release notes. Raw .git and normalized source URL controls produced the same release-note section. This was offline HTTP replay, not a live authenticated Renovate bot PR. Missing-token and revision-only controls did not invent release notes.

All images/registries were local task resources; no test images were pushed publicly and no Renovate dependency was added to Dalec. Harnesses, exact export/config hashes, logs and consumer evidence are retained separately. Upstream release notes do not necessarily describe Dalec-specific CVE patches, toolchain changes or revision-only rebuilds.

The previous head's upstream checks passed. CI for this follow-up and maintainer approval remain separate gates.

🤖 Generated by GitHub Copilot

Infer component provenance from a single supported Git source, preserve explicit labels and opt-outs, and remove inherited base provenance. Isolate target label merges and cover container/package metadata behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Sylvain Boily <4981802+djsly@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 15, 2026 02:23
@netlify

netlify Bot commented Sep 15, 2026

Copy link
Copy Markdown

Deploy Preview for dalec ready!

Name Link
🔨 Latest commit 921fcb3
🔍 Latest deploy log https://app.netlify.com/projects/dalec/deploys/6aac26affed3380008bb9b9d
😎 Deploy Preview https://deploy-preview-1241--dalec.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Non-canonical numeric IPv4 hosts can bypass loopback filtering and produce unsafe source labels.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds automatic upstream source labels to container images while preserving overrides and removing inherited base-image provenance.

Changes:

  • Infers and normalizes source repository URLs.
  • Adds unit and Linux/Windows integration coverage.
  • Documents inference, overrides, opt-outs, and limitations.
File summaries
File Description
imgconfig.go Implements source-label inference and provenance cleanup.
imgconfig_test.go Tests normalization, precedence, and platform behavior.
test/linux_target_test.go Adds container and package-output regression tests.
website/content/image.md Documents source-label behavior.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread imgconfig.go
Reject numeric IPv4-like hostnames when strict IP parsing fails, preventing consumer-specific shortened, integer, octal, and hexadecimal forms from bypassing local-address exclusions. Cover URL normalization and inherited image provenance behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Sylvain Boily <4981802+djsly@users.noreply.github.com>
@djsly
djsly deployed to deploy-docs September 16, 2026 14:47 — with GitHub Actions Active
Select the standard Git or SSH port explicitly and use a tagged switch in platform tests. Add coverage for custom and cross-protocol ports to preserve conservative URL inference.

Signed-off-by: Sylvain Boily <4981802+djsly@users.noreply.github.com>

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@djsly

djsly commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

Investigated the three failed jobs in CI run 35002253803:

  • lint: fixed both PR-introduced staticcheck findings (QF1001 and QF1003) in 54e0492. The CI-pinned golangci-lint v2.12.2 now reports 0 issues locally on Linux. The full Linux short suite, frontend build, root-package race tests, custom lint and generation checks also pass. Added port regression cases to verify the simplification preserves URL-inference behavior.
  • Both E2E matrix jobs: fail building the unchanged Azure Linux runc fixture because the current Go toolchain rejects GOEXPERIMENT=nosystemcrypto with “has been removed; use MS_GO_NOSYSTEMCRYPTO=1.” The same failure occurs on main at 11ca9cd: E2E diff/merge=0, E2E diff/merge=1. test/fixtures/moby-runc.yml is byte-identical here and on current main (blob 0fe49c1b2d43f7d106680a93a753113f171dc7de). I have kept this unrelated fixture/toolchain repair out of the source-label PR; the shared baseline needs that fix before E2E can turn green.

On the previous PR head, all 16 integration matrix jobs, unit tests, CodeQL and dependency review passed. These are results for fca840d, not yet the new-head rerun.

🤖 Generated by GitHub Copilot

@djsly
djsly deployed to deploy-docs September 16, 2026 19:11 — with GitHub Actions Active
@cpuguy83

Copy link
Copy Markdown
Collaborator

I think this needs to be opt-in via a frontend input (not build-arg which is a special frontend input, though that does mean that docker-cli users can't enable it).

We don't want to suprise people by automatically doing this and they've got some closed source project they are building.

I also think we should only do this for http(s) URL's.

No need to drop .git from the URL unless its some problem with tooling that's picking this up.

Github will redirect the .git URL, at least.

Require the raw dalec.image-source-label frontend input before inferring or replacing inherited provenance. Preserve inherited labels when absent or false, support only HTTP(S) repository URLs, and retain the .git suffix. Wire normal/minimal Linux, RPM dependency and Windows container paths, document the behavior, and cover enabled/disabled and package-output controls.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Sylvain Boily <4981802+djsly@users.noreply.github.com>
@djsly djsly changed the title Populate upstream source labels for container images Opt in to upstream image source labels via frontend input Sep 16, 2026
@djsly

djsly commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

@cpuguy83 Addressed in 41328ac:

  • Inference now requires the raw frontend input dalec.image-source-label=true, not a build argument. Absent/false preserves inherited provenance and existing behavior.
  • Only HTTP(S) URLs are eligible; .git is retained. The docs and PR description now supersede the original automatic-default proposal.
  • Fresh real-image validation confirms the new behavior: compiled and ran CoreDNS v1.12.0, then unmodified Renovate 44.90.2 selected the latest stable fixture tag and fetched its OCI config, returning exactly https://github.com/coredns/coredns.git. Absent/false controls retain the seeded base provenance; enabled plus an explicit empty source returns no sourceUrl, including no inherited legacy fallback.

All 50 source-label scenarios across DEB normal/minimal, RPM normal/deps-only and Windows cross-container metadata passed, as did both package-only controls, the full Linux short suite, targeted race tests and CI-pinned lint (0 issues). Precise commands and caveats are in the updated PR body. The consumer check proves source discovery, not release-note or packaging-tag mapping.

Ready for another review; new-head CI is separate from these completed local gates.

🤖 Generated by GitHub Copilot

@djsly

djsly commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

@cpuguy83 The opt-in redesign is implemented and validated, and ready for review. Could you confirm this matches the design you had in mind?

Inference requires the raw frontend input dalec.image-source-label=true (not a build argument). Absent/false preserves inherited provenance; enabled inference accepts HTTP(S) only and retains .git.

The real-image checks and Renovate 44.90.2 source discovery passed, returning https://github.com/coredns/coredns.git. The numeric IPv4 review finding is also fixed and its thread resolved. Full validation details are in the updated PR description.

🤖 Generated by GitHub Copilot

@djsly
djsly deployed to deploy-docs September 17, 2026 06:55 — with GitHub Actions Active

@cpuguy83 cpuguy83 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two correctness issues to address: valid SourceHut HTTPS repositories are excluded, and deps-only images acquire the source label of an application they do not contain.

review by GPT-6 Astra

Comment thread imgconfig.go Outdated
Comment thread targets/linux/rpm/distro/container.go Outdated
Allow tilde-prefixed paths in HTTP(S) repository URLs while retaining local-path and numeric-host safeguards. Use only the synthetic dependencies package sources for RPM deps-only image configuration, preserving explicit image settings and opt-in inherited-provenance cleanup. Cover SourceHut URLs, dependency-only contents, explicit OCI and legacy labels, and document the distinction.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Sylvain Boily <4981802+djsly@users.noreply.github.com>
@djsly
djsly deployed to deploy-docs September 17, 2026 19:07 — with GitHub Actions Active
@cpuguy83
cpuguy83 merged commit 1f3da58 into project-dalec:main Sep 17, 2026
32 checks passed

This branch was successfully deployed

1 active deployment
deploy-docs 921fcb32 Deployed Sep 17, 2026 by djsly via Generate docs website to GitHub Pages #1288
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[REQ] Automatically populate upstream OCI source labels for Renovate changelog discovery

3 participants