fix(ci): fix staging release - #917
Merged
Merged
Conversation
kmannislands
requested review from
gilescope
and removed request for
a team
September 11, 2026 15:29
|
| Branch | Total Count |
|---|---|
| main | 2815 |
| This PR | 2821 |
| Difference | +6 (0.21%) |
Keep up the great work migrating from Earthly to Earthbuild! 🚀
💡 Tips for finding more occurrences
Run locally to see detailed breakdown:
./.github/scripts/count-earthly.shNote that the goal is not to reach 0.
There is anticipated to be at least some occurrences of earthly in the source code due to backwards compatibility with config files and language constructs.
kmannislands
force-pushed
the
fix-staging-release-staging-test
branch
from
September 11, 2026 16:22
9d6f618 to
7d62948
Compare
janishorsts
approved these changes
Sep 11, 2026
janishorsts
left a comment
Collaborator
There was a problem hiding this comment.
LGTM. Unifying the TAG is neat.
Two independent failures in the staging release workflow: 1. The buildkitd image tag did not match the one built. build-earthly.yml computes TAG_SUFFIX from RUNS_ON (e.g. "ubuntu-26.04") and pushes +ci-release under it, but the tarball/artifact steps -- and every "Retrieve earthbuild" step in ci-staging-deploy.yml -- re-derived the suffix as "latest". "latest" is what the separate CI workflow pushes, so the staging release only resolved once CI had pushed that tag for the same SHA: it failed on the commit push and passed on a re-run. The suffix is now emitted as a workflow_call output and consumed by the callers, so the workflow only ever references images its own build-earthly job pushed. 2. "pull ping error" in the two release-dockerhub jobs. Those targets are multi-platform and --push already publishes them to ghcr; loading a copy back into the runner's docker daemon buys nothing and is what trips the pull ping (#912). They now run with --no-output. The smoke tests pull the pushed image, so they are unaffected. release-binaries deliberately keeps output enabled -- it produces local release artifacts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
--ci implies --no-output --strict, and --strict only gates LOCALLY and interactive debugging. The only LOCALLY in the repo is +fmt-go, which is not in the release graph, and ci.yml already runs `earth --ci --push +ci-release` against these same targets. Policy is now "--ci unless we can't": the copy-buildkitd job takes it too (+copy-img is RUN --push skopeo, nothing to output). release-binaries is the one exception -- +signed-release writes ./release/release/ via SAVE ARTIFACT AS LOCAL, which --no-output would suppress -- and now says so. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
--ci is the norm in this workflow and reading a tag suffix from the job that produced it is unremarkable, so neither needs explaining. Drop the repeated rationale and keep one comment on release-binaries, the single job that opts out of --ci, with a pointer to PR #858 (--no-image-output) as the eventual fix. Comments only; no behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
kmannislands
force-pushed
the
fix-staging-release-staging-test
branch
from
September 11, 2026 16:37
b4f47bf to
40ef656
Compare
kmannislands
enabled auto-merge (squash)
September 11, 2026 17:13
kmannislands
disabled auto-merge
September 11, 2026 19:37
gilescope
pushed a commit
that referenced
this pull request
Sep 21, 2026
fix(ci): fix staging release (#917) Line 29 of docs-internals/plan-merge-main.md. Nothing under engine/. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQ2c9xLfQvSY8YGLr7o5tv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The staging release workflow has two independent failures.
1. It referenced a buildkitd image it never built
build-earthly.ymlcomputesTAG_SUFFIXfromRUNS_ON(soubuntu-26.04for the staging caller) and pushes+ci-releaseunder that suffix — but its own "Save buildkitd image tarball" / "Build earth binary artifact" steps, and all fiveRetrieve earthbuild from build-earthly jobsteps inci-staging-deploy.yml, re-derived the suffix aslatest.-latestis what the CI workflow (ci.yml) pushes. Both workflows fire on the same push tomain, so the staging release was implicitly depending on a run it has no edge to — which is exactly the "wrong point in the graph / missing dependency" symptom: it fails immediately on the commit, and succeeds if you retry it after CI has finished. The staging workflow's ownbuild-earthlyjob meanwhile pushed-ubuntu-26.04images that nothing consumed.Fix:
build-earthly.ymlnow exposes the suffix it actually pushed as aworkflow_calloutput, and the callers use it instead of assuming one. The workflow is now self-contained — no cross-workflow ordering required.2.
pull ping errorin the release-dockerhub jobs./release+release-dockerhubbuilds multi-platform images and--pushes them to ghcr. Without--no-output, EarthBuild also loads them back into the runner's docker daemon through the embedded registry — the pull-ping path that keeps failing (see #912 and Janis's comment on #884).Nothing in those jobs needs the local copy: the smoke tests run
docker run ghcr.io/earthbuild/earthbuild:<tag>, which pulls the image--pushjust published. So bothrelease-dockerhubinvocations now pass--no-output.Chose
--no-outputover--cideliberately:--cialso implies--strict, which is a broader behaviour change than this needs.release-binariesintentionally keeps output enabled —+signed-releasewrites the release artifacts to./release/release/, so--no-outputwould break it.copy-buildkitd-to-dockerhubis untouched: it isRUN --pushskopeo only, with noSAVE IMAGEto output.Testing
Branch name ends in
-staging-test, so thestaging releaseworkflow runs on this PR branch.🤖 Generated with Claude Code