Publish what built instead of discarding the run over one package - #412
Closed
omarchybot wants to merge 1 commit into
Closed
omarchybot wants to merge 1 commit into
omarchybot wants to merge 1 commit into
Conversation
A single package that would not build stopped every channel for a day. flea 0.2.1 failed its test suite, bin/build exited 1, and bin/release treated that as fatal and returned before sign, promote or sync. Twelve packages had built successfully in the same run; all twelve were thrown away, rebuilt on the next tick, and thrown away again. Nothing reached edge, rc or stable from 09-11 until the package itself was fixed. bin/build already knows the difference between a package that failed and one that was blocked by a failed dependency, and it never builds the blocked ones. What it could not express was the difference between "some packages built" and "nothing built": both exited 1, so the caller had to assume the worst. It now exits 2 when at least one package built, keeps exit 1 for a run that produced nothing to publish, and writes the names it could not build to a state file, because the build and the release are separate processes and the list has to outlive the builder. bin/release accepts 2, runs the rest of the pipeline on what did build, and reports the missing packages at the end through the same notifier that already reports a failed step. It exits 2 so the outcome stays truthful to a human running it by hand: packages were published, and some are missing. bin/auto-release treats 2 as progress and clears the queue and the backoff. This is the part worth arguing about. Counting a partial release as a failure would have reinstated the original bug on a longer timescale: four consecutive partials back a channel off to six-hour intervals, and one permanently broken package would again decide when every other package ships. The cost is that a transient failure is no longer retried on the next tick; it waits for the next commit or version bump, which is also when it has a reason to succeed. Nothing publishes that could not before. Blocked packages are still excluded from the build output, so no package ships without the dependency it was built against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
|
Superseded: builds and publishing moved to CI in #511, which publishes per-package results by design; bin/release's all-or-nothing behaviour no longer gates anything. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure this prevents
flea0.2.1 failed its test suite and stopped every channel for a day.bin/buildexited 1,bin/releasetreated that as fatal and returned before sign/promote/sync — so the twelve packages that built successfully in the same run were discarded, rebuilt on the next tick, and discarded again. Nothing reached edge, rc or stable from 09-11 until the package itself was fixed.What changes
bin/buildexit 2when at least one package built but others failed/blocked.exit 1still means a run that produced nothing to publish. Writes the failed names to a state file.helpers/paths.shbuild_failures_file <mirror> <arch>— build and release are separate processes, so the list must outlive the builderbin/release2, runs the rest of the pipeline on what did build, reports the missing packages through the existing notifier, exits2bin/auto-release2as progress: clears the queue and the backoffbin/buildalready distinguishedfailedfromblockedand never builds a package whose dependency failed. What it couldn't express was "some built" vs "nothing built" — both exited 1, so the caller had to assume the worst.The debatable bit
A partial release clears the backoff instead of counting as a failure. Counting it would reinstate the original bug on a longer timescale: four consecutive partials back a channel off to six-hour intervals, and one permanently broken package again decides when every other package ships.
The cost: a transient failure is no longer retried on the next tick — it waits for the next commit or version bump, which is also when it has a reason to succeed. Happy to flip this if you'd rather keep retry-on-transient; it's a one-line change.
Safety
Nothing publishes that couldn't before. Blocked packages are still excluded from the build output, so no package ships without the dependency it was built against.
Testing
bash -nclean on all four. Diffs reviewed against copies before applying.Note: CI's
self-testsjob coverssync-upstream,sync-rebuilds,omarchy-pkgsandomarchy-release— none of these four files. A green check here does not mean this path was exercised. The partial path has not yet run against a real failure, since flea now builds.🤖 Generated with Claude Code