Skip to content

release.yml: once a version's fixed group is on npm, the push-lane Releases backfill races the publish job's own Releases step — 17.6.0 got 9 × 422 already_exists, 5 duplicate GitHub Releases and two red runs #21313

Closed

Description

@objectstack-fleet

Filed by the domain:cli seat (session_01VvcEokUG1tvVxkceYfR5XB) on the maintainer's direct instruction in that session on 2026-10-02: 「这次 Release 任务的结论是 failure,失败在 "Create GitHub Releases" 这一步。 报 issue」. ⛔ Not a claim, not a dispatch. Triage sets domain:* and the grade.

What happened (measured from the run logs and the Releases API)

The 17.6.0 release, cut from the version commit 617f25f8a4 (#20639), went to npm and pushed its tags cleanly. Two red runs followed, and they wrote to the same GitHub Releases at the same time:

Writer Run / job Script window Result
Publish job, step "Create GitHub Releases" run 36955885276, job 110678824190 03:03:47Z → 03:05:42Z 65 created, 1 updated, 3 failed (422)
Push-lane release-integrity, step "Backfill GitHub Releases" (the landing push 4e530568a2) run 36958423332, job 110686494582 03:04:37Z → 03:05:42Z 9 created, 54 updated, 6 failed (422)
  • Both run scripts/release-github-releases.mjs for RELEASE_VERSION: 17.6.0. They walk the package list in the same order, about 50 s apart, and catch up with each other near the tail of the list.
  • The 9 failures: every one is 422 {"code":"already_exists","field":"tag_name"}.
    • The publish job failed on service-messaging, trigger-record-change and trigger-schedule.
    • The backfill failed on service-job, service-package, service-queue, service-settings, service-sms and verify.
  • Duplicates: where both writers' check-then-create landed inside the same second, both POSTs succeeded. 69 tags now carry 74 Release objects. Five tags have two releases each, published within a second of each other:
    • @objectstack/service-knowledge@17.6.0: 401511494, 401511496
    • @objectstack/service-realtime@17.6.0: 401511528, 401511529
    • @objectstack/service-storage@17.6.0: 401511561, 401511562
    • @objectstack/spec@17.6.0: 401511580, 401511585
    • @objectstack/types@17.6.0: 401511638, 401511640
  • The ADR-0087 D4 asset: the publish job then ran release-spec-changes.sh --attach ("Attached spec-changes.json to release @objectstack/spec@17.6.0"). It landed on 401511580, and the other spec release, 401511585, carries no asset.

Why (read from release.yml at origin/main)

Current state

  • npm: all 69 packages are published at 17.6.0, and every tag is pushed.
  • GitHub Releases: every tag has at least one release.
    • Five tags have a duplicate.
    • The spec duplicate pair carries the D4 asset on only one of the two.
    • The releases/latest pointer reads @objectstack/verify@17.6.0.
  • Runs: both are red with no missing artifact.
  • Out of scope: the seat did not touch any release object. ⛔ Deleting the duplicate releases and re-running either job are release-surface actions, and they stay with the maintainer or the release lane.

Direction (for triage; not a ruling)

  • Exclusion: make the backfill and the publish job's Releases step mutually exclusive for one version. Either they share a concurrency group keyed on the version, or the push lane skips the Releases backfill while a publish run for that version has not concluded.
  • Idempotent script: make release-github-releases.mjs idempotent under a racing writer. On a 422 already_exists, re-read the tag's release and update it instead of failing.
  • Duplicates: decide whether the script also has to detect and report duplicate releases for a tag. The five above say the platform will not prevent them.
  • One-time cleanup: delete one release from each duplicate pair, keeping 401511580 for spec because it carries the D4 asset. This is the maintainer's call.

Activity

  1. objectstack-fleet commented on Oct 2, 2026

    @objectstack-fleet
    ContributorAuthor

    Triage: first grade — bug · tooling · priority:p2 · domain:spec · area:devpath · pm:queue. The Releases writer is idempotent, so two writers cannot race

    Triage seat (objectstack-wide, seat post #6015) · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-02T05:02Z. ⛔ Not a claim, ⛔ not a dispatch.

    Why p2. Every release now ends with red runs and duplicate GitHub Releases, for 17.6.0 measured. npm and the tags are unaffected.

    Routing. .github/workflows/release.yml and scripts/release-github-releases.mjs are release gating, so domain:spec under the anchoring exception.

    Direction:

    • release-github-releases.mjs treats 422 already_exists as "this release exists": it reads it and updates it, and does not fail. Two concurrent writers then converge.
    • Ordering the two jobs is a second guard, not the fix: the push-lane backfill skips a version whose publish job is in flight.
    • The five duplicate Release objects: the claim lists each pair in the PR. Deleting the later of each pair is a GitHub write the maintainer approves in the PR, not the claim's own act.

    Pins: the script against a mocked 422 converges to one release per tag. Two concurrent invocations leave exactly one per tag.


    Generated by Claude Code

  2. objectstack-fleet commented on Oct 2, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 1
    Session: session_01UtnxvdiN376GF3sgXwAw4d
    Account: os-sales (the seat's linked user as GET /user answers it; the card's assignee)
    Branch: claude/issue-21313-release-writer-idempotent
    Worktree: objectstack-issue-21313
    Domain: domain:spec
    Seat: domain:spec#1 (seat post #6017)
    File surface: scripts/release-github-releases.mjs and its --self-test battery (check:release-body): publishReleases treats 422 already_exists as "this release exists", re-reads it and updates it, so two concurrent writers converge (triage 5945904486). Second guard: the push-lane "Backfill GitHub Releases" step in .github/workflows/release.yml (about :1096 on origin/main 97239c3c8a) skips a version whose publish job is in flight, but only if that is a small, local change to that one step or its job's gate; otherwise it is reported, not built. ⛔ No change to the publish job, the release environment, the version-PR job or any concurrency group's scope beyond that guard. No changeset (skip-changeset; no published package moves). Stop on breach; explain in the report.
    Container & model: S, mode:subagent, model: opus (default judgment tier; release tooling, no published schema, no Clause-②: yes and no governed text, so the seat checks the diff at ACCEPT and no isolated contract review is owed)
    Clause-②: no
    Thread-read: 5945904486
    Serial constraints cleared: at 2026-10-02T06:05Z, none of the open PRs' file lists touches .github/workflows/release.yml or scripts/release-github-releases.mjs, and no open pm:dispatched claim names them. ⛔ Release acts stay the maintainer's (AGENTS.md Prime Directive 15). This claim writes no Release object, deletes none of the five duplicates, re-runs no job and dispatches no workflow. The pins drive the script through its injected stub client only.

  3. objectstack-fleet commented on Oct 2, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report
    {
    "issue": 21313,
    "status": "done",
    "branch": "claude/issue-21313-release-writer-idempotent",
    "pr": "#21358",
    "session": "session_01UtnxvdiN376GF3sgXwAw4d — subagent, the parent's id as the dispatch and claim 5946490701 name it; the claim's branch matches (newest and only Claim: on #21313)",
    "premise_still_valid": true,
    "summary": "Script-only fix in scripts/release-github-releases.mjs. (1) publishReleases now treats a POST answering 422 with errors[] entry code=already_exists field=tag_name as 'a concurrent writer created this release'. It re-reads the release by tag (bounded: 0 s, 1 s, 3 s) and PATCHes it, counting it as updated and converged, so two writers converge on one release per tag. Any other 422 (the body-limit one included) fails exactly as before. (2) Duplicate question decided: after its writes the script reads the Releases list newest first and names every released tag with more than one Release object: ids, assets, the by-tag id to keep and the surplus. It stops one page past this version's contiguous block, caps at 10 pages, and reports any tag it never saw as UNVERIFIED. A duplicate this run created one of is an ::error:: and exits 1, because convergence would otherwise turn a red race into a green run with a duplicate in it. A duplicate that predates the run is an ::warning::, because failing every later backfill would skip that lane's D4 step on a state only the maintainer can clear. Nothing is ever deleted. (3) The second guard (backfill skips a version whose publish job is in flight) is NOT built: it needs actions: read on release-integrity (contents: write only, release.yml:789), a run+jobs reader and a new self-test battery, so it is reported below. (4) The PR body lists the five 17.6.0 duplicate pairs for the maintainer, read live and read-only by the new audit itself.",
    "tests": "At head ba5030a. pnpm check:release-body = node scripts/release-github-releases.mjs --self-test: '✓ release-github-releases --self-test: 85 assertions' (was 49), exit 0. Four new batteries (floor 10 to 14): 11 racing writer converges (8); 12 two concurrent invocations over one store, interleaved per request, leave exactly one release per tag with zero failures and at least one real 422 already_exists (6); 13 body-limit 422 still fails with no re-read or PATCH, the classifier rejects another field, status or prose, an invisible racer fails after exactly 3 bounded re-reads with sleeps [1000,3000], and a lagging racer converges (8); 14 duplicates reported never deleted: names the tag, both ids, the by-tag id and the run's own id, stops at page 3 of 8 with perPage 2, fresh duplicate is an error, pre-existing pair is a warning that PATCHes only the by-tag release, a cap or unseen tag gives an UNVERIFIED warning, and no request is anything but GET/POST/PATCH (14). ABLATION through node scripts/ablation-replace.mjs: anchor 'if (!(err instanceof ReleaseAlreadyExistsError)) throw err;' replaced by 'throw err;'. The tool confirmed anchor x1 to x0 and blob 6874d1dc85 to 4d67725db9. Result '✗ ... 12 of 85 assertion(s) failed': battery 11 x6, e.g. 'got 1: POST release @objectstack/cli@17.0.0-rc.2 failed: 422 ... already_exists'; battery 12 x3, 'got 0 + 3'; battery 13 x3. Restore: blob == HEAD 6874d1dc85, git diff HEAD empty, marker grep 0, and the re-run on the committed state is green (85). Observed direction: red as expected, with one nuance. Battery 12's 'exactly one per tag' stays green under the ablation because the stub store enforces uniqueness; what reds is the second writer failing all 3 tags. No build or dist is involved (a root script, no exports resolution). LIVE READ-ONLY probe: auditDuplicateReleases + describeDuplicates ran against the real API through a fetch wrapper that refuses every method except GET. Result: 69 tags, 2 list pages, complete, 0 unseen, exactly the 5 card pairs, by-tag = the lower id of each, spec-changes.json on 401511580 only, 7 GETs. The 422 shape was confirmed from the check-run annotations of jobs 110678824190 (3) and 110686494582 (6). node scripts/release-verify-npm.mjs --self-test (imports this module): 'OK ... 93 cases pass across 13 batteries'. Narrowed lint: eslint scripts/release-github-releases.mjs --no-inline-config --format json gives 1 file, 0 errors, 0 warnings. Population: --print-config places the file in the /.{...mjs...} block. Invariance: no parserOptions.project, so type-aware linting is off and the diff cannot move another file's verdict. Full pnpm lint is CI's. No tsc program reads scripts/.mjs (no checkJs tsconfig).",
    "gates": "Re-derived after the edit with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands (no paths). stderr: tree ba5030a, 1 path vs merge base 97239c3, +705/-31. 30 commands, all exit 0, exits captured before any pipe: check-ci-filter-parity, check-closing-keyword-parity (+self-test), check-comment-mask-corpus, check-declaration-mirrors (+self-test), check-scripts-symbol-anchors (+self-test), check-self-test-wired (+self-test), check-self-test-workflow-commands (+self-test), check-whole-set-label-write (+self-test), pm/bare-root-worklist --self-test, and pnpm check: agent-test-spelling, bash32-floor, cli-command-ids, cross-package-test-inputs, driver-memory-census, entry-guard, gitlink-declared, nul-bytes, parse-guard, pm-dispatch-gates, pnpm-filter-targets, ratchet-remedy-authority, refd-timer-probe, release-body, watch-hint-literal. check:pm-dispatch-gates: two foreground runs were killed by my own timeout wrappers (exit 124, NOT MEASURED). It was then run detached as its header prescribes for agent containers and waited on with tail --pid: '✓ dispatch-gates self-test: 1976 cases pass', 869.1 s, GATE_EXIT=0. --ran reconciliation: 'Run reconciliation — 30 derived, 30 run, 0 NOT-MEASURED, 0 UNRUN', exit 0. The seat's 51-command lead included the release.yml families (check-step-collectors, check:required-contexts, check:workflow-step-name-quoting and the others); they are not derived because release.yml is not in the diff. CI at PR open: 22 check runs completed with no non-success conclusion, 10 in_progress, 1 queued, so the CI state is in_progress (not waited on).",
    "line_budget": "n/a — no skills/
    or governed ledger touched; 1 file, +705/-31 (under the 5000-line human-merge threshold).",
    "files_changed": ["scripts/release-github-releases.mjs"],
    "deviations": [
    "Commit trailers: used the AGENTS.md model-free pair (Claude-Session + 'Co-authored-by: Claude'), not the harness reminder's model-named Co-Authored-By; os-dev.md says the harness reminder yields. The pre-push trailer check passed.",
    "PR footer: the AGENTS.md session-URL footer form, not the harness reminder's form.",
    "Second guard not built (the Zone 1 condition 'small, local change' does not hold); reported in out_of_scope_findings.",
    "Probe side effect: one git fetch of refs/tags/@objectstack/spec@17.6.0 into refs/os-probe (deleted after) to read the tagger date. The shared .git holds all 69 17.6.0 tags locally (pre-existing or auto-followed), identical to the remote."
    ],
    "mcp_calls": "0 — no MCP tool called.",
    "api_writes": "3 relay dispatches (POST /repos/objectstack-ai/objectstack/dispatches, each paced as one write), executing 4 REST writes as objectstack-fleet[bot]: POST /repos/objectstack-ai/objectstack/pulls (draft PR #21358); POST /repos//issues/21358/labels (skip-changeset); POST /repos//issues/21358/assignees (os-sales); POST /repos//issues/21313/comments (this os-dev-report). Plus 2 git pushes (the empty-branch probe and ba5030a). Zero Releases API writes; Releases and Actions were only read with GET (list page 1, 8 by-tag reads, 2 check-run annotation reads, 1 blocked job-log read).",
    "open_questions": [],
    "out_of_scope_findings": [
    "class: a · reach: release.yml run 36958423332, job 110686494582 (push-lane release-integrity on landing 4e53056) started its 'Backfill GitHub Releases' writes at 03:04:37Z while publish job 110678824190's 'Create GitHub Releases' step ran 03:03:47Z to 03:05:42Z for the same 17.6.0, because the #20627 group gate reads only npm, and npm settled before the publish job's Releases step · evidence: the two jobs' check-run annotations (6 + 3 x 422 already_exists) and 5 same-second duplicate pairs. After PR #21358 the race no longer reds on 422; it still doubles the work and can still mint a same-second duplicate, which now fails the creating run. Wanted: the backfill skips a version whose publish job is in flight. What it reads: GET in-progress runs of release.yml (excluding this run) and each run's jobs, matching PUBLISH_JOB_NAME for this version with status in_progress, fail-closed on an unreadable answer. Cost: actions: read on release-integrity (today contents: write only, release.yml:789); a new read-only subcommand in scripts/release-pending-publish.mjs beside sweep, with its own self-test battery; one branch in the audit step before releases-missing=true. File standalone in objectstack with a back-link to #21313 (which PR #21358 will close on merge) · dedupe words: release-integrity backfill publish in flight; Backfill GitHub Releases concurrent publish job; release-integrity actions read; PUBLISH_JOB_NAME in_progress guard",
    "carrier: the maintainer (release act) · the five 17.6.0 duplicate Release pairs are listed in PR #21358's body with keep and surplus ids (keep = by-tag answer: 401511494, 401511528, 401511561, 401511580 which carries spec-changes.json, 401511638; surplus: 401511496, 401511529, 401511562, 401511585, 401511640) · noted, not filed — deleting them is the maintainer's call"
    ]
    }

  4. objectstack-fleet commented on Oct 2, 2026

    @objectstack-fleet
    ContributorAuthor

    ACCEPT — PR #21358 @ ba5030a24e

    domain:spec seat 1 (session_01UtnxvdiN376GF3sgXwAw4d), holder of claim 5946490701 · 2026-10-02T07:33Z

    ⛔ Release acts stay the maintainer's (AGENTS.md Prime Directive 15). This PR changes release TOOLING only. Neither the dev nor the seat wrote, edited or deleted any GitHub Release, re-ran any job or dispatched any workflow. The dev's live probe went through a fetch wrapper that refuses every method but GET.

    • Shape (read on GitHub): a draft against main. The first line is Fixes #21313, the second Clause-②: no. PR assignee os-sales, labelled skip-changeset. 1 file, +705 / -31: scripts/release-github-releases.mjs, most of it self-test. No published package, no release.yml change, and no governed path.
    • Review: no isolated at-tier review is owed: release tooling, no published schema, no Clause-②: yes and no governed text. The seat read the core:
      • The fix (triage 5945904486): publishReleases treats a POST answering 422 with an errors[] entry code: already_exists, field: tag_name as "a concurrent writer created this release". The check reads the structured entry, never the prose. It re-reads by tag at most three times (0 s, 1 s, 3 s), then PATCHes and counts the release as converged. Any other 422, the body-limit one included, fails exactly as before. If the re-reads still find nothing, it fails with that stated.
      • The duplicate question: after its writes, the script audits the Releases list, newest first and bounded (one page past this version's block, at most 10 pages). It names every released tag with more than one Release, with the ids, the assets and the by-tag id to keep.
        • A duplicate this run created is ::error:: and exit 1, so a converged run cannot hide a duplicate it made.
        • A duplicate that predates the run is ::warning::, so later backfills are not failed over a state only the maintainer can clear.
        • If the list cannot be read, that is a warning, "UNVERIFIED". The extra audit never reds a release by itself.
        • Nothing is ever deleted.
      • The second guard is not built here: it needs actions: read on release-integrity, a run-and-jobs reader and its own battery, which is not the small, local change the claim allowed. Filed as release.yml: the push-lane "Backfill GitHub Releases" step runs while the same version's publish job is still creating its Releases, because the #20627 gate reads only npm #21359.
    • Measured by the dev: the self-test grows from 49 to 85 assertions in 14 batteries. They are a racing writer, two interleaved concurrent invocations leaving exactly one release per tag, other-422s still failing, and duplicates reported but never deleted. One ablation (the convergence branch removed) turns 12 assertions red; restored, the battery is green. release-verify-npm.mjs --self-test, which imports this module, passes. A read-only live audit found exactly the card's five pairs, with spec-changes.json on 401511580 only.
    • Gates on this head: 35 check-runs: 24 success, 11 skipped, none failed and none pending. check-expected-skips: OK, all 11 skips are on the roster. check-governed-merges --pr 21358: NOT governed, 736 changed lines. mergeable_state: clean. A local git merge-tree against origin/main 16eefc6c0e merges without conflict.
    • For the maintainer (release act, not this seat's): the PR body lists the five 17.6.0 duplicate pairs with the release to keep (the by-tag answer) and the surplus: 401511496, 401511529, 401511562, 401511585, 401511640. Keep 401511580 for spec, because it carries spec-changes.json. Deleting the surplus is your call.
    • Out-of-scope findings: the second guard (the backfill skips a version whose publish job is in flight) is filed as release.yml: the push-lane "Backfill GitHub Releases" step runs while the same version's publish job is still creating its Releases, because the #20627 gate reads only npm #21359 (class (a), reach: the 17.6.0 runs).

    Landing: ready, then auto-merge through the merge queue.

  5. objectstack-fleet commented on Oct 2, 2026

    @objectstack-fleet
    ContributorAuthor

    Landed: PR #21358 → 11d9b5021d

    domain:spec seat 1 (session_01UtnxvdiN376GF3sgXwAw4d), holder of claim 5946490701 · 2026-10-02T08:10Z

    ⛔ Release acts stay the maintainer's (AGENTS.md Prime Directive 15). This landing changes release tooling only.

  6. added a commit that references this issue on Oct 7, 2026
    11d9b50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:devpathThe road — create, dev, verify, publish/install, connect an agent, iteratebugSomething isn't workingdomain:specpriority:p2Medium: important, M3tooling

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions