Skip to content

Drift bots write release notes, not bot narration - #1409

Merged
askalf merged 8 commits into
masterfrom
fix/drift-release-note-voice
Sep 24, 2026
Merged

askalf merged 8 commits into
masterfrom
fix/drift-release-note-voice

Conversation

@askalf

@askalf askalf commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

The drift bots' CHANGELOG bullets narrated the machinery: which workflow drafted the entry, what would auto-handle a re-capture, and that the PR auto-merged. They also used em dashes. The gating review blocks release text like that on this repo. It requested changes on dario#1405 (rule reads-as-generated), so every drift PR would stop at the same place.

Each of the three bots' bullets now says what changed for a user:

  • scripts/auto-draft-drift-fix.mjs: - **Claude Code 2.1.281 is in the supported range.** \SUPPORTED_CC_RANGE.maxTested` moves from `2.1.280` to `2.1.281`.`
  • scripts/label-sync.mjs: - **Template labels follow Claude Code 2.1.281.** \_version`, `_supportedMaxTested` and the `user-agent` header now read `2.1.281`. A live capture against Claude Code 2.1.281 matched the bundled template, so the request shape is unchanged and `_captured` keeps the date of the last real capture.`
  • scripts/rebake-release-prep.mjs: - **The bundled template follows Claude Code's current request shape.** A live capture no longer matched \src/cc-template-data.json`, so the template was re-captured from it. Requests that fall back to the bundled template send the new shape.`

No script, test or workflow parses these bullets. test/resolve-release-conflicts.mjs uses the old strings only as fixture content in a CHANGELOG it rewrites. The PR bodies the bots write are unchanged.

Tests: node test/auto-draft-drift-fix.mjs passes 66, node test/label-sync.mjs passes 17, and node test/resolve-release-conflicts.mjs passes 52. node scripts/rebake-release-prep.mjs, run against a copy of master, writes the new bullet under a dated 6.11.3 heading; the run was reverted.

dario#1405's own CHANGELOG lines were rewritten to the first two strings in 11c3725.

The CHANGELOG bullets from auto-draft-drift-fix.mjs and label-sync.mjs said which
workflow drafted them, what would auto-handle a re-capture and that the PR
auto-merged. They also carried em dashes. The gating review blocks release text
like that on this repo (dario#1405, rule reads-as-generated), so every drift PR
would stop there. Each bullet now says what changed for a user: the supported
range moved, or the template labels follow a new Claude Code version with the
request shape unchanged.
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Live test: ✅ PASS

Head 8bd4f811a0102b72002f91344ab9d3cc5656c77d on the self-hosted runner: All stages PASS: install/build/test(243/243)/cli/doctor/live-probe(pong)

Stages
install: PASS
build: PASS
test: PASS
cli: PASS
doctor: PASS (exit 1, 18 rows; no credentials, reported cleanly)
live: PASS (pong)

Full workflow run

askalf pushed a commit that referenced this pull request Sep 24, 2026
The gating review asked for the drift bullets without the drafting and
merging narration or em dashes. The wording matches what #1409 teaches
the drift bots to write.
The third drift bot, rebake-release-prep, still wrote the old narration:
an em dash, the watcher's workflow name and "wire-fingerprint drift".
Its bullet now says what the other two bots' bullets say: what a user
gets. test/resolve-release-conflicts.mjs uses the old string only as
fixture content in a CHANGELOG it rewrites.
@github-actions github-actions Bot added the tests Test suite changes label Sep 24, 2026
@askalf askalf added the verified Adversarially verified at this head label Sep 24, 2026
@askalf

askalf commented Sep 24, 2026

Copy link
Copy Markdown
Owner Author

Verification at f5fe6f4

Verified against base 15ba401 (master). The change holds at 825276c: all three appendUnreleased callers under scripts/ (auto-draft-drift-fix.mjs, label-sync.mjs, rebake-release-prep.mjs) now file a bullet with no em dash and no workflow, watcher or merge narration. The earlier blocked verification at 2647987 found the third caller missed; 825276c closed it. What was missing at 825276c was any test that executes a bullet: the three test files the body cites import the pure helpers, and the old strings survive in them only as CHANGELOG fixtures. So I pushed one, test/drift-release-notes.mjs (69 lines, f5fe6f4), which stages a temp repo (package.json, lockfile, CHANGELOG, src/live-fingerprint.ts, src/cc-template-data.json, drift-report.json), spawns each release-prep script, and reads back the bullet under the promoted 6.11.4 heading. test/all.test.mjs picks it up automatically.

Boundaries rebuilt from the diff

Row Producer Pinned by
auto-draft bullet: names both CC versions, no em dash, no narration scripts/auto-draft-drift-fix.mjs:211 drift-release-notes.mjs block 1
label-sync bullet: names the target version, no em dash, no narration scripts/label-sync.mjs:79 block 2
rebake bullet: no em dash, no narration scripts/rebake-release-prep.mjs:45 block 3
no fourth producer grep -rn "appendUnreleased(" scripts .github lists exactly these three; the workflow-side writers (drift-pr-heal.yml, cc-drift-auto-release.yml) only rename headings or extract a section measured, not tested
old strings consumed nowhere grep for Auto-drafted, Template label refresh, Template rebake, CC drift patch outside CHANGELOG.md and node_modules: no hits; test/resolve-release-conflicts.mjs carries them as fixture text only measured, not tested
PR bodies and titles the bots write unchanged by the diff (auto-draft-drift-fix.mjs:224-273 still carry em dashes and workflow names, but they are PR text, not release notes) out of scope, stated

Test evidence

Head f5fe6f4 (source identical to 825276c), node test/drift-release-notes.mjs:

=== auto-draft-drift-fix.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  OK no em dash
  OK no workflow or merge narration

=== label-sync.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  OK no em dash
  OK no workflow or merge narration

=== rebake-release-prep.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  OK no em dash
  OK no workflow or merge narration

15 pass, 0 fail

Base 15ba401 (same test file copied into a base worktree, scripts at base):

=== auto-draft-drift-fix.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  FAIL no em dash :: - **CC drift patch** — `SUPPORTED_CC_RANGE.maxTested` bumped `2.1.280` → `2.1.281` for CC v2.1.281. Auto-drafted by `cc-drift-watch.yml`. Template re-capture, if needed, is auto-handled by `cc-drift-template-watch.yml`.
  FAIL no workflow or merge narration :: Auto-drafted

=== label-sync.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  FAIL no em dash :: - **Template label refresh** — `_version`, `_supportedMaxTested`, and the `user-agent` header bumped to `2.1.281` to track `@anthropic-ai/claude-code@latest`. The live wire shape is unchanged — cc-drift-template-watch ran `capture-and-bake --check` against live CC v2.1.281 and found zero shape drift vs the bundle — so this is a label refresh, not a re-capture (`_captured` stays at the last real capture). Auto-merged; clears the `sdk-drift` early-warning signal.
  FAIL no workflow or merge narration :: cc-drift-template-watch

=== rebake-release-prep.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  FAIL no em dash :: - **Template rebake** — re-captured `src/cc-template-data.json` after cc-drift-template-watch detected wire-fingerprint drift against a live CC capture. Bundled fallback template now matches the current CC wire shape.
  FAIL no workflow or merge narration :: cc-drift-template-watch

9 pass, 6 fail

The 6 failing checks are the discriminators, two per script, so a regression in any one script fails on its own. The 9 checks that pass on base (exits 0, bullet filed, names the versions) are prerequisites inside each block rather than standalone controls: without them an empty bullet would satisfy the em-dash and narration checks trivially. The PR's earlier round at 2647987 is what they guard against in practice (a script that still writes the old bullet is caught by the discriminators; a script that crashes or files nothing is caught by these).

Also run at head

node test/auto-draft-drift-fix.mjs       66 pass, 0 fail
node test/label-sync.mjs                 17 pass, 0 fail
node test/resolve-release-conflicts.mjs  52 pass, 0 fail
node --test --test-name-pattern=drift-release-notes test/all.test.mjs   pass 1, fail 0
BASE_SHA=15ba401 node scripts/check-changelog.mjs   no src/ changes, nothing to record
node scripts/preflight.mjs               clean

CI at 825276c: every job green (build 18/20/22, test, analyze, actionlint, validate-package-json, docker-cap-drop-smoke, live-test, CodeQL). CI at f5fe6f4 is the same source plus one test file; it was still queued when this was posted.

Not run: nothing in the change needs the built dist/, a network, or the self-hosted runner.

Rules: test-comment-density-matches-neighbours=covered(test/label-sync.mjs and test/auto-draft-drift-fix.mjs at base carry file-level and helper comments; the new file's comments match that density and narrate no patch) | no-control-cases-in-the-suite=covered(the 9 base-passing checks are per-block prerequisites, no standalone control test; see the table above) | ledger-row-needs-its-fixture=covered(the body's "no script, test or workflow parses these bullets" row was re-measured by grep, and each producer got its own fixture block) | run-every-ci-step-not-just-the-red-one=covered(preflight, check-changelog and the touched test files run locally; CI matrix green at 825276c) | mutate-the-rejected-alternatives=unreachable(the body rejects no alternative implementation; the change is three string literals) | side-effect-change-needs-its-test=unreachable(no computed value changes source, only literal text) | moved-transform-test-enters-above=unreachable(nothing moved between layers; the test enters at the spawned script, the same layer the workflows use)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the GPT gating lane (gating review).

Request changes: the newly added public comments narrate this patch and its review history rather than documenting code behavior. rule:reads-as-generated

Blocking — generated-writing tell (scripts/auto-draft-drift-fix.mjs:209-210)

// Release-note voice: what changed for a user, no bot narration (the gating review
// blocks generated-sounding release text on this public repo; dario#1405).

This comment describes the rationale for this particular patch and names the prior review instead of explaining behavior needed to maintain the script. The same added comment appears in scripts/label-sync.mjs:77-78. This is public generated-style patch narration, which first-party strict mode blocks.

const driftBullet =

Blocking — generated-writing tell (test/drift-release-notes.mjs:1-4)

// The CHANGELOG bullets the drift bots write are the release notes users read
// on the GitHub release (extract-release-notes.mjs lifts the section verbatim).
// The gating review on dario#1405 blocked bullets that narrated the machinery
// instead of the change, so each bot's release-prep script is run end to end

The new test opens with a narration of the review and patch history. It is not test-local documentation and makes the test read as generated rather than following a concise test-file idiom.

import { spawnSync } from 'node:child_process';

The end-to-end test itself is useful: it executes all three release-note producers and checks their resulting bullets, and the current CI checks are green.

I reviewed the four changed files and the current verification comment and CI results; I did not run the local test suite.

@sprayberry-secondread sprayberry-secondread left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.

Reviewed by the Claude second-opinion lane (independent second read; the gating review is posted separately).

Verdict: no blocking issues at f5fe6f4. All three bullets are rewritten as described, nothing in the repo parses the old strings, and the new test fails on base for every script.

What I checked

The PR body against the diff

  • scripts/auto-draft-drift-fix.mjs:212-213, scripts/label-sync.mjs:79-83 and scripts/rebake-release-prep.mjs:46-48 produce exactly the three bullets the body quotes. I read them back from a real run of node test/drift-release-notes.mjs at head.
  • "No script, test or workflow parses these bullets": I grepped scripts/ src/ test/ .github/ for CC drift patch, Template label refresh, Template rebake and Auto-drafted by. The only hits are fixture text in test/resolve-release-conflicts.mjs, a fixture in test/auto-draft-drift-fix.mjs:207, and the PR-body header at scripts/auto-draft-drift-fix.mjs:248. The body says PR bodies are out of scope, and that header is in the PR body. appendUnreleased inserts the bullet verbatim and does not care what it says.
  • Are the new bullets true? The label-sync bullet says "A live capture ... matched the bundled template". That only runs on the exit_code == '3' step of cc-drift-template-watch.yml, which is the path where the shape matched. The rebake bullet says the capture "no longer matched", which is the exit-2 path. In the auto-draft bullet, ${ccVersion} and ${after} are the same value (patchMaxTested returns after: newVersion = ccVersion), so the header and the body of the bullet cannot name different versions.

Boundaries

  • Only the string templates changed. No predicates, guards or index expressions changed. target and ccVersion are already checked for empty or missing before these lines run (label-sync.mjs:44, auto-draft-drift-fix.mjs:84), so an empty version cannot reach the new text.
  • rebake-release-prep.mjs now uses '\'' inside a single-quoted literal. It renders as Claude Code's, which I confirmed in the test output.

The new test (test/drift-release-notes.mjs), per variant

  • I copied the head test next to the base (master) versions of the three scripts and ran it: 9 pass, 6 fail. no em dash and no workflow or merge narration fail on base for all three variants (auto-draft, label-sync, rebake), so each rewrite is pinned by a check that can fail.
  • At head: 15 pass, 0 fail. test/all.test.mjs finds every test/*.mjs through readdirSync, so CI runs the file without any wiring. The CI test job is green.

Non-blocking notes

  1. Low: the rebake variant's names the versions involved check is always true. test/drift-release-notes.mjs:61

      ['rebake-release-prep.mjs', [], []],

    and :66

      check('names the versions involved', expectVersions.every((v) => bullet.includes(v)), bullet);

    [].every(...) is true, so for this variant the check passes whatever the bullet says. It doesn't hide a gap: the rebake bullet names no version before or after the change, and the two checks that pin the rewrite fail on base. But it prints an OK line that asserts nothing. Suggested fix:

    if (expectVersions.length) check('names the versions involved', expectVersions.every((v) => bullet.includes(v)), bullet);
  2. Low: the new source comments narrate why the patch was written. scripts/auto-draft-drift-fix.mjs:209-210 and scripts/label-sync.mjs:77-78:

    // Release-note voice: what changed for a user, no bot narration (the gating review
    // blocks generated-sounding release text on this public repo; dario#1405).

    The test header at test/drift-release-notes.mjs:3-4 says the same thing. The first half states a durable rule and is useful. The second half cites the review and PR that prompted the change, which is history that belongs in the commit message. This PR is meant to remove that kind of narration from what the bots write, so the comments could lose it too. Suggested fix:

    // Release-note voice: what changed for a user, not how the bot produced it.

What's good

Each script is run end to end against a staged repo, and the test reads the bullet back from the CHANGELOG under the promoted version. That tests what users will actually see, instead of comparing string literals. The rewritten bullets are also more accurate than the old ones. For example, "Auto-merged" in the label-sync bullet was only true when auto-merge succeeded.

SECOND READ: READY

@askalf askalf removed the verified Adversarially verified at this head label Sep 24, 2026
@askalf askalf added the verified Adversarially verified at this head label Sep 24, 2026
@askalf

askalf commented Sep 24, 2026

Copy link
Copy Markdown
Owner Author

Verification at 68e0858

Base master@2ea19eac825a3585016c5f401dd48141cdbe820f. Production diff at this head is byte-identical to 9a8d7eb (git diff 9a8d7eb HEAD -- scripts is empty); the one commit I added, 68e0858, touches test/drift-release-notes.mjs only.

What this head answers

Redline's CHANGES_REQUESTED at f5fe6f4 named two added comments citing the gating review and dario#1405. Grepping every added line in base..HEAD under scripts/ and test/ for 1405|gating review|instead of|rather than|before the fix|(control)|previously|ensure|gracefully|robust returns 0, and a node scan for U+2014 / U+2192 in added lines returns 0. The script comments are gone; the test header is two lines saying what the test does.

Boundaries rebuilt from the diff

Row Producer of the string class Test
every appendUnreleased( caller (git grep gives exactly three: auto-draft-drift-fix.mjs:212, label-sync.mjs:84, rebake-release-prep.mjs:51) all three rewritten, all three executed end to end drift-release-notes.mjs, one block per script
no other writer of CHANGELOG bullets in workflows (no >> CHANGELOG, sed, tee in .github/workflows/*.yml) measured, nothing to test ruled out
the bullet is what users read: extract-release-notes.mjs 6.11.4 --file on each staged CHANGELOG lifts exactly the new bullet probe below probe, both arms
consumers of the old headline strings git grep outside CHANGELOG.md hits only fixture strings in test/auto-draft-drift-fix.mjs and test/resolve-release-conflicts.mjs, which the scripts under test never read ruled out; those files still pass 66 / 52
arrow between versions (X → Y), the other generated tell in the old auto-draft bullet and the one CLAUDE.md names as a bullet-arrow pattern added at 68e0858 no arrow between versions, three blocks
label-sync version check silently true on an empty expectVersions (Second Read note at f5fe6f4) if (expectVersions.length) at 9a8d7eb rebake block prints no version row

What I ran

node test/drift-release-notes.mjs on the head worktree, then the same file copied into a worktree detached at 2ea19ea and removed afterwards.

Head (68e0858):

=== auto-draft-drift-fix.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  OK no em dash
  OK no arrow between versions
  OK no workflow or merge narration

=== label-sync.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  OK no em dash
  OK no arrow between versions
  OK no workflow or merge narration

=== rebake-release-prep.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK no em dash
  OK no arrow between versions
  OK no workflow or merge narration

17 pass, 0 fail

Base (2ea19ea):

=== auto-draft-drift-fix.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  FAIL no em dash :: - **CC drift patch** — `SUPPORTED_CC_RANGE.maxTested` bumped `2.1.280` → `2.1.281` for CC v2.1.281. Auto-drafted by `cc-drift-watch.yml`. Template re-capture, if needed, is auto-handled by `cc-drift-template-watch.yml`.
  FAIL no arrow between versions :: - **CC drift patch** — `SUPPORTED_CC_RANGE.maxTested` bumped `2.1.280` → `2.1.281` for CC v2.1.281. Auto-drafted by `cc-drift-watch.yml`. Template re-capture, if needed, is auto-handled by `cc-drift-template-watch.yml`.
  FAIL no workflow or merge narration :: Auto-drafted

=== label-sync.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  OK names the versions involved
  FAIL no em dash :: - **Template label refresh** — `_version`, `_supportedMaxTested`, and the `user-agent` header bumped to `2.1.281` to track `@anthropic-ai/claude-code@latest`. The live wire shape is unchanged — cc-drift-template-watch ran `capture-and-bake --check` against live CC v2.1.281 and found zero shape drift vs the bundle — so this is a label refresh, not a re-capture (`_captured` stays at the last real capture). Auto-merged; clears the `sdk-drift` early-warning signal.
  OK no arrow between versions
  FAIL no workflow or merge narration :: cc-drift-template-watch

=== rebake-release-prep.mjs release note ===
  OK exits 0
  OK bullet filed under the promoted version
  FAIL no em dash :: - **Template rebake** — re-captured `src/cc-template-data.json` after cc-drift-template-watch detected wire-fingerprint drift against a live CC capture. Bundled fallback template now matches the current CC wire shape.
  OK no arrow between versions
  FAIL no workflow or merge narration :: cc-drift-template-watch

10 pass, 7 fail

Per-check arms. The 10 checks that pass on base are per-block prerequisites (the script exits 0, files a bullet, names its versions) that make the failing checks meaningful; they are not standalone controls and none is named as one. The no arrow between versions check passes on base for label-sync and rebake because their old bullets carried no arrow; it discriminates on auto-draft, which is the only producer that ever wrote one.

Mutants of the production files (each restored, git diff --stat -- scripts empty after)

Mutant Result at 9a8d7eb Result at 68e0858
A: auto-draft writes `${before}` → `${after}` instead of moves from ... to ... 14 pass, 0 fail, killed by nothing 16 pass, 1 fail: no arrow between versions
B: label-sync appends Auto-merged. 13/1: no workflow or merge narration :: Auto-merged same
C: rebake headline request shape** — a live capture 13/1: no em dash same

Mutant A is why 68e0858 exists: the arrow was in the old auto-draft bullet, it is the second generated tell alongside the em dash, and nothing on the branch pinned its removal.

Artefact check

/agent-output/fp/dario/vfy1409-probe-notes.mjs stages the same repo, runs each script, then runs scripts/extract-release-notes.mjs 6.11.4 --file <staged CHANGELOG> (the thing cc-drift-auto-release.yml uses for the GitHub release body). Head release bodies are exactly the three bullets in the PR description; base release bodies are the three old bullets with em dashes, the arrow and workflow names.

CI chain at this head

node scripts/preflight.mjs prints preflight: clean.; BASE_SHA=2ea19eac8 node scripts/check-changelog.mjs prints no src/ changes — nothing to record.; node --test --test-name-pattern=drift-release-notes test/all.test.mjs runs the auto-discovered file green (281 ms). At 9a8d7eb the full GitHub matrix was green (test job 107664150926 executed drift-release-notes); the run at 68e0858 was pending when this was posted. node test/auto-draft-drift-fix.mjs 66/0, node test/label-sync.mjs 17/0, node test/resolve-release-conflicts.mjs 52/0.

Not run

The three scripts inside their real workflows (cc-drift-watch.yml, cc-drift-template-watch.yml); they need the self-hosted runner and a live Claude Code capture. The bullet is produced before any of that runs, so the staged repo exercises the same code path.

Rules: comment-cites-its-own-review=covered(tells grep over base..HEAD added lines, 0 hits) | generated-release-note-voice=covered(no em dash, no arrow between versions, no workflow or merge narration) | reads-as-generated=covered(same greps; test header two lines) | no-control-cases-in-the-suite=covered(base arm per check, no check named or shipped as a control) | ledger-row-needs-its-fixture=covered(one staged repo per producer, three producers) | mutate-the-rejected-alternatives=covered(mutants A, B, C above) | run-the-artefact-the-fix-produces=covered(extract-release-notes probe, both arms) | run-every-ci-step-not-just-the-red-one=covered(preflight, check-changelog, all.test.mjs) | moved-transform-test-enters-above=unreachable(no cap or filter moved; strings replaced in place) | idempotence-test-asserts-only-agreement=unreachable(every check asserts the expected text, not agreement)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the GPT gating lane (gating review).

Verdict: changes requested: the added test-local helper has generated-style JSDoc; remove it to match the repository's test idiom. rule:reads-as-generated

Blocking — generated-writing tell

test/drift-release-notes.mjs:24

/** Stage the files a release-prep script reads, run it, return the bullet it filed under the new version. */

runBot is a test-local helper, so this JSDoc is public generated-style narration rather than necessary API documentation. This repository's first-party generated-writing gate applies to new test comments as well; leaving it makes the newly added test read as generated even though the helper itself is otherwise straightforward.

function runBot(script, args) {

I read the live-head verification evidence and the complete 75-line diff, including the staged end-to-end test. CI is green at this head. The production string changes cover all three appendUnreleased callers, and the test exercises each one; the remaining blocker is confined to the added test documentation.

What's good: the test asserts the user-visible changelog output rather than only inspecting the string literals, and its base/head evidence distinguishes the relevant release-note regressions.

@sprayberry-secondread sprayberry-secondread left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.

Reviewed by the Claude second-opinion lane (independent second read; the gating review is posted separately).

Verdict: No blocking issues at 68e0858. The three bullet strings are the only runtime change, the new test fails on master and passes here, and the PR body's claims match the diff.

What I checked

Boundaries rebuilt from the diff

The diff adds no predicates, guards or index expressions to the scripts. It only swaps three template literals. The interpolated values are the same ones the old strings used, and each comes from an existing guard:

Row Input Fixed code Pinned by
scripts/auto-draft-drift-fix.mjs:210-211, ccVersion/before/after missing ccVersion, or maxTested not older exits earlier, before the bullet (:84, :110), unchanged test/auto-draft-drift-fix.mjs (existing)
same, normal bump 2.1.280 to 2.1.281 bullet names both versions, no em dash, no → test/drift-release-notes.mjs "names the versions involved", "no em dash", "no arrow between versions"
scripts/label-sync.mjs:78-81, target empty target exits earlier (:43-44), unchanged existing
same, target = 2.1.281 bullet names target three times "names the versions involved"
scripts/rebake-release-prep.mjs:46-48 no interpolation constant string "no em dash", "no workflow or merge narration"
all three: the bullet lands under the dated promoted heading, not [Unreleased] appendUnreleased(..., /^## \[<after>\] - <today>/) unchanged "bullet filed under the promoted version"

Every new assertion checked against a variant where it should fail

  • I ran test/drift-release-notes.mjs against master's copies of the three scripts: 10 pass, 7 fail. That covers "no em dash" for all three scripts, "no arrow" for auto-draft, and "no workflow or merge narration" for all three (Auto-drafted, cc-drift-template-watch, cc-drift-template-watch). The fix is what makes these pass.
  • At head: 17 pass, 0 fail.
  • "bullet filed under the promoted version": I changed the rebake script in a temp copy so it never calls appendUnreleased. That assertion failed (16 pass, 1 fail), so it catches a bullet that goes missing or lands in the wrong place.
  • "names the versions involved" also passes on master. It guards against a rewrite that drops the versions, not against the voice problem. That is fine as a content check, and the per-variant narration and em dash checks do the fix-specific work.
  • test/all.test.mjs finds the new file automatically (it is not in EXCLUDED), and the CI test job passed at this head.

PR body claims checked against the diff

  • "No script, test or workflow parses these bullets": I grepped scripts/, test/ and .github/ for the old bullet prefixes. The only hits are fixture strings in test/auto-draft-drift-fix.mjs:207,216 and test/resolve-release-conflicts.mjs, plus the PR-body heading scripts/auto-draft-drift-fix.mjs:246 ('## Auto-drafted by cc-drift-watch.yml'), which is PR body text, not the CHANGELOG. Confirmed.
  • "The PR bodies the bots write are unchanged": buildPrBody and the workflow commit messages are untouched by the diff. Confirmed.
  • The new wording matches what each path does: label-sync runs only on exit 3, where capture-and-bake.mjs:476-477 found no shape drift, so "matched the bundled template" is accurate. The rebake bullet runs only after a re-capture, so "no longer matched ... re-captured" is accurate.
  • I did not check the 11c3725 claim about dario#1405, because it is outside this diff.

CI: all checks pass at 68e0858 (auto-merge skipped).

What's good: the test runs the real scripts end to end against a staged repo and reads the CHANGELOG back, so it pins the actual release-note output rather than a copied string.

SECOND READ: READY

askalf commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner Author

Verification at e781b47

Done from the maintainer's session. The OSS Breaker is on its last $11 of today's $150 while it verifies #1413, and this head needed only a re-check.

68e0858..e781b47 deletes one line: the JSDoc on runBot in test/drift-release-notes.mjs that Redline's gating review blocked. No other line changed. The three scripts and the rest of the test are byte-identical to 68e0858, the head both seats read.

check at e781b47 result
node test/drift-release-notes.mjs 17 pass, 0 fail
the same test against master's three scripts 10 pass, 7 fail: no em dash (all three), no arrow (auto-draft), no workflow or merge narration (all three)
test/auto-draft-drift-fix.mjs / label-sync.mjs / resolve-release-conflicts.mjs 66/0, 17/0, 52/0
node scripts/preflight.mjs clean
node --test test/all.test.mjs 242 of 243. The one failure is oauth-detector.mjs, which fails the same way on master in this container (no detectable CC binary); CI's runner passes it.

@sprayberry-secondread sprayberry-secondread left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.

Reviewed by the Claude second-opinion lane (independent second read; the gating review is posted separately).

Verdict: no blocking issues at e781b47. The three bullet strings are rewritten as described, and the new test fails against master's scripts in every variant.

What I checked

Diff vs body. The diff touches only the three bullet / driftBullet template strings plus one new test. buildPrBody and the workflow PR-body text are untouched, so "The PR bodies the bots write are unchanged" holds. I grepped scripts/, test/, .github/ and src/ for the old headlines (CC drift patch, Template label refresh, Template rebake). The only hits are fixture strings in test/resolve-release-conflicts.mjs:94-167 and test/auto-draft-drift-fix.mjs:207,216 (a promoteUnreleased fixture that isn't named in the body and is also just content). Nothing parses the bullets, so "No script, test or workflow parses these bullets" holds.

Bullet facts vs the code path that writes them.

  • scripts/auto-draft-drift-fix.mjs:210-211: after from patchMaxTested is newVersion === ccVersion, so "Claude Code X is in the supported range" and "moves from before to after" agree. ccVersion is non-null by the guard at :84. before is non-null once patched is truthy, and the script exits earlier otherwise (:141).
  • scripts/label-sync.mjs:78-81 runs only on the exit_code == '3' path (cc-drift-template-watch.yml:356,382), which is the "live capture matched, label only" case. The new sentence "A live capture against Claude Code X matched the bundled template" is accurate. target is non-empty by the guard at :44.
  • scripts/rebake-release-prep.mjs:46-48 runs only on the exit_code == '2' path (:171,252), which is the shape-drift case. "A live capture no longer matched src/cc-template-data.json" is accurate.

Boundaries. Each bullet is a template literal with no new predicates. The only inputs are ccVersion/before/after/target, and the existing guards cover their empty or null cases before the bullet is built (above). There is no new reachable row that needs a test.

Can the test fail? I ran test/drift-release-notes.mjs against a copy of scripts/ with the three scripts reverted to origin/master: 10 pass, 7 fail. The failures are em dash ×3, arrow ×1 (auto-draft), and narration ×3 (Auto-drafted, cc-drift-template-watch ×2). At head it is 17/17. Every variant has at least one assertion that fails without the change. test/all.test.mjs picks the file up by directory listing (:55-56), and CI test is green.

Findings

  • info, test/drift-release-notes.mjs: check('no arrow between versions', !bullet.includes('\u2192'), bullet); holds with or without the change for the label-sync.mjs and rebake-release-prep.mjs variants, because master's bullets for those two had no arrow. That's fine: it is a regression guard, and both variants still have discriminating assertions (em dash, narration). I'm naming it only because the gate asks for every per-variant non-discriminating assertion.
  • info, test/drift-release-notes.mjs: const root = mkdtempSync(join(tmpdir(), 'dario-drift-notes-')); is never removed, so each run leaves three staged repos in tmpdir(). A rmSync(root, { recursive: true, force: true }) after reading the CHANGELOG would clean up. This is not blocking.

What's good: the test runs the real scripts end to end against a staged repo instead of copying the strings. That makes it the check that would catch the next bot bullet regressing to narration.

Not done: I didn't run the full suite (CI is the signal, and all checks pass). I didn't run the rebake script against a real master copy; the body reports that run.

SECOND READ: READY

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.

Reviewed by the GPT gating lane (gating review).

Verdict: changes requested: the new test retains a policy-narrating comment rather than a code-oriented explanation. rule:reads-as-generated

Blocking — generated-writing tell

test/drift-release-notes.mjs:17

// Workflow names, merge mechanics and watcher verbs: none of it is a change a user can observe.

This comment explains why the release-note wording is considered acceptable, rather than what NARRATION matches or how the test uses it. It also restates the policy encoded by the immediately following regular expression. In this public test file, that patch/policy narration reads as generated and violates the repository's generated-writing gate. The NARRATION predicate is still clear without it.

const NARRATION = /\.yml\b|auto-drafted|auto-handled|auto-merged|cc-drift-template-watch|capture-and-bake|sdk-drift|early-warning|detected/i;

I reviewed the live head e781b47bff121183dd614c8b8c72e09963c472b6, the four-file diff, the current-head verification comment, commit messages, and green CI. The end-to-end regression test exercises all three changed release-note producers and its base/head evidence discriminates the rewritten text; no other blocking correctness issue was retained.

@askalf

askalf commented Sep 24, 2026

Copy link
Copy Markdown
Owner Author

Verification at 8bd4f81

e781b47..8bd4f81 deletes one line: the comment above NARRATION in test/drift-release-notes.mjs that Redline's gating review at e781b47 blocked. The three scripts and every other line of the test are byte-identical to e781b47, which was verified and which the Second Read found READY.

check at 8bd4f81 result
node test/drift-release-notes.mjs 17 pass, 0 fail
test/auto-draft-drift-fix.mjs / label-sync.mjs / resolve-release-conflicts.mjs 66/0, 17/0, 52/0
node scripts/preflight.mjs clean

The base-versus-head evidence at e781b47 still holds: no script changed, so the same test fails 7 of 17 against master's scripts.

@sprayberry-secondread sprayberry-secondread left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.

Reviewed by the Claude second-opinion lane (independent second read; the gating review is posted separately).

Verdict: No blocking issues at 8bd4f81. The three bullet strings change as the PR body says, the new test fails on the old strings, and nothing parses the old wording.

What I checked

The diff against the body. All three quoted bullets match the diff exactly:

  • scripts/auto-draft-drift-fix.mjs:210-211: `- **Claude Code ${ccVersion} is in the supported range.** ` + `\`SUPPORTED_CC_RANGE.maxTested\` moves from \`${before}\` to \`${after}\`.`. It uses the same ccVersion/before/after bindings as before (lines 82, 140), so the values going in are the same.
  • scripts/label-sync.mjs:78-81: the new bullet says "A live capture against Claude Code ${target} matched the bundled template". I checked that against the caller. cc-drift-template-watch.yml runs label-sync.mjs "$target" only under if: steps.check.outputs.exit_code == '3', and target comes from label-target.txt. capture-and-bake.mjs:476 writes that file from the live capture's _version. So the bullet's claim is true on the only path that reaches it.
  • scripts/rebake-release-prep.mjs:46-48: "A live capture no longer matched src/cc-template-data.json". This script is only called on the exit-2 rebake path (workflow line 252), so the claim holds there too.

"No script, test or workflow parses these bullets." I grepped scripts/, test/ and .github/ for the old lead-ins (CC drift patch, Template label refresh, Template rebake, Auto-drafted). Outside the three writers, the only hits are fixture strings in test/resolve-release-conflicts.mjs and test/auto-draft-drift-fix.mjs:207,216, which seed and read back a pre-existing Unreleased entry. check-changelog.mjs and extract-release-notes.mjs key on ## [x.y.z] headings, not bullet text. The claim holds.

Can the new test fail? I ran test/drift-release-notes.mjs at head: 17 pass, 0 fail. Then I ran it against a copy with the three scripts restored from master and kept the new test: 10 pass, 7 fail. Every negative assertion fails on the old strings in each variant where the old string had that tell:

  • auto-draft: no em dash, no arrow between versions, no workflow or merge narration (matched Auto-drafted)
  • label-sync: no em dash, no workflow or merge narration (matched cc-drift-template-watch)
  • rebake: no em dash, no workflow or merge narration (matched cc-drift-template-watch)

no arrow between versions holds on base for label-sync and rebake, because neither old string had an arrow. That makes it a guard for those two, not a regression pin. It is not vacuous for auto-draft, which is the one bullet that had the arrow. exits 0, bullet filed under the promoted version and names the versions involved are positive properties that hold both before and after. They check that the test harness actually ran the script and found the bullet, so a missing bullet cannot pass the negative checks as an empty string. That is the right shape.

Wiring. test/all.test.mjs picks up every test/*.mjs via readdirSync minus a fixed exclusion list, so npm test / CI runs the new file with no registration step.

Boundaries. The diff adds no predicates, guards, comparisons or index expressions. The only changes are string templates over values the scripts already validated upstream (ccVersion null-checked at line 84, target empty-checked at line 44 of label-sync). No boundary rows to rebuild.

Tells in the changed lines. The added lines have no em dash, arrow, or history narration. The one comment in the new test describes what the test does.

Not checked

  • I did not run test/auto-draft-drift-fix.mjs, test/label-sync.mjs or test/resolve-release-conflicts.mjs locally. When I read the checks, CI test was still pending at this head. build (18/20/22), actionlint and CodeQL were green.
  • The 11c3725 rewrite on dario#1405 is outside this diff.

SECOND READ: READY

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer.

Reviewed by the GPT gating lane (gating review).

Verdict: approved — no blocking issues found.

I reviewed live head 8bd4f811a0102b72002f91344ab9d3cc5656c77d, the four-file diff, the current-head verification evidence, commit messages, and current CI. The requested removal is present: the test now defines NARRATION directly, while the end-to-end test continues to exercise all three changed release-note producers. The current verification records 17 passing checks and the prior base/head run still discriminates the changed bullets. All required CI checks are green.

What's good: the production change remains limited to the three user-facing release-note strings, and the regression test reads each generated CHANGELOG bullet back from a staged repository rather than only inspecting string helpers.

Not run locally: the repository suite, per review environment policy; CI is green.

@askalf
askalf merged commit ede2da0 into master Sep 24, 2026
15 checks passed
@askalf
askalf deleted the fix/drift-release-note-voice branch September 24, 2026 19:46
askalf added a commit that referenced this pull request Sep 25, 2026
Redline blocked dario#1418 for the em dash in the heading this workflow writes into every label-refresh PR body. Plain punctuation, like #1409 did for the release notes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests Test suite changes verified Adversarially verified at this head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants