Repository navigation
fix(release): raise the Communique response token limit so release notes are not cut off - #295
Conversation
…tes are not cut off Communique 1.1.3 sends max_tokens=4096 when communique.toml does not set it. One submit_release_notes call (changelog, title and body) needs more than that once a release has about 15 commits, so every call was cut off and Release Please failed with communique::malformed_submission whenever the last cut-off call had no changelog. Set max_tokens = 16384 under [defaults] (the default Communique 1.2.2 moved to), pin it with a test, and add a hint to the runner error for this failure.
|
@codex review |
|
@codex security review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Keep them coming! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Why
The
Release Pleaseworkflow onmainfails on most pushes, so release PR #172 (chore(release): 0.6.0) stopped updating. The failing step is "Run release-please with Communique notes". It ends withcommunique::malformed_submission: "missingchangelog; missingrelease_body", three times.Root cause
Communique 1.1.3 sends
max_tokens: 4096to the model whencommunique.tomldoes not set it (jdx/communique@v1.1.3src/generate.rs:177.unwrap_or(4096), passed on insrc/providers/anthropic.rs:147). This repo only had it commented out (communique.toml:20,#max_tokens = 4096).Communique makes the model send the changelog, the title and the release body in one
submit_release_notescall, even with--concise. That call needs more than 4096 tokens once a release has about 15 commits. The model's answer is cut off, so the call arrives without its last fields. Communique retries 3 times (src/agent.rs:15,:229), then keeps the last attempt only if it has a changelog or a release body (parse_submission_lenient,:69and:235). Whether a run passes depends on which field the cut-off landed in.Evidence from the run logs (
gh run view <id> --log):{ "release_title": "..." }and nothing else. There is nothing to salvage.WARN submit_release_notes was malformed 3 times, followed by the list of missing fields (most oftenrelease_body) andsalvaging the last attempt. For example 50860e8 (run 41652009410) passed only because its last attempt kept the changelog. The 7 successful runs with 8 to 15 commits (for example run 40970948045) have no such warning. The first warning is in run 40973294039 at 16 commits.Tokens: N input + M output. Regressing M on the number of tool calls (Running N toolslines) over the 33 runs gives M = 12359 + 47.9 x tool calls (R^2 = 0.983). The intercept is 3 x 4096 = 12288 (the three cut-off submissions) and the slope is the cost of one tool call.Not the cause: the model (
COMMUNIQUE_MODELis empty, so the defaultclaude-opus-4-7was used on passing and failing runs alike), the Communique version (1.1.3 in every run, pinned inmise.tomlandmise.lock), and the input size (the input only moves the point where the cut-off starts).release.ymlrunscommunique generate <tag>with the samecommunique.toml. It has the same limit, so the next real release would hit the same cut-off.What changed
communique.toml:max_tokens = 16384under[defaults], with a comment that names the failure. 16384 is the default Communique 1.2.2 chose. It is 4 times the old limit. The 0.6.0 changelog in the release PR is about 9 KB, and the release body is longer.src/tools/release-please-runner.ts: newdescribeCommuniqueFailure. When Communique's stderr hascommunique::malformed_submission, the runner error now ends with a hint to raisemax_tokensincommunique.toml. Other failures are unchanged.test/unit/tools/release-please-runner.test.ts: tests for the hint, and a test thatcommunique.tomlsetsmax_tokens >= 16384inside[defaults]. Communique ignores a key outside[defaults]without a warning (checked below), so the test checks the section too.docs/RELEASE-PROCESS.mdandsrc/tools/README.md: a recovery note and the new helper.No workflow, lockfile, CLI JSON or schema change.
Validation
Tests that fail on
mainand pass here (node_modules/.bin/vitest run test/unit/tools/release-please-runner.test.ts): onorigin/main3 fail (describeCommuniqueFailure is not a functiontwice, andmax_tokensmissing in[defaults]). On this branch 32 of 32 pass.Gates on 6f859c5:
npm run format:check: passnpm run lint: 0 warnings, 0 errorsnpm run typecheck: passnpm run test:unit: 139 files, 1877 tests passLive check without any real key or paid model: the real Communique 1.1.3 binary (the one pinned in
mise.toml) against a local stand-in for the Anthropic API that recordsmax_tokensand cuts the answer off when it does not fit. The stand-in is my model of the cut-off, and the log analysis above is the evidence that the real API does it. It checks what the real binary sends and how the real runner reacts:max_tokenscreateCommuniqueChangelogNotes()origin/maincommunique.tomlcommunique::malformed_submissionwith the same text as the CI log, plus the new hintmax_tokensat the top level instead of[defaults]I could not run the workflow itself on this branch. It checks out
main, and a run would call the paid model and write to the release PR.After merge, the next push to
mainruns Release Please with the new limit. TheWARN ... salvagingline should be gone from that run's log, and the run needs only one submission.Generated with
xum• Model:anthropic:claude-sonnet-5-5• Thinking:xhigh