Skip to content

fix(release): raise the Communique response token limit so release notes are not cut off - #295

Merged
ThomasK33 merged 1 commit into
mainfrom
fix/release-notes-max-tokens
Nov 3, 2026
Merged

ThomasK33 merged 1 commit into
mainfrom
fix/release-notes-max-tokens

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Why

The Release Please workflow on main fails on most pushes, so release PR #172 (chore(release): 0.6.0) stopped updating. The failing step is "Run release-please with Communique notes". It ends with communique::malformed_submission: "missing changelog; missing release_body", three times.

Root cause

Communique 1.1.3 sends max_tokens: 4096 to the model when communique.toml does not set it (jdx/communique@v1.1.3 src/generate.rs:177 .unwrap_or(4096), passed on in src/providers/anthropic.rs:147). This repo only had it commented out (communique.toml:20, #max_tokens = 4096).

Communique makes the model send the changelog, the title and the release body in one submit_release_notes call, even with --concise. That call needs more than 4096 tokens once a release has about 15 commits. The model's answer is cut off, so the call arrives without its last fields. Communique retries 3 times (src/agent.rs:15, :229), then keeps the last attempt only if it has a changelog or a release body (parse_submission_lenient, :69 and :235). Whether a run passes depends on which field the cut-off landed in.

Evidence from the run logs (gh run view <id> --log):

  • The failing runs (run 41675468500 on cd7ba02, and the same on 307c0e9, 00e2561 and f19008c): the last attempt Communique received was { "release_title": "..." } and nothing else. There is nothing to salvage.
  • The passing runs were broken in the same way. All 33 successful runs I read with 16 to 77 commits since v0.5.0 print the line WARN submit_release_notes was malformed 3 times, followed by the list of missing fields (most often release_body) and salvaging the last attempt. For example 50860e8 (run 41652009410) passed only because its last attempt kept the changelog. The 7 successful runs with 8 to 15 commits (for example run 40970948045) have no such warning. The first warning is in run 40973294039 at 16 commits.
  • Every attempt used the full 4096 tokens (this fits the numbers, I did not see the stop reason in the logs). The success runs print Tokens: N input + M output. Regressing M on the number of tool calls (Running N tools lines) over the 33 runs gives M = 12359 + 47.9 x tool calls (R^2 = 0.983). The intercept is 3 x 4096 = 12288 (the three cut-off submissions) and the slope is the cost of one tool call.
  • Upstream agrees. Communique 1.2.2 raised its default from 4,096 to 16,384 and fixed "large releases could exhaust the response budget while producing the changelog field" (jdx/communique PR screenshot: add --out <path> like record export #223 and wait: a wait without --timeout never returns; use a finite default #224).

Not the cause: the model (COMMUNIQUE_MODEL is empty, so the default claude-opus-4-7 was used on passing and failing runs alike), the Communique version (1.1.3 in every run, pinned in mise.toml and mise.lock), and the input size (the input only moves the point where the cut-off starts).

release.yml runs communique generate <tag> with the same communique.toml. It has the same limit, so the next real release would hit the same cut-off.

What changed

  • communique.toml: max_tokens = 16384 under [defaults], with a comment that names the failure. 16384 is the default Communique 1.2.2 chose. It is 4 times the old limit. The 0.6.0 changelog in the release PR is about 9 KB, and the release body is longer.
  • src/tools/release-please-runner.ts: new describeCommuniqueFailure. When Communique's stderr has communique::malformed_submission, the runner error now ends with a hint to raise max_tokens in communique.toml. Other failures are unchanged.
  • test/unit/tools/release-please-runner.test.ts: tests for the hint, and a test that communique.toml sets max_tokens >= 16384 inside [defaults]. Communique ignores a key outside [defaults] without a warning (checked below), so the test checks the section too.
  • docs/RELEASE-PROCESS.md and src/tools/README.md: a recovery note and the new helper.

No workflow, lockfile, CLI JSON or schema change.

Validation

Tests that fail on main and pass here (node_modules/.bin/vitest run test/unit/tools/release-please-runner.test.ts): on origin/main 3 fail (describeCommuniqueFailure is not a function twice, and max_tokens missing in [defaults]). On this branch 32 of 32 pass.

Gates on 6f859c5:

  • npm run format:check: pass
  • npm run lint: 0 warnings, 0 errors
  • npm run typecheck: pass
  • npm run test:unit: 139 files, 1877 tests pass

Live check without any real key or paid model: the real Communique 1.1.3 binary (the one pinned in mise.toml) against a local stand-in for the Anthropic API that records max_tokens and cuts the answer off when it does not fit. The stand-in is my model of the cut-off, and the log analysis above is the evidence that the real API does it. It checks what the real binary sends and how the real runner reacts:

Config Request max_tokens Result through createCommuniqueChangelogNotes()
origin/main communique.toml 4096 on all 3 attempts communique::malformed_submission with the same text as the CI log, plus the new hint
this branch 16384 first answer accepted, notes written
max_tokens at the top level instead of [defaults] 4096 key ignored silently

I could not run the workflow itself on this branch. It checks out main, and a run would call the paid model and write to the release PR.

After merge, the next push to main runs Release Please with the new limit. The WARN ... salvaging line should be gone from that run's log, and the run needs only one submission.


Generated with xum • Model: anthropic:claude-sonnet-5-5 • Thinking: xhigh

…tes are not cut off

Communique 1.1.3 sends max_tokens=4096 when communique.toml does not set it.
One submit_release_notes call (changelog, title and body) needs more than
that once a release has about 15 commits, so every call was cut off and
Release Please failed with communique::malformed_submission whenever the
last cut-off call had no changelog. Set max_tokens = 16384 under [defaults]
(the default Communique 1.2.2 moved to), pin it with a test, and add a hint
to the runner error for this failure.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-03T17:15:18.979906Z 6f859c5 Manual request
🔒 Security Review ✅ Completed 2026-11-03T17:15:40.595447Z 6f859c5 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 6f859c50fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 6f859c50fa

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit f3fafb7 Nov 3, 2026
14 checks passed
@ThomasK33
ThomasK33 deleted the fix/release-notes-max-tokens branch November 3, 2026 18:04
@github-actions github-actions Bot mentioned this pull request Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant