Skip to content

fix(runtime): bound malformed compaction retries - #3751

Open
somewan820 wants to merge 4 commits into
apache:mainfrom
somewan820:fix/compaction-malformed-summary-liveness
Open

fix(runtime): bound malformed compaction retries#3751
somewan820 wants to merge 4 commits into
apache:mainfrom
somewan820:fix/compaction-malformed-summary-liveness

Conversation

@somewan820

Copy link
Copy Markdown
Contributor

Summary

  • repair a malformed text checkpoint once with stricter instructions while keeping the existing checkpoint validator unchanged
  • stop redispatching the same malformed compaction input across later steps and Turns; changed history, model, connection, or context-budget inputs remain eligible
  • preserve the granular failure detail through Runtime Host epoch 49 and show actionable Desktop recovery guidance

Fixes #3743

Verification

  • npm run build
  • npm run lint
  • npm run format:check
  • npm run typecheck
  • npx knip --workspace apps/desktop
  • npx knip --workspace packages/ui
  • focused Runtime, Runtime Host, protocol, and Desktop suites: 418 passed
  • node scripts/protocol-epoch-check.mjs --base origin/main --head HEAD (epoch 48 -> 49)

npm test passed the affected Runtime (3,052 tests) and Desktop (1,497 tests) workspaces, but the concurrent root run exited nonzero on three unrelated timing/process tests. Each passed when rerun in isolation:

  • Storage root authority: 28 passed, 3 platform skips
  • Runtime Host continuation: 8 passed
  • Runtime Host kernel: 66 passed

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Codex inspected the incident evidence and repository, implemented the Runtime/Runtime Host/Desktop changes, added tests, and drafted this PR description. The commit includes a Generated-by: Codex trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes - described under Summary above
  • No

@xxhZs xxhZs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 480f9cfaa9933f5d410a3589a8eae59ea9ad58cb.

The core liveness fix is sound: strict checkpoint validation remains intact, malformed summaries get one bounded repair attempt, the Turn latch prevents repeated calls in the same Turn, and the Session-backend fingerprint circuit stops unchanged bad input from being retried across Turns. The granular failure reason is also preserved through Runtime Event → Runtime Host failed-turn state → Desktop, with the compatibility epoch advanced to 49. Exact-head test is green. I found no P1 correctness issue.

I do see two contract gaps worth resolving before approval:

  • [P2] Recovery behavior is only closed on Desktop. CLI/TUI still consume the generic context_budget_exhausted class and do not use contextBudgetExhaustedDetail. If #3743 is explicitly Desktop-only, please record CLI/TUI as follow-up scope; otherwise the cross-surface recovery contract is incomplete.
  • [P2] Fingerprint invalidation is not regression-locked. The implementation is intended to re-enable compaction when model, connection, context-window budget, or request shape changes, but tests only prove source-history invalidation. Please add a parameterized regression covering unchanged input plus at least model, connection, and context-window changes; this fingerprint is the core cross-Turn safety/liveness boundary.

Non-blocking follow-ups:

  • [P3] Repair coverage only exercises malformed_summary_missing_section; truncated and too_small_for_fold share the path but have no second-call regression, and the repair prompt is not reason-specific.
  • [P3] Desktop recovery is actionable copy rather than an actionable control: deriveFailedTurnRecovery() returns an action, but the turn view model retains only the label, so there is no direct settings/model/new-task affordance.

Recommendation: this can proceed to maintainer review as the Desktop liveness fix, but I would clarify the CLI/TUI scope and add the configuration-change fingerprint tests before approval.

AI-assisted review disclosure: OpenAI Codex inspected the exact-head diff, protocol/runtime/host/Desktop flow, focused tests, and live CI state; I verified the findings against the source before posting.

简体中文

核心限流与熔断方向正确,没有发现 P1;当前主要缺口是 CLI/TUI 未消费细分失败原因,以及跨 Turn 指纹对 model/connection/context-window 变化的契约缺少回归测试。另有 repair 场景覆盖和 Desktop 一键恢复能力两个非阻断缺口。

@somewan820
somewan820 force-pushed the fix/compaction-malformed-summary-liveness branch from 480f9cf to cbe797d Compare August 25, 2026 06:35
@somewan820

Copy link
Copy Markdown
Contributor Author
Review follow-up

Addressed the fingerprint-invalidation P2 in cbe797d81.

The new table-driven regression verifies the complete cross-Turn circuit contract:

  • unchanged compaction input remains blocked;
  • a model change is eligible again;
  • a connection change is eligible again;
  • a context-window budget change is eligible again;
  • a request-shape change is eligible again.

The existing source-history invalidation regression remains in place.

Scope clarification for the other P2: #3743 and this PR are explicitly scoped to Desktop + Runtime Host. CLI/TUI currently retain the generic context_budget_exhausted presentation and should be handled as follow-up surface work rather than expanding this liveness patch. The two P3 items from the review also remain non-blocking follow-ups.

The branch was rebased onto main@902d78523. After rebuilding @maka/core, the full Runtime suite passes: 3,058 tests, 3,045 passed, 13 skipped, 0 failed. Biome and git diff --check are clean. CI is running on exact head cbe797d81.

This is ready for maintainer review after CI completes.

AI assistance disclosure: Codex implemented the review follow-up, ran the verification, and drafted this response; I verified the result before posting.

简体中文

已在 cbe797d81 补齐指纹失效回归测试:相同输入仍被阻止;模型、连接、上下文窗口预算或 request shape 变化后都会重新允许压缩;原有的源历史变化测试继续保留。

范围方面,#3743 和本 PR 明确限定为 Desktop + Runtime Host。CLI/TUI 目前仍展示通用的 context_budget_exhausted,应作为后续界面工作处理,不在这个活性修复中扩展范围。review 提到的两个 P3 也保留为非阻断后续项。

分支已 rebase 到 main@902d78523。重建 @maka/core 后,Runtime 完整测试为 3,058 tests、3,045 passed、13 skipped、0 failed;Biome 和 git diff --check 均通过。CI 正在新 head cbe797d81 上运行。

CI 通过后即可请求 maintainer review。

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found a blocking issue.

[P2] Repair failure overwrites the initial malformed cause and retries indefinitely

Once initial.defect is malformed_summary_*, the repair budget is spent, but generateSummary(1) can itself throw output_length or be wrapped as provider_error. That final reason replaces the malformed fact, so neither the session-backend fingerprint circuit nor the Turn latch blocks the next identical compactHistory — leading to 4 physical calls across two compactions instead of 2.

Fix: arm the circuit with the original malformed reason whenever repair is entered and does not produce a checkpoint.

Checks on cbe797d811 are test: success.

简体中文存在修复失败覆盖原始缺陷导致的重试环。

@somewan820

Copy link
Copy Markdown
Contributor Author
Review follow-up

Fixed the blocking repair-failure P2 in 83f7e56d5.

Once repair is entered, every path that fails to produce a valid checkpoint now reports the original malformed reason to the caller. This covers repair output_length, provider exceptions (wrapped as provider_error), empty output, and a second malformed completion. The secondary repair failure is retained as the error cause, while the session fingerprint circuit and Turn latch receive the stable original malformed defect.

The end-to-end regression reproduced the review's exact failure on the prior head: two identical compactHistory calls made 4 physical provider calls. It now asserts 2 total calls (initial + one repair); the second compaction dispatches none and returns malformed_summary_missing_section.

Verification on 83f7e56d5:

  • Runtime build passed
  • focused summarizer/backend tests passed
  • full Runtime suite: 3,061 tests; 3,048 passed, 13 skipped, 0 failed
  • Biome and git diff --check passed

AI assistance disclosure: Codex reproduced the finding, implemented the fix and regression coverage, ran verification, and drafted this response; I verified the result before posting.

简体中文

已在 83f7e56d5 修复该阻塞问题。

一旦进入 repair,只要最终没有生成有效 checkpoint,对外都会保留第一次的 malformed 原因;repair 阶段的 output_length、provider 异常(内部归类为 provider_error)、空输出和再次 malformed 都不会再覆盖它。第二次修复失败的具体原因仍保留在 error cause 中,而 session 指纹熔断和 Turn latch 会收到稳定的原始 malformed 缺陷。

端到端回归在旧 head 上复现了 review 描述的问题:相同输入连续两次 compactHistory 会产生 4 次物理 provider 调用。修复后总调用数为 2(首次生成 + 一次 repair),第二次 compaction 不再发请求,并继续返回 malformed_summary_missing_section

83f7e56d5 验证结果:Runtime build 通过;相关 summarizer/backend 测试通过;Runtime 完整套件 3,061 tests、3,048 passed、13 skipped、0 failed;Biome 与 git diff --check 通过。

@somewan820
somewan820 force-pushed the fix/compaction-malformed-summary-liveness branch 2 times, most recently from 3e9a3d3 to 943d344 Compare August 25, 2026 14:17

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no code blocking issues, but the change cannot be merged as is.

[P2] RUNTIME_HOST_COMPATIBILITY_EPOCH collision and merge conflict

Head 943d344af sets RUNTIME_HOST_COMPATIBILITY_EPOCH = 50 for compaction detail, but main already uses 50 for WorkHub coordination summaries (merged #3798). The two 50s have different meanings, so incompatible peers would accept or reject the wrong fields. Also packages/runtime-host/src/protocol/index.ts is CONFLICTING.

Fix: rebase onto current main and bump epoch to 51 (strictly greater than main's 50), keeping the comment load-bearing.

Hosted test: SUCCESS (32858535454) on code.

简体中文epoch 与 main 上的 50 语义不同,需要重排并升到 51。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

Preserve granular context-budget failure details through Runtime Host and show actionable Desktop recovery guidance.

Generated-by: Codex
@somewan820
somewan820 force-pushed the fix/compaction-malformed-summary-liveness branch from 943d344 to 4c37dc5 Compare August 26, 2026 02:40
@somewan820
somewan820 requested a review from Astro-Han August 26, 2026 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(runtime): malformed compaction summaries can permanently strand a session

3 participants