Repository navigation
Polyglot re-audit remediation: P0 security/eval hygiene + P1-T2 MCP origin groundwork - #93
Open
AnzoBenjamin wants to merge 209 commits into
Open
AnzoBenjamin wants to merge 209 commits into
AnzoBenjamin wants to merge 209 commits into
Conversation
…wser OpenRouter user-agent Keep remaining full-suite flakes from depending on monorepo cwd or a bun-only user-agent string.
The Unicode property-escape regex patterns in findingTextSchema and coverageEntrySchema (\p{Cc} / \p{Cf}) are rejected by strict provider-side schema validators, so every general-agent LLM request died with 400 Invalid request parameters before any generation.
Add a pattern-free providerInputSchema twin with identical bounds and carry the hygiene rules in describe text; handler enforcement on inputSchema is unchanged. Align AuditFindingsInput with z.input so the api-abi alias and omittable noIssuesFound match the generated WriteAuditFindingsParams contract, add portability/type-surface regression tests, and regenerate the tool type mirrors.
Document the dual-surface schema contract in common/knowledge.md: providerInputSchema is the JSON-Schema-portable provider/wire surface that compileToolDefinitions derives the wire JSON Schema and generated tool types from, while inputSchema stays the handler-enforced surface with the control/format-char and single-line regexes plus the superRefine cross-field rules. Record that AuditFindingsInput = z.input<typeof inputSchema> is the accepted call surface (legacy api-abi findings[].domain alias, omittable noIssuesFound), and that the generated tool-type mirrors are produced by bun run scripts/generate-tool-definitions.ts and must never be hand-edited; cli/knowledge.md adds the matching initial-agent-type-sources.generated.ts mirror rule. Refreshing both knowledge files in this commit makes their timestamps newer than cli/src and common/src, satisfying the guard:memory-drift staleness check that rejected the push of the preceding fix commit.
Committed-prune no longer credits unreviewed bytes into gatePassedFiles; fingerprint-mismatch docs now match collectReviewerAttestationIssues; reviewerOriginFromGateId generalizes to specialist gateIds and owed-revalidation seeding covers every finding family; condone keys are reviewer-namespaced with legacy-key migration; agentFindingSchema gains an optional reviewer field and both repair-progress checks scope open-id sets to the owning family.
…budget removal Parallel wave 1 (M3-T3 races, M3-T4 retry/fallback, M4-T3 CI truth, M5-T9 docs) plus 11 reliability-reviewer gate repair cycles: indexer lock release/reclaim atomicity (verify-before-rename, token-verified heartbeat, scratch sweep), detached IndexManager forwarding with epoch mirroring and pending-delta drain, agent-runtime gemini-with-fallbacks error-class gating/abort-aware retry across all legs, sdk retry-config runWithRetryPolicy abort semantics, cli exit-path queue draining with attachment folding and ordered shutdown, CI truth fixes (flake ledger, empty-glob failure, gate-helper drift check), docs overhaul (knowledge.md filled, doc citation checker), 60s queue watchdog removal and unlimited-default no-verdict budgets, plus advisory hardening (sync-throw lock guard, tightened budget resolver signatures).
Bounded stream/regex/traversal guards across agent-runtime (LRU cachedRegExp, 64KB tool-call buffer budget, MAX_JSON_CANDIDATES=32 candidate loop, visited-set hardening in publishSelfMutatedPaths), precompiled language-profile signal regexes, and scripts/measure-perf-guards-baseline.ts with measured before/after parity evidence per performance-specialist findings. CLI: ask-user Esc data-loss skip-guard, testable exit-queue-drain helper, tool-result error-scan confined to the tool-result envelope. Adds failure-path test campaigns (process-structured-edit, workspace-path-leases, orchestration-ledger, context-consolidation, failover-integration un-skip) and session STATUS checkpoints for the remediation wave.
Add afterAll registry cleanup in background-agent-jobs.test.ts so never-settled background jobs no longer bleed across test files (full agent-runtime suite 1870 pass / 0 fail). Unblock the pre-push gate: reword the knowledge.md test command, add a perf-evidence mention, add dated knowledge-refresh entries to cli/knowledge.md and common/knowledge.md, and allowlist check-doc-citations.ts plus the new M3-T2 benchmark in scripts/.coverage-allow. Retarget the base2 gateFileMarker parity harness to readGateFileContentMarkerUncached with GATE_MARKER_CACHE_MAX and a fresh transient Map so the cache-stamping path resolves. Add scripts/measure-m3-t2-hot-paths.ts (7 rows, all parity/contract assertions passing) and record the session STATUS.md checkpoint with the audit scratchpad artifacts.
Pin the MAX_PROTECTED_CONTENT_SCAN_CHARS boundary both directions with two new tool-result-eviction tests (path beyond the 5M-char cap evicts/tombstones fail-open; path within the cap stays protected), correct measurement design in measure-m3-t2-hot-paths.ts (CASE 6 after row times the full shipped run shape including the snapshot walk; CASE 5 discloses the no-eviction steady state; CASE 7 note cites the eviction/tombstone contract with the scan cap separately pinned), mark PLAN.md M3-T2 done with the validation checkpoint receipt, record the certification checkpoint and full-repo sweep classification in STATUS.md, and capture the parallel-validation OOM/timeout lesson in LESSONS.md.
… mock-leak teardown Address all three BLOCKING reliability-reviewer findings: fix the /exit drain test's clearQueue mock to honor the one-at-a-time contract (splicing mock, 2 drains + 1 terminal check) and add a partial-failure test proving the drain continues past a failed saveToHistory; add a file-walker test pinning the dir-lstat re-verify against a directory swapped to an out-of-project symlink mid-walk (TOCTOU); add three guarded-submit busy-path tests via /resume-plan (busy+streaming queues, busy+chainInProgress queues, idle sends with EXECUTE_PLAN mode). Fix the cross-file bleed class: sdk e2e setupE2eMocks spies were never restored, so the leaked promptAiSdk mock broke llm-ontrim-retry in full-suite runs — add idempotent teardownE2eMocks and wire afterAll teardown into the 13 sdk/e2e files that trigger setup; correct concurrent-streams assertions to check real content isolation instead of reference inequality; restore a wide mockLayout in build-mode-buttons afterAll so its xs layout no longer bleeds into status-bar.test.tsx. Append M3-T3 certification checkpoint to STATUS.md; checkbox flip deferred until the automated gate passes over the changed test files.
Complete the cross-file-bleed fix across all workspaces: the 3 remaining agents/e2e suites that call setupE2eMocks directly now restore spies via afterAll(() => teardownE2eMocks()), matching the sdk/e2e pattern (13 files previously, 3 now). Bun's shared module registry keeps mocks alive past each file's suite, so teardown prevents bleed into later files. No active bleed existed in agents; suites re-run green (9 tests) and typecheck passes. STATUS.md records the checkpoint and gate evidence.
Add dated knowledge-refresh entry recording the M3-T3 test-only wave (/exit drain splicing contract, guarded-submit /resume-plan queueing tests, build-mode-buttons mockLayout teardown) so cli/knowledge.md stays newer than src/ and the pre-push staleness guard keeps passing.
…review Mark M1-T7 as in-progress in the harness-remediation session plan and set the current-task pointer, so the worktree is fully committed and clean for the queued committed-surface review that mints the plan-gate receipt.
…d fixtures, 25min test budget
ci.yml: '/' in matrix.package made every packages/* job fail its flake-ledger upload step deterministically (artifact names reject '/'), so sanitize via FLAKE_SUITE=${SUITE//\//-}; test-job timeout_minutes 10->25 because all 3 agent-runtime attempts of run 36056290709 died at exactly the 600s cap mid-suite under --isolate budget, not a hang. check-job.test.ts: deterministic makeDeadPid() (spawnSync'd exited child) replaces the hardcoded pid 1234 fixture, which can be a live daemon pid on ubuntu runners and failed 3 liveness tests in CI only; plus import/dead-code cleanup from two reviewer rounds. PLAN.md: M1-T7 flipped done with the plan-gate receipt.
Three related reliability fixes. (1) CLI wrapper auto-update hardening: async binary-version probe (2s timeout) with metadata self-heal, in-memory-only packaged-version fallback that persists no unverified platform claim, poisoned-pendingVersion recovery via a 24h failedPendingVersion backoff, failed staged updates that keep the working binary, --update exit(1) on failed apply, crash-signature quarantine bounded by a 3-attempts/24h crashHeal budget preserved across reinstalls, wrapper-skew stderr notice at child exit, bounded background update check with backoff, and an atomic writeMetadataPatch helper (mirrored in the codecane staging wrapper). (2) Editor output ceiling: optional maxOutputTokens added to the template/definition type surfaces, forwarded into the AI SDK stream params, with an explicit 32000 ceiling on the editor and repair-editor so provider defaults (~4k on the Anthropic path) no longer cut large multi-edit edit_transaction payloads mid-JSON. (3) Transport truncation coverage: classification/repair scan bound raised from 256KB to 1MiB with candidate-capped recovery for oversized payloads so huge truncated tool-call payloads are classified and repaired instead of surfacing as generic syntax errors. Regression tests cover all three; the initial-agents-dir type mirror was regenerated.
…ening wave Satisfies the pre-push memory-drift staleness guard: dated 2026-09-25 knowledge-refresh entries added after 0dec8a9 touched cli/src/ and common/src/ without refreshing the knowledge files.
Replace the flat numbered result dumps in code_search, glob, list_directory, and query_index with grouped, collapsible renderings so large result sets stay scannable. Add a shared renderer (discovery-results.tsx) providing CollapsibleGroup with count badges and an ellipsis-N-more affordance, statusGlyph checks/crosses/spinner, pattern-match highlighting, and cwd-relative path shortening. code-search parses formatted and raw ripgrep output into per-file groups, glob groups files by directory, list_directory renders one collapsible group with a dirs/files badge and defensively filters null/non-object directory entries, query_index drops the redundant Status line in favor of a header glyph with a State line only for non-ready states, and DiscoveryOutput gains an opt-in showHeader prop. Tests rewritten and extended for all of the above.
…t turn Raise the editor/repair-editor maxOutputTokens from 32000 to 64000 (the opus-4.7 maximum) because 32k still truncated large multi-edit edit_transaction payloads mid-JSON. Bound the final set_output receipt transcript to the newest 60 messages with oversized strings capped at 2000 chars instead of inlining the full history, and add a generator-side receipt retry with progressively tighter bounds (20/800, then 6/300) when the receipt is rejected, since the runtime's missing-output retry loop skips handleSteps agents. In executeSubagent, stop re-throwing non-cancellation subagent errors through Promise.allSettled: a child that committed its edits but failed receipt delivery no longer fails the whole parent turn and instead degrades to a structured error output (mapped to an explicit partial diagnostic by the spawned-output normalizer); cancellation and abort errors still propagate.
Document the grouped discovery tool output rendering (CollapsibleGroup, statusGlyph, shortenPath, highlighted pattern matches) across code_search, glob, list_directory, and query_index. This entry keeps cli/knowledge.md newer than its sibling src/ so the pre-push memory-drift staleness guard passes.
…ider output token ceiling - sdk: treat NoOutputGeneratedError as retryable (no content yielded) and failover-eligible so empty provider streams retry with backoff then failover to the backup model instead of killing the run; also drop the dead rateLimitErrorDetails variable and the unused params parameter of withDefaultRequestTimeout (3 call sites updated). - agent-runtime: harden spawn receipts — credit receipt-backed findings per-finding instead of zeroing them on any overclaim, make buildRuntimeAgentReceipt never-throw with a field-complete failed-receipt fallback, add an explicit diagnostic when edit_transaction results parse to zero mutation receipts, and preserve mutation receipts through tool-result eviction (slim stale file_mutation_result instead of tombstoning; leave commit_receipt untouched). - sdk: forward provider-config context.outputTokens (defaultCapabilities/modelCapabilities) as maxOutputTokens on streaming requests when the agent template leaves it unset; document the ceiling and precedence in docs/configuration.md.
Indexer/code-map reliability sweep (M4-S6): persisted-adjacency coverage validation, content-keyed fileTypeSet + per-index IDF caches, chunk-memo project scoping and FIFO bound, loud/skipped-vector embedder batch handling, file-walker lstat re-verify, parse path-traversal guard, grammar-wasm byte cap/backoff. Buffbench evals hardening (M5-T7): judge-model config + calibration gold set, boundary-aware classifyCommand, idiomScore clamp, classifyBreadth and peakConcurrency fixes, EvalDataV2 file validation, true-median measured runs. write_audit_findings: recover findings entries serialized as per-entry JSON strings while still failing closed on malformed input; widen AuditFindingsInput accordingly. executeSubagent: contain child-abort so a subagent crash no longer fails the parent turn. cli index-command: widen waitUntilReady return type for real-vs-mock IndexManager. Adds harness-remediation session STATUS + resolution findings artifacts.
… wave Unblocks the pre-push check:ci-local guard:memory-drift gate that rejected the prior push for stale cli/common knowledge relative to sibling src changes.
…derable crash) a text content block containing markdown nested <ContentWithMarkdown> inside <text>; ContentWithMarkdown returns a native <markdown> renderable for markdown content, and OpenTUI's TextNodeRenderable.add throws 'only accepts strings, TextNodeRenderable instances, or StyledText instances' at append-initial-child — every assistant response then hit the per-block error boundary and rendered 'Could not render this text block.' The fix branches on hasMarkdown: markdown content renders ContentWithMarkdown at box level inside a full-width <box>; plain prose keeps the unchanged <text> wrapMode/fg path. Regression suite single-block-markdown.test.tsx (3 tests, real OpenTUI testRender harness) pins fallback absence for assistant + user markdown blocks and the plain-path behavior. Verified live in a tmux dev run: two turns render real content, zero fallback errors. Full CLI suite 3620/0; cli typecheck green.
…ory explosion) SyntaxStyle.fromStyles() allocates a native (Zig-side) handle that is never GC'd and has no finalizer; createMarkdownSyntaxStyle was called on every render, so each streaming chunk leaked one native handle and the compiled binary's memory exploded as soon as the agent responded. The factory now memoizes one handle per distinct palette object via a module-level WeakMap (undefined-palette path memoized once), with clearMarkdownSyntaxStyleCacheForTests() following the repo's *ForTests reset convention. Verified live: compiled binary in a scratch dir under a 3G MemoryMax cap — idle 254MB, post-msg2 settle flat at ~321MB (fluctuation <250KB over 50s), transient streaming peaks reclaimed by GC; zero 'Could not render' occurrences; TUI responsive throughout. New test suite opentui-syntax-style.test.ts (5 tests) pins memoization, per-object identity, mapping fidelity, and the reset.
- Bounded LRU caches for markdown/code SyntaxStyle native handles (fixes streaming memory explosion; one handle per distinct style record, never destroyed while referenced) - setImmediate macrotask yield gates (8ms performance.now() slices, per-invocation state) in code-map parse and indexer build/update loops so cold index builds stop starving the TUI event loop - Durable recorded gate-on/gate-off baselines + deterministic yield/cache regression pins; headingFg fallback mirrored in cache key; tsAliasCacheByRoot independently bounded
Memoize Language+Parser on the shared languageTable entry before any tags-query work. In compiled binaries whose release layout omits the tree-sitter-queries .scm files, the old code threw AFTER Language.load succeeded, left cfg.parser unset, memoized nothing, and made EVERY parsed file re-run a full Language.load on the same grammar wasm (observed: one wasm opened 853x in one cold index build, ~5GB churn, boot freeze). Query resolution is now fail-open: a missing/unreadable .scm or a Query compile failure leaves cfg.query undefined with the cached grammar intact. Added a one-time-per-language DEBUG_PARSING log latch (loggedQueryUnavailable) and an existence-checked dirname(process.execPath)/tree-sitter-queries/ candidate in resolveQueryPath for the binary release layout; corrected the resolveQueryPath doc comment. Regression pin 'missing tags query keeps the grammar cached (compiled-binary reload-storm pin)' asserts loadLanguage called exactly once, cfg.parser cached, cfg.query undefined, and a second createLanguageConfig call is a cheap cache hit; uses createRequire(import.meta.url) so the wasm resolve is ESM-portable per review feedback.
…isolation The 2s UI chip poll called getStatus(), which scheduled ensureBuilt() and could re-arm sticky forceRefresh, pinning 'idx stale'/'idx refreshing' chips with zero CPU activity; scheduling now lives only in waitUntilReady/query/queryBlended. Regression test: an IndexManager for root A never loads/serves the index persisted under another root (home-dir index interference class).
…(reviewer advisories) Fix review- misplaced claims: advanceEpochAfterForward is single-threaded synchronous JS (not lock-atomic), and the adoptMergedIndex lost-CAS branch serves the on-disk snapshot while discarding the adopter's merged edges.
…croll snap-back and zoom breakage) Replace the multiplicative height-cache calibration with real per-message measurements: message wrappers expose a measurable host box whose rendered height is captured on mount/resize, width-tagged, and retained on unmount, so spacers and offsets use real heights instead of scaled estimates, and the calibration effect is deleted entirely. Fix the chat scroll glitch while scrolling up: scroll compensation is now anchored by message ID across commits (computeAnchorDelta) rather than a positional index, so prepends/prunes/resets above the viewport can no longer compensate the wrong message or clamp scrollTop to 0 (snap-back). The compensating write goes through a new adjustScrollTop API in useChatScrollbox that folds into in-flight animations, defers re-verification against settled geometry (bounded verify chains that re-apply residuals OpenTUI clamps swallow), and arms its programmatic flag only for writes that actually move the position — leak-safe attribution so a swallowed write cannot misattribute the next genuine user scroll. All scroll-position readers use the canonical verticalScrollBar.scrollPosition source to avoid transient divergence. Fix zoom out/in: width-tagged measurements go stale on a terminal width change, so the hook re-derives estimates at the new width, skips delta anchoring across the change, and re-pins to the settled bottom when the user was at the bottom (isAtBottomNow probe + deferred settled-bottom verification once the re-wrapped layout commits). Grow the use-viewport-window test suite from ~15 to 145 cases covering height resolution, anchoring deltas, verify-chain state machines, and programmatic-flag attribution.
Templates that cross a JSON boundary (bundled-agents prebuild, parent->child bridge) arrive with plain JSON-Schema schema members; asSchema routes them to the zod-v3 converter and crashes the compiled binary with 'undefined is not an object (evaluating H.typeName)'. New coerceJsonSchemaMember/ensureAgentTemplateZodSchemas in common/src/templates/agent-validation.ts coerce such members to zod at template resolution (agent-registry local/database/codebuff-fallback branches, with the specific-version-only cache policy applied to the fallback branch) and defensively in prompts.ts buildAgentToolInputSchema. A spawned child that never calls set_output and has no harvested answer now settles as a retryable partial/missing_output receipt instead of completed-with-null (spawn-agent-utils empty-output guard). Chat viewport snapped back to the bottom whenever an agent response arrived: classifyScrollChangeMovement in use-scroll-management classifies movement-less scrollbar 'change' events (transient reflow geometry during message append) as geometry-only so auto-follow is not re-armed for a scrolled-up user, and chat.tsx keeps the scrollbar track in layout whenever content overflows so the content wrap width stays stable for width-tagged measured heights.
Follow-up hardening after the 98e4adc spawn repairs: the supervised-spawn JSON request boundary was still degrading live zod schemas (JSON.stringify makes empty husks of them) and silently dropping function-valued handleSteps, and the registry handed out raw JSON-Schema configs instead of the zod-converted templates. assembleLocalAgentTemplates now returns validateAgents' zod map, serializeAgentTemplatesForTransport demotes schema members to JSON-Schema and stringifies function handleSteps, and getAgentTemplate gates databaseAgentCache.set to specific versions only (including the codebuff-defaulted fallback branch). Scroll-to-latest snap-back: during streaming the transient maxScroll sits a fraction of a row above the integer-rounded position, so the auto-scroll effect re-wrote a within-epsilon pin on every messages change and the settled-lower layout's re-clamp snapped the viewport back up. shouldSkipAutoScrollWrite now skips the within-epsilon pin write (never the safety clamp), and VERIFY_SETTLED_BOTTOM deferred verification is one-directional — it only pushes up toward a settled bottom, never pulls down. Index cache bounds: the shared 8MB cap made a 47MB metadata.json load as null, feeding a skip-to-full-rebuild-to-rewrite-oversized stall on headless runs. The caps are split (generous 64MB for the authoritative metadata.json, historical 8MB for the rebuildable sidecars), saveIndex persists tiered-reduced snapshots (queryData/parseData, then contentSample, then chunks) or skips persisting entirely rather than writing an artifact loadIndex refuses, the CAS read is stat-guarded, the oversized-skip warning is deduped, and loadChunkSidecar/readSemanticVectorCache are stat-first. SDK session boot: computeProjectIndex passes a 64MB maxTotalBytes budget to getFileTokenScores with debug-level truncation logging, so cold start cannot spend unbounded wall-clock time in the pre-LLM tree-sitter pass.
… input-deaf TUI at startup)
detectTerminalTheme() ran two sequential OSC probes (OSC 11, then fallback OSC 10) under a shared 600ms global budget via withTimeout, but each probe armed its own independent 250ms internal timer. On fast startup (compiled binary) the global race abandoned probe 2; main() then created the OpenTUI renderer which claims stdin raw mode, and the abandoned probe's zombie timer fired afterwards, calling setRawMode(false) + stdin.pause() and silently disabling the renderer's raw mode (log evidence: 'OSC-CLEANUP wasRaw=false raw-now=true' right after the renderer's raw-mode ON). The compiled CLI was input-deaf: keystrokes echoed by the tty over the UI and Enter dead.
Fix: clamp each probe's internal timeout to the shared global deadline minus a 50ms safety guard via computeQueryTimeoutMs (1ms floor so the probe's own cleanup still runs and never leaks the stdin listener/fd), guaranteeing all probe cleanups complete strictly before the renderer takes stdin. Thread an optional { deadlineMs } through sendOscQuery/queryTerminalOSC/detectTerminalThemeCore; detectTerminalTheme computes the deadline before withTimeout. Add a computeQueryTimeoutMs describe block covering undefined/far-future/clamped/expired/invariant cases. Validated with CLI typecheck, 62/62 terminal-color-detection tests, and two consecutive tmux passes against the rebuilt binary confirming raw mode (-icanon -echo), in-box keystrokes, and Enter submission.
Add 2026-10-07 knowledge-refresh entries to cli/knowledge.md (OSC probe deadline clamp fix) and common/knowledge.md (agent-validation zod-schema preservation surfaces). These keep both knowledge.md files newer than their sibling src/ directories for the pre-push guard:memory-drift gate, which previously failed the push with 2 staleness findings.
…(mock-module guard)
Replace full-replacement mock.module('@codebuff/code-map/parse') calls in initial-session-state.test.ts with the repo's capture-spread-delegate pattern: snapshot the real module as { ...realParse } before registration, spread it before overriding getFileTokenScores, and restore { ...realParse } in finally instead of a single-export husk. This keeps un-overridden exports intact for sibling suites sharing a bun test worker, clearing 3 guard:mock-module findings. Validation: SDK typecheck 0, 20/20 file tests pass, guard:mock-module 0 findings, reviewer gate LOOKS_GOOD.
…schema boundary (fixes H.typeName crash)
Bundled agent templates (code-reviewer outputSchema, context-pruner params) go through convertJsonSchemaToZod, producing z.intersection(...) schemas that zod-v4 z.toJSONSchema rejects. The old fallback in serializeSchemaMemberForTransport left the LIVE zod member in place on the spawn transport boundary; JSON.stringify degraded it into a husk (only the ~standard marker survives as an own property; _zod internals and safeParse are non-enumerable and lost). The child's ensureAgentTemplateZodSchemas treated the husk as plain JSON Schema, and the AI SDK's asSchema routed it to the zod-v3 converter, which read def.typeName on undefined and crashed spawning reviewer-family and other bundled agents from compiled binaries.
Changes in common/src/templates/agent-validation.ts: (1) new isDegradedZodHusk detector keyed on the own '~standard' property with _zod and safeParse both absent (~standard is zod-v4's runtime interface marker, never a JSON-Schema keyword, so real JSON Schema stays out of the path); (2) coerceJsonSchemaMember re-converts husks from their own shape when possible, falling back to a permissive { type: 'object' } conversion, so a husk never survives to the model surface; (3) serializeSchemaMemberForTransport returns a permissive { type: 'object' } JSON-Schema fallback on z.toJSONSchema throw instead of leaving the live zod member — a working agent with relaxed member validation beats a crashed one. The child re-coerces the fallback through coerceJsonSchemaMember into a valid live zod schema.
Tests in packages/agent-runtime/src/__tests__/prompts-schema-handling.test.ts: husk re-coercion into live zod; the old 'leaves non-JSON-expressible schemas untouched' test replaced with 'transports as a permissive JSON-Schema fallback, not a crash husk' (built via the real pipeline with a throw-guard); and a code-reviewer-shaped full JSON round-trip test asserting every member reaches the child as a live zod schema and asSchema (if resolvable) does not throw.
Validation: common + agent-runtime typechecks exit 0; 114/114 tests pass across prompts-schema-handling, agent-registry, and spawn-agent-utils-output suites; reviewer gate LOOKS_GOOD.
…ntable any (restores strict spawn schema validation)
convertJsonSchemaToZod builds a base union containing a z.custom(...) object-branch for type-less members under anyOf (e.g. the bundled code-reviewer outputSchema's findings.items.anyOf), and zod-v4's z.toJSONSchema throws on custom types by default ('Custom types cannot be represented in JSON Schema'). That throw was the root cause superseded by 079d190's temporary permissive {type:'object'} fallback, which discarded member-specific validation strictness at the supervised-spawn transport boundary. An executed bun probe confirmed passing unrepresentable:'any' serializes the exact pipeline schema with full structure preserved (properties/required/anyOf/allOf intact; only the degenerate custom branch becomes {} under the parent's allOf object branch, so objectness is still enforced) and that the result JSON round-trips cleanly. serializeSchemaMemberForTransport now passes { io, unrepresentable: 'any' } with a WHY comment, keeping the permissive return only as a last-resort guard in the catch; the rewritten prompts-schema-handling test asserts transported members are not the bare husk and do keep structure (type:'object', required:['findings'], findings.type==='array') plus JSON round-trip stability; knowledge.md gains the dated staleness-guard entry for 079d190.
localAgentTemplates cross a JSON.stringify transport boundary into the supervised child entrypoint, where live zod schemas degrade to plain JSON-Schema objects or zod-v4 husks carrying only the ~standard marker. loopAgentSteps indexes those templates directly without the getAgentTemplate coercion backstop, so an uncoerced husk reaches the AI SDK's asSchema, which reads schema._def.typeName on undefined and crashes the child run (H.typeName). Map every transported template through ensureAgentTemplateZodSchemas at the deserialization boundary, use reference-identity change as the degradation signal, and report re-coerced template ids to stderr only — never into the receipt envelope.
message-block-helpers: add 4000-char caps with truncation markers, budget-aware cappedJsonStringify with depth/cycle guards, a depth-capped media pre-check, reference-preserving block rewrites, depth guards on tree walkers, and a capped reviewer formatter so pathological agent result shapes degrade to markers instead of overflowing the stack or flooding the TUI. index-status: show an idx degraded chip with force-redraw, add a layered peek cache (200ms peek TTL / 30s config TTL / 5s failure backoff) and a root-drift guard so the status line stops hammering the indexer on every render.
…en spawn schema/prompt surfaces thinker: merge the trailing run of per-delta assistant fragments so streamed reports are not harvested as a tail fragment, stopping at tool-call boundaries. Template/registry: re-coerce zod schemas on every database-cache hit in getAgentTemplate and harden agent-validation/prompt handling; add an optional-params hint (with enum values, bounded) to catalog lines so parents stop guessing child param shapes. Agent prompts: clarify doc-writer blocked-vs-noop status, give security-reviewer a concrete native-JSON output example and manual fingerprint echo rule, forbid echoing declared schemas in specialist output, and make tmux-cli's command param explicitly required with a resilient seq counter. Supervision: update the parent bridge server accordingly.
Add four structured output formatters to the extractSpawnAgentResultContent dispatch chain — formatFilePickerStructuredOutput, formatBasherStructuredOutput, formatGeneralAgentReceiptStructuredOutput, formatGeneralAgentSummaryStructuredOutput — so file-picker, basher, general-agent, and repair-editor results render as human-readable text instead of raw JSON blobs. Wire live activity display in handleSubagentStart: spawn_agents batch children receive the agent prompt as a textType:'reasoning' block (keeping the hasStreamedTextContent guard clear so final structured content still injects on completion); organic subagents receive it as a plain textType:'text' block.
- Add formatLibrarianStructuredOutput to render { status, answer,
relevantFiles } from librarian agents as a status line, capped answer,
and file list instead of raw JSON; wired into extractSpawnAgentResultContent
alongside the existing general-summary and external-CLI paths
- Remove checkBlockIsUnderParent, which was never called after the
moveSpawnAgentBlock refactor left it dead
- Replace (resultValue as any) casts in transformAskUserBlocks with
isRecordValue narrowing for proper type safety
…locs
ContentWithMarkdown returns a native <markdown> renderable that OpenTUI
rejects when nested inside a <text> node. All three UserTextDisplay /
UserTextWithInlineCopy / UserBlockTextWithInlineCopy paths now detect
markdown first via the new useContentHasMarkdown hook and render through
a <box> wrapper instead of falling into the <text> path.
Streaming performance: replace the per-tick O(C) hasMarkdown scan with
an incremental monotonic latch (useContentHasMarkdown). A ref carries
the scan frontier so only newly appended bytes are regex-scanned per
tick, O(C) total. The exported scanChunkForStraddledLink state machine
carries [text](url) state across chunk boundaries so straddled links are
detected without re-scanning. The exported hasNewlineThreshold helper
replaces split('\n').length with an allocation-free early-exit scan.
Pause getDiffStatsAsync and index-status polling while isStreaming via
shouldSpawnDiffStatsForStreamingTransition (falling-edge-only spawn
policy) and a streaming guard on the 10s interval, eliminating
floor(T/10) extra git subprocesses per stream.
Fix hasMarkdown regex backtracking: .* in the link pattern replaced with
[^\]]* / [^)]* so the regex cannot catastrophically backtrack on long
non-link content.
Fix 3 failing extractSpawnAgentResultContent tests: add deferral guards
in formatGeneralAgentSummaryStructuredOutput for outputKind==='browser-use',
outputKind==='external-cli', and typeof overallStatus==='string' so those
payloads reach their specialized formatters before the bare-summary path.
Extract resolveMentionReplacement + buildMentionReplacement to
cli/src/utils/mention-helpers.ts so the three mention-selection sites in
chat.tsx share one tested implementation. Also null-guard + try/catch the
followup click handler to prevent synchronous throws from crashing the
event listener.
New tests: chat-mention-helpers.test.ts (13 tests covering agent/file
split boundary, fallback, cursor splice), user-content-copy.test.ts (15
tests pinning split-equivalence and allocation-free invariant of
hasNewlineThreshold, state-machine transitions of scanChunkForStraddledLink,
and incremental-scan equivalence with whole-content hasMarkdown).
…sher timeout, code-search -l flag, security-reviewer family field
- basher: expect timeout_seconds: -1 (no timeout) instead of 300 - audit-contracts: expect allow-interactions + workspace-write for browser-use
Subagent blocks now render expanded by default so their full live stream is visible; grid-row sizing stays decoupled and manual collapse still works. The index 'degraded' classifier no longer trips on budget-driven coverage truncation or a negligible parse-diagnostic count: it now requires a build error or a corpus-scaled failure floor (ceil(fileCount*0.05) clamped to [1,10]). updateMetadataIndex self-heals stale parse diagnostics on the no-change path by re-parsing still-walked uncached diagnosed files, recomputing diagnostics/coverage, and dropping diagnostics for vanished files, so a transient parser regression no longer sticks. The status chip surfaces the reason as 'idx degraded - N parse err' via an additive diagnosticsCount peek field.
Memory V2 staleness-guard entry documenting the subagent-streaming (expanded-by-default) and idx-degraded classifier fixes already shipped in cf067f3. Keeps cli/knowledge.md newer than cli/src so the pre-push memory-drift guard passes.
…wn boundary
serializeAgentTemplatesForTransport is now null-safe: an unset localAgentTemplates catalog (top-level orchestrators, tests, some programmatic spawns) serializes as empty instead of crashing on Object.entries(undefined) — fixes the 7 pre-existing test-packages/agent-runtime CI failures.
coerceJsonSchemaMember's non-husk convertJsonSchemaToZod call is now try/catch-guarded, degrading to buildNativeFallbackSchema on a plain object that is not valid JSON Schema. This mirrors the existing husk branch and preserves the function's representability guarantee: every coerced member is a live zod schema whose z.toJSONSchema does not throw (the AI SDK asSchema compatibility condition that crashes the spawn otherwise).
The four test-surface helpers (isDegradedZodHusk, isRepresentableZodSchema, buildNativeFallbackSchema, serializeSchemaMemberForTransport) are exported so the new agent-validation.test.ts describe block can pin the boundary directly: serialize→coerce round trip of a live zod member, husk detection (including non-objects and plain JSON Schema rejection), degenerate {type:object} coercion to the loose-object fallback, and the last-resort {type:'object'} transport fallback for live-but-unrepresentable members.
…mory-drift guard) Records the 67a7527 agent-validation hardening (null-safe serializeAgentTemplatesForTransport, try/catch-guarded coerceJsonSchemaMember non-husk branch, exported crash-hardening surface + 4-case suite) as the staleness-guard touch keeping common/knowledge.md newer than common/src so the pre-push memory-drift check passes.
…ansport round trip
Enforce the documented settle wall-clock deadline as an absolute, never-reset maxLifetimeMs cap (=request.timeoutMs ?? SETTLE_DEFAULT_TIMEOUT_MS) alongside the idle-aware window: a supervised child that continuously emits stdout/stderr or bridge activity resets the idleTimeoutMs window but can no longer outrun the deadline, since the legacy single wall-clock termTimer branch is skipped whenever an idle window is armed. Bridge-request activity resets: parent-bridge-server takes an onRequestActivity hook fired on every well-formed request (before method/budget checks), supervised-spawn fans it out to the supervisor via an onActivity listener set, and captureStream treats chunk arrival as a secondary activity signal; a commitKill latch keeps late activity events and timer rearms neutralized after the deadline fires. Timeout-specific crash messages: mapSettledOutcomeToAgentOutput now distinguishes crashReason=timeout ("exceeded its activity deadline") from the generic receipt-crash fallback and names other transport reasons; the settle path passes settled.crashReason through. Serialize-side round-trip verification eliminates the last zod coercion gap: serializeSchemaMemberForTransport JSON round-trips every unrepresentable:'any' emission and re-converts it with convertJsonSchemaToZod, requiring the result to be representable under BOTH input and output io modes (isRepresentableZodSchema is now a dual-mode probe, catching transforms like z.string().transform(s => new Date(s)) whose output side is unrepresentable); unverified emissions degrade to {type:'object'} instead of crashing the child's asSchema — only z.never() actually degrades (26/26 tests), while record/map and other degenerate-but-round-trippable shapes transport verbatim with structure preserved. Also guards Object.entries(fileContext.customToolDefinitions ?? {}) in run-agent-step against hand-rolled fileContext objects schema-defaulting cannot cover.
…ld coercion owns fallback
The 2026-10-09b serialize-side round-trip predicate required the re-converted schema to be representable under both io modes. That broke the shipped pipeline-intersection contract: prompts-schema-handling.test.ts 'pipeline intersection schemas transport with full structure, not a permissive husk' failed in CI (test-packages/agent-runtime) because the code-reviewer-shaped emission's re-conversion carries a z.custom base-union branch, live but unrepresentable, and was wrongly degraded to the {type:object} permissive husk.
Corrected predicate in serializeSchemaMemberForTransport: the JSON round-tripped emission must merely be PARSEABLE JSON Schema (convertJsonSchemaToZod must not throw). Unrepresentability is owned child-side — coerceJsonSchemaMember is hardened (try/catch + representability guard → native fallback) — so transporting unrepresentable-emission members (z.custom pipeline-union branch, z.never()'s degenerate emission) verbatim is safe and preserves their full structure. Degrading them would silently lose member-specific validation strictness.
Tests corrected to the probe3-verified oracle: agent-validation.test.ts now asserts unrepresentable-emission members transport verbatim and the degrade branch is defensive-only for unparseable emissions. knowledge.md gains the 2026-10-09c amendment documenting the corrected predicate and the CI regression it fixes (memory-drift guard touch).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Polyglot re-audit remediation — protocol reliability, compaction quality, durability groundwork
Branch:
feat/polyglot-reaudit-roadmap. Roadmap + specs live under.agents/sessions/polyglot-roadmap-v2/(SPEC.md D1–D28, PLAN.md task ledger).Landed this series (per-commit changelog)
Protocol reliability (D19–D24 / PR-T1..T6)
outcomeenum onagentReceiptSchema; runtime derives outcome (crashed > missing_output > schema_invalid > truncated > ok) and downgrades a falsely-completedreceipt topartial. Closes the D24 status fail-open.last_messagefragment merge + content-addressed scratch persistence of full subagent output with a bounded parent pointer.toolArgNormalizationStats. Slice 2 (schema flattening) closed as superseded with recorded rationale + walker-recursion regression test.OPENBUFF_EDIT_BLOCKS(default off, byte-identical flag-off). Deterministic eval 10/10 (block/json 0.795–0.984). Live completion-rate A/B remains a manual gated step (no provider keys in CI).scripts/memory-drift-guard.tsaccepts a recorded LOOKS_GOOD review receipt instead of a timestamp-only knowledge.md touch.context-pruner'answers' in valuecrash fix (guards non-object tool-part values).Compaction quality & archive recall (D25–D28 / CQ-T1..T4)
enforceKnowledgeMemoryBudgetsexcludes open blockers + reviewer receipts from ceiling eviction, with a superseded-review collapse keeping the hard ceiling enforceable (performance-specialist hardened). Retention eval is now the blocking floor (9→11 scenarios).tool_result_evictionsnapshots with step provenance, bounded by the persisted 8×200×4k caps); round-trip recall test.recall_context— in-memory bun:sqlite FTS5 index per call with fail-open to the byte-identical scanner;indexState/indexErrormake empty-from-healthy distinguishable from failed-index.Durability groundwork (P2)
AgentState.compactionArchivesurvives across sessions through the checkpoint, chat-state, and SDK restore seams (no persistence code needed fixing).IdGenonAgentRuntimeDeps(optional, additive) routing the run-path identity mint sites (agentStepId,toolCallId) throughidGen.uuid(); three spy tests converted to inject a mock idGen. The determinism prerequisite for P2-T2 replay..agents/sessions/polyglot-roadmap-v2/P2-T2-DESIGN.md: append-only bun:sqlite run-journal schema, four write points, replay-by-re-driving-the-generator resume model with tool-result idempotency short-circuit, and the kill-9 resume test.Hardening
refreshTokenCount,maxTokens,tokenizer, …) are kept while every real credential key still redacts.Testing
Per-slice suites green; full
common1450/1450 andagent-runtime1958/1958 at the CQ-T2 checkpoint; monorepo typecheck clean; pre-pushcheck:ci-local(incl. the D23 memory-drift guard) green on each push.Out of scope / follow-ups
Date.nowconversion (archivedAt, ledger timestamp, background job timestamps, record-decision, formatCurrentDate), thetool-stream-parserxml-id + otherrandomUUIDsites (need wider deps threading, TODO-commented), and an enforcement lint against new direct calls.OPENBUFF_EDIT_BLOCKScan flip on.This change is