Skip to content

fix(service-automation): a failing parallel keeps every branch's steps in the run log - #23970

Merged
objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-23928-parallel-partial-steps
Oct 16, 2026
Merged

objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-23928-parallel-partial-steps

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #23928

Clause-②: no

What an operator now sees on a failed parallel run

The run's step log lists every branch's steps under the block's own failure step. Each is tagged parentNodeId, regionKind: 'parallel-branch' and its branch index: a branch that completed, with its success step, and the failing branch's steps up to and including the one that failed, with its error. The run summary counts what those steps did. On the card's fan-out (branch 0 an inbox notify, branch 1 an http POST failing with fetch failed) the run now reads selected: 1, acted: 1, failed: 2. Before, it read acted: 0 beside a notification that had already been delivered, and acted: 0 on a failed run reads as "nothing happened, safe to re-run".

The change: builtin/parallel-node.ts, execute

  • Each branch runs engine.runRegion(...) with its own partialSteps sink. A failing branch's tagged region steps land in its sink before runRegion rethrows, and a completed branch returns its steps instead. So each step is collected exactly once, in branch declaration order, the order the success path already used.
  • Each branch's promise is wrapped so it never rejects, and its rejection is recorded in settle order. The block therefore reports its failure once every branch has settled, so a sibling that finishes after another branch failed keeps its step too. The failure reported is the first one in settle order, which is the one Promise.all rejected with before, so the error text is unchanged.
  • A branch failure is still returned as { success: false, error }, now with childSteps. A region-suspension refusal is still re-thrown unchanged, now carrying the branch steps through attachPartialSteps.

No engine.ts change. The engine already folds a returned failure's childSteps right behind the container's own step (engine.ts:11812, the arm try_catch uses), and a thrown error's partial steps the same way (engine.ts:11689).

Why return for a branch failure, and throw only for the refusal

The deciding constraint is partial-steps.ts:40-47: a step-record fix must not move accept/reject behaviour, and the returned and thrown failure paths route differently. loop used to throw, so it kept throwing and carried its steps on the error (loop-node.ts:151-159). parallel already returned its branch failure (parallel-node.ts:106 at base). The same rule points the other way here: keep returning, and put childSteps on the result. A switch to a throw would move the block's step code from NODE_FAILURE to EXECUTION_ERROR, change $error's shape and rewrite the run's error text, the same costs partial-steps.ts names. The region-suspension refusal was already thrown (parallel-node.ts:104 at base) so that a fault edge cannot route it. It keeps the throw and gets attachPartialSteps, which leaves the error's identity, message and refusal marking untouched. That is loop's mechanism, on the one path where parallel already threw.

Tests

  • Red at base, 10 of 10. parallel-node.test.ts (8 new pins) and the new parallel-failed-branch-steps.integration.test.ts (2 pins), run on commit 836fb42 (pins only, source at base 7277c07): Tests 10 failed | 7 passed (17). Readings: expected [] to deeply equal [ 'notify_owner@par/0:success', ... ] (no branch steps), expected 0 to be greater than or equal to 1 (acted), expected [ 'par', 'recover' ] to deeply equal [ 'par', 'notify_owner', 'x', ... ] (the fault edge routes at base but the branch steps are missing), and expected [] to deeply equal [ 'slow' ] (at base the failed run returned before the slow sibling had even delivered).
  • Green with the fix: the same two files give Tests 17 passed (17). The full package suite (pnpm --filter @objectstack/service-automation test) gives Test Files 188 passed (188), Tests 2519 passed (2519). pnpm --filter @objectstack/service-automation typecheck exits 0: tsc --noEmit, then check:test-typecheck: OK ... 0 file(s) / 0 error(s).
  • Real engine: parallel-failed-branch-steps.integration.test.ts runs the real notify node on a real inline MessagingService with a recording inbox channel, and the real http node with the global fetch rejecting TypeError('fetch failed'), the card's failure. It asserts the step log carries notify_owner success on branch 0 and slack_post failure with its error on branch 1, and that the live result and the persisted run row both read selected: 1, acted: 1, unmeasured: 1 (the failed POST) and failed: 2. The other pins cover: the failing branch's steps up to and including its failure, a sibling that settles after the failure, the reverse control (no effect, acted: 0), a fault edge routing as before with the branch steps between the block and its handler, the refused pause (failed stays 1), and loop { parallel } folding each step exactly once.
  • Ablation through scripts/ablation-replace.mjs (WRAP mode: anchor hit x1 to x0, blob changed, restore proven blob == HEAD 47a3dfb296d0 and git diff HEAD empty, each leg):
    • Leg 1 drops childSteps from the returned failure (blob to 861c8de3a97f): Tests 9 failed | 8 passed (17). Every returned-failure pin goes red, including both integration pins; the pause pin stays green because it rides the other limb.
    • Leg 2 reverts the refusal rethrow to a bare throw err; (blob to d99181112ccd), run with region-durable-suspension-refusal.test.ts too: Tests 1 failed | 24 passed (25). Only the pause pin goes red (expected [] to include 'notify_owner@par/0:success'); the existing region-refusal suite stays green both ways.
    • The join-before-failure limb has no single-line ablation. Its red reading is the base run above (expected [] to deeply equal [ 'slow' ]).

Gates

Derived with node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at 4c26145 (this head, after merging origin/main 161aee1, which touched only packages/cli and driver-turso): 63 commands. 62 exited 0, exit captured before any pipe. pnpm check:dual-build-cjs-loads exited 3, PREREQUISITE NOT MET — this gate reads built output, and some package has no dist/. That means it measured nothing here (it needs the whole-workspace build), so it is NOT MEASURED locally and left to CI's Lint & Repo Gates. --ran reconciliation: 63 derived, 62 run, 0 NOT-MEASURED, 1 UNRUN (that one), plus the package test and typecheck recorded outside the derivation.

Clause-②: no: the diff changes no exported declaration. registerParallelNode's signature is untouched and everything else is inside execute. The built declaration closure was not diffed against base.

Acceptance notes

  • Timing: a failed block now reports its failure (and a fault handler starts) once every branch has settled, not at the first rejection. A slow sibling therefore delays the handler, bounded by its nodes' own timeouts. This is the documented implicit join. Before, a sibling could finish its effect after the run had already recorded the failure.
  • Stale comments in engine.ts, which this PR does not touch (another PR holds it): the returned-failure fold says try_catch "is still the only" producer of childSteps on a failing result, and runRegion's doc says callers that pass no sink are "loop, parallel". Both now pass one. Carrier: none.
  • Observation, preserved on purpose: when one branch fails at runtime FIRST and another later raises a region-suspension refusal, the block returns the first (routable) failure, exactly as Promise.all chose before. Only the first failure in settle order is acted on, so routing does not move. Read from source and not measured through a public door, so not filed. Carrier: none.
  • The Studio Runs panel (objectui) was not driven. The card measured it rendering no branch rows under a failed parallel. Whether it now renders them is not measured here.

Generated by Claude Code

…nch's steps

Red at base: a parallel whose branch fails returns no childSteps, so the
run log drops the delivered sibling's step and the failing branch's own
step, and the run summary reports acted 0.

Claude-Session: https://claude.ai/code/session_018VqJPGA1NDNSCZsnecsChP
Co-authored-by: Claude <noreply@anthropic.com>
A parallel whose branch fails now returns its failure with childSteps:
the steps of the branches that completed and the failing branch's own
steps up to and including the one that failed, each tagged with the
block, regionKind 'parallel-branch' and its branch index. Each branch
runs with its own runRegion partialSteps sink, and the block reports
its failure once every branch has settled, so a sibling that finishes
after the failure keeps its step too.

The failure is still RETURNED, so a fault edge on the block routes
exactly as before, with the same step code, error text and $error; the
engine already folds a returned failure's childSteps behind the block's
own step. A region-suspension refusal is still re-thrown unchanged, now
carrying the branch steps through attachPartialSteps, as loop does.

Adds a real-engine pin: the real notify node on an inline messaging
service and the real http node failing with fetch failed, showing the
run log keeps both branch steps and the summary counts the delivery.

Claude-Session: https://claude.ai/code/session_018VqJPGA1NDNSCZsnecsChP
Co-authored-by: Claude <noreply@anthropic.com>
…ing its branch steps

Claude-Session: https://claude.ai/code/session_018VqJPGA1NDNSCZsnecsChP
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling labels Oct 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

1 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 6 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 6d1660dc3b163c30c3a02d34de5cfc219940d958 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 6cacbb2209b2999eb91f6b26a2f3c739e415b1c0 — the merge of head 4c26145a936ebdddca382e625844318929dc2b67 into base 6d1660dc3b163c30c3a02d34de5cfc219940d958, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 6cacbb2209b2999eb91f6b26a2f3c739e415b1c0 && git checkout 6cacbb2209b2999eb91f6b26a2f3c739e415b1c0
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 6d1660dc3b163c30c3a02d34de5cfc219940d958 4c26145a936ebdddca382e625844318929dc2b67 && git checkout -B drift-repro 6d1660dc3b163c30c3a02d34de5cfc219940d958 && git merge --no-ff 4c26145a936ebdddca382e625844318929dc2b67

node scripts/docs-audit/affected-docs.mjs --json 6d1660dc3b163c30c3a02d34de5cfc219940d958

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对 · ACCEPT · domain:services seat 2 · session_018VqJPGA1NDNSCZsnecsChP · 2026-10-16T15:13Z

Reviewed against GitHub at head 4c26145a93, from the diff and the dev's report on #23928.

  • Form:
    • Draft, base main. The first line is Fixes #23928, then Clause-②: no. No other card's number sits beside a closing keyword.
    • 4 files, all on claim 6217245767's surface: parallel-node.ts, its unit test, a new real-engine integration test, and one patch changeset.
    • check-governed-merges.mjs --pr 23970: NOT governed, 578 changed lines. mergeable_state: clean.
  • The fix, read in the diff (builtin/parallel-node.ts):
    • Each branch runs runRegion with its own partialSteps sink, inside a wrapper that never rejects. So the block reports its failure only after every branch has settled.
    • The failure carries childSteps: every branch's steps in declaration order. That covers the completed siblings, and the failing branch's steps up to and including the failing step, each tagged parentNodeId / regionKind: 'parallel-branch' / branch.
    • Routing unchanged. A branch failure is still returned, so a fault edge on the parallel routes it with the same step code, error text (the first rejection in settle order, which is what Promise.all rejected with) and $error.
    • The region-suspension refusal is still thrown, now carrying the steps through attachPartialSteps, which is the loop precedent (fix(service-automation): keep a dying loop's completed body steps in the run log #14185).
    • No engine.ts change is needed. The engine's returned-failure fold is generic: engine.ts ~:11812 pushes any failing result's childSteps right behind the block's own step. Only its comment still says try_catch is the only producer. That is noted as prose drift, and engine.ts is held by PR feat(spec)!: reserve the retired callout node types http_request, http_call and webhook; flow-node-http-callout-rename leaves the load path (F5-http of #23543) #23954.
  • Measured (the dev's runs):
    • Red at base, on the pins alone: 10 failed. A delivered branch-0 notify left no step and acted was 0; at base the block returned before a slow sibling delivered.
    • Green with the fix: 17 of 17. The package suite passes 2,519 of 2,519, typecheck exit 0.
    • The card's shape on a real engine (a real notify, and an http branch failing fetch failed): the step log holds notify_owner success on branch 0 and slack_post failure on branch 1. The summary reads selected 1, acted 1, failed 2, where it read acted 0 before.
    • Ablations: dropping childSteps from the returned failure turned 9 tests red. Reverting the refusal's attachPartialSteps turned only the pause pin red. Both restores were proven by blob.
  • Gates: 63 derived. 62 exit 0. 1 not measured locally, check:dual-build-cjs-loads, which needs every dist and is green in CI.
  • CI: 40 check-runs, latest per name: 34 success, 0 failure. 6 skipped, all on the roster: Build Docs, Console Pin Gate, the Dogfood Regression Gate matrix, Dogfood Verify CLI, Packed-tarball smoke, Temporal Conformance.
  • Accepted:
    • The behaviour change the changeset states: a failed block now waits for its slower siblings before its fault handler starts. At base, those siblings still ran to completion, only off the record. The join makes their effects visible and counted.
    • Clause-②: no by reasoning rather than a declaration diff. No exported signature moves; every change sits inside the executor's execute.
  • Out of scope, noted and not filed:

ACCEPT. This act marks the PR ready and enables auto-merge through the fleet relay. It lands through the merge queue.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

2 participants