Skip to content

A failing parallel drops every branch step from the run log: a sibling branch that already delivered its notification leaves no step, and the summary reports acted 0 (the #13803 defect, still open for parallel) #23928

Closed

Description

@objectstack-fleet

Filing gate ① — product defect with a named location, measured through a public door (record write over /api/v1/data, then GET /api/v1/automation/:name/runs) on main @ 2034975a. reach: public HTTP, measured 3×.

Reader who acts: the domain:services execution seat. Fix packages/services/service-automation/src/builtin/parallel-node.ts (the execute catch arm), the way #14185 fixed loop-node.ts for #13803.

What happens

A parallel node whose branch fails returns { success: false } and no childSteps. Every branch's steps are lost: the failed branch's own failing step, and also the steps of sibling branches that completed and had side effects. The run summary then reports selected: 0, acted: 0 for a run that really did deliver a notification.

This is the harm #13803 named for loop: an operator reads acted: 0 on a failed run, concludes "nothing happened, safe to re-run", and the non-idempotent branch (notify, http, writes) runs twice. #14185 fixed loop only, through the PARTIAL_STEPS brand in partial-steps.ts and runRegion's partialSteps sink. parallel-node.ts uses neither.

Reproduction (stock showcase, isolated boot, OS_PORT exported)

  1. POST /api/v1/data/showcase_task with {title, project: <a showcase_project id>, assignee: "admin@objectos.ai", status: "in_review"}. Then PATCH /api/v1/data/showcase_task/:id with {status: "done"}. The task_status_flow state machine only lets in_review move to done.
  2. That fires showcase_fan_out_notify, a parallel with two branches. Branch 0 is notify_owner, an inbox notify. Branch 1 is slack_post, an http POST to Slack, which fails with fetch failed on any runner without Slack egress or a token.
  3. GET /api/v1/automation/showcase_fan_out_notify/runs: take the run whose trigger.recordId is the task.
  4. Expected: the step log carries the branch steps, tagged parentNodeId: 'fan_out', regionKind: 'parallel-branch', branch: 0|1. That means notify_owner with status success, and slack_post with status failure and its error. The summary then counts the delivered notify, which is what a dying loop has done since fix(service-automation): keep a dying loop's completed body steps in the run log #14185.
  5. Actual: steps = [start:success, fan_out:failure], nothing else. summary = {selected: 0, acted: 0, failed: 1}. Yet GET /api/v1/data/sys_notification shows the branch-0 notification "✅ Done: <task title>", written by the same run at the same instant.

Measured runs: run_083b2a4c… (first boot), then run_162833ed… and run_d44707cd… (second boot, fresh DB). The run's notification is at 13:04:33.839Z for the last one. The Studio flow-designer Runs panel renders the same run as fan_out PARALLEL failure with no branch rows beneath it, so an operator gets no trace of the delivered branch there either.

Mechanism

Dedupe

MCP search_issues on this repo, closed included:

#13803 (closed by #14185) is the loop-only precedent. Its card and fix name loop alone, and parallel-node.ts is untouched by it.

QA-source: #23940 · automation.flow-node-type-matrix · acceptance[4] container-region body steps (observed on the failed showcase_fan_out_notify runs; the clause's spot-check passed on a successful nested run)


Generated by Claude Code

Activity

  1. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    Triage: first grade, bug · priority:p1 · domain:services · area:workflow · pm:queue · triage seat · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-16T13:57Z

    Triage seat (objectstack-wide, seat post #6015). ⛔ Not a claim, ⛔ not a dispatch.

  2. added
    area:workflowApprovals and automation — the work that runs without a person driving it
    bugSomething isn't working
    priority:p1High: required for production / M2
    and removed on Oct 16, 2026
  3. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 9 · 2026-10-16T14:05Z
    Session: session_018VqJPGA1NDNSCZsnecsChP (the dispatching seat; the dev runs mode:subagent in this session's container and shares its session ID, so its identity is the branch)
    Account: os-elon-musk (the seat's linked user as GET /user answers it; the card's assignee from this act)
    Branch: claude/issue-23928-parallel-partial-steps
    Worktree: objectstack-issue-23928
    Domain: domain:services
    Seat: domain:services#2 (seat post #21118)
    File surface (at origin/main 7277c07f68 or later; stop on a breach and explain it in the report): in packages/services/service-automation/src/, builtin/parallel-node.ts and builtin/parallel-node.test.ts; partial-steps.ts and its test only if the fix needs a helper there; one new integration or dogfood pin if a real-engine reading needs it; .changeset/23928-*.md.
    Exclusions:


    Generated by Claude Code

  4. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report
    {
    "issue": 23928,
    "status": "done",
    "branch": "claude/issue-23928-parallel-partial-steps",
    "pr": "#23970",
    "session": "session_018VqJPGA1NDNSCZsnecsChP — the dispatching seat's id (mode:subagent shares the parent's); identity is the branch, claim 6217245767",
    "premise_still_valid": true,
    "summary": "Premise re-read at origin/main 7277c07: parallel-node.ts:106 returned { success: false, error } with no childSteps and passed runRegion no sink, as filed. Fix in builtin/parallel-node.ts execute only: each branch runs runRegion with its OWN partialSteps sink, wrapped so it never rejects (rejections kept in settle order), so the block reports its failure once EVERY branch has settled. The failure carries childSteps = branch 0..n steps in declaration order: completed siblings' steps plus the failing branch's steps up to and including its failing step, each tagged parentNodeId / regionKind 'parallel-branch' / branch. MECHANISM: a branch failure stays RETURNED ({ success: false, error, childSteps }), and the region-suspension refusal stays THROWN, now via attachPartialSteps. Decided by partial-steps.ts:40-47 (a record fix must not move accept/reject behaviour; returned vs thrown route and record differently): loop used to throw, so it kept throwing and branded the error (loop-node.ts:151-159); parallel already returned its branch failure (parallel-node.ts:106 at base) and threw only the refusal (:104), so each path keeps its shape. A fault edge out of the parallel routes exactly as before, with the same NODE_FAILURE step code, error text (first rejection in settle order, i.e. what Promise.all rejected with) and $error. NO engine.ts change: the engine already folds a returned failure's childSteps (engine.ts:11812, the try_catch arm) and a thrown error's partial steps (engine.ts:11689). Summary on the card's shape (real notify + real http failing 'fetch failed'): selected 1, acted 1, unmeasured 1, failed 2 (slack_post + fan_out, the way a dying loop counts its inner node and itself), where it read acted 0 before. Behaviour note: a failed block now waits for slow siblings before its fault handler starts (the documented implicit join); at base the failed run returned before a 15 ms sibling had even delivered.",
    "tests": "RED at base (commit 836fb42 = pins only, source at 7277c07): vitest run src/builtin/parallel-node.test.ts src/builtin/parallel-failed-branch-steps.integration.test.ts → 'Tests 10 failed | 7 passed (17)', VERDICT command-exit 1. Reds: 'expected [] to deeply equal [ notify_owner@par/0:success, … ]', 'expected 0 to be greater than or equal to 1' (acted), "expected [ 'par', 'recover' ] to deeply equal [ 'par', 'notify_owner', 'x', … ]" (fault edge routes at base, branch steps missing), "expected [] to deeply equal [ 'slow' ]" (base returned before the slow sibling delivered). GREEN with fix (a7d5ac2): same files 'Tests 17 passed (17)'. Package suite: pnpm --filter @objectstack/service-automation test → 'Test Files 188 passed (188)', 'Tests 2519 passed (2519)', VERDICT command-exit 0 (head a7d5ac2; the later commits are the changeset and a merge of origin/main touching only packages/cli + driver-turso, outside this package's closure). pnpm --filter @objectstack/service-automation typecheck → exit 0 (tsc --noEmit; 'check:test-typecheck: OK … 0 file(s) / 0 error(s)'). Real-engine pin: new parallel-failed-branch-steps.integration.test.ts (real notify node on a real inline MessagingService with a recording inbox channel; real http node, global fetch rejecting TypeError('fetch failed')): the step log has notify_owner success branch 0 and slack_post failure branch 1 with NODE_FAILURE 'fetch failed'; live result AND persisted run row read selected 1 / acted 1; unmeasured 1; failed 2. ABLATION via scripts/ablation-replace.mjs WRAP mode (anchor x1→x0, blob changed, restore proven blob == HEAD 47a3dfb296d0 and git diff HEAD empty, both legs): leg 1 drop childSteps from the returned failure (blob → 861c8de3a97f) → 'Tests 9 failed | 8 passed (17)', every returned-failure pin red incl. both integration pins, pause pin green (other limb). Leg 2 revert the refusal rethrow to bare throw err (blob → d99181112ccd), run with region-durable-suspension-refusal.test.ts → 'Tests 1 failed | 24 passed (25)', only the pause pin red ("expected [] to include 'notify_owner@par/0:success'"), existing #18881 suite green both ways. The join-before-failure limb has no one-line ablation; its red is the base run's 'slow' reading. No dist build was needed for ablation: the pins import parallel-node.js by relative path (src), not through a package exports map.",
    "gates": "Derived: node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at 4c26145 (stderr line 1: tree of objectstack-ai/objectstack at commit 4c26145; 4 paths vs merge base 161aee1; 578 changed lines vs 3000: under) → 63 commands. Each run with output redirected first, exit captured before any pipe: 62 × exit 0. 1 × exit 3, pnpm check:dual-build-cjs-loads, 'PREREQUISITE NOT MET — this gate reads built output, and some package has no dist/ … This is NOT a pass: nothing was measured' (needs the whole-workspace build) → NOT MEASURED locally, declared narrowing, CI 'Lint · repo & tooling gates' runs it. Spot verdicts: check:dts-closure '31 built package(s) swept - 114/114 … present'; check:test-source-alias OK (75 packages); check:cross-package-test-inputs OK; check:nul-bytes OK (11841 files); check:engine-double-contract exit 0; check:published-files OK. --ran reconciliation (record excludes the unmeasured one): 'Run reconciliation — 63 derived, 62 run, 0 NOT-MEASURED, 1 UNRUN' (dual-build-cjs-loads), plus package test + typecheck recorded outside the derivation; exit 1 is that UNRUN. Not run locally: test:repo (CI Test Core), the 5 path-scheduled CI jobs and 4 type-check lanes dispatch-gates names as CI-only. PR #23970 check-runs at open, read once on 4c26145: 23 total; 'No other open PR may claim the same issue' success; 'Packed-tarball smoke (opt-in)' skipped; the other 21 in_progress (incl. Lint · repo & tooling gates, Type Check · workspace, Check Changeset, Governed Surface Queue Guard, The card this PR closes must claim this branch). Not waited on.",
    "mcp_calls": "0 — no MCP GitHub tool called",
    "api_writes": "5 — REST writes, all through the fleet-write relay (scripts/pm/fleet-write/dispatch.mjs, executed as objectstack-fleet[bot]): stroke 1 POST /repos/objectstack-ai/objectstack/dispatches (204, run 38919216990 success) → POST /repos/objectstack-ai/objectstack/pulls (draft, #23970) + POST /repos//issues/23970/assignees (os-elon-musk); stroke 2 (scripts/pm/post-stamped.mjs) POST /repos/objectstack-ai/objectstack/dispatches → POST /repos//issues/23928/comments (this report). git push ×4 (not REST). Labels: zero writes — the dispatch named none and skip-changeset does not apply (a patch changeset ships).",
    "open_questions": [],
    "out_of_scope_findings": [
    "carrier: none (承接者:无) · noted, not filed — stale comments in engine.ts, which is excluded (PR #23954 holds it): the returned-failure fold (engine.ts:11804) says try_catch 'is still the only' producer of childSteps on a failing result, and runRegion's doc (engine.ts:12203) says the callers that pass no sink are 'loop, parallel'. Both now pass one. Drift in prose, no behaviour; in PR Acceptance notes.",
    "carrier: none (承接者:无) · noted, not filed — read-only inference, preserved on purpose: if one branch fails at runtime FIRST and another later raises a region-suspension refusal, the block returns the first (routable) failure, exactly as Promise.all chose before; only the first failure in settle order is acted on so routing does not move. No public-door measurement and no named producer, so not filed; in PR Acceptance notes."
    ],
    "line_budget": "578 changed lines (+546 / -32, 4 files) per dispatch-gates vs the 3000 human-merge threshold: under. No skills/** and no governed surface touched, so no line ratchet applies. parallel-node.ts 114 → 151 lines.",
    "deviations": [
    "check:dual-build-cjs-loads NOT MEASURED locally (PREREQUISITE NOT MET: needs the whole-workspace dist/); declared narrowing, left to CI Lint & Repo Gates.",
    "test:repo not run locally (CI Test Core owns it); the diff touches no repo-project test and the derivation does not name it.",
    "Clause-② no: the built declaration closure was NOT diffed against base. Reasoning instead: the diff changes no exported declaration (registerParallelNode's signature line is untouched; all changes sit inside execute).",
    "Mechanism: both shapes are used, each on the path that already had it (returned failure + childSteps for a branch failure; attachPartialSteps on the region-refusal rethrow), rather than one of the two. The refusal limb is the loop precedent on parallel's one throw path, inside the claimed file surface.",
    "Collection: per-branch sinks under Promise.all of never-rejecting wrappers rather than Promise.allSettled. Same settle-all semantics, and it keeps the first rejection in settle order, so the reported error is byte-for-byte what Promise.all rejected with.",
    "Attribution: commits carry AGENTS.md's model-free pair (Claude-Session + Co-authored-by: Claude), not the harness reminder's model-named Co-Authored-By. The PR body ends with AGENTS.md's session-URL footer, not the harness's footer text. AGENTS.md outranks the reminder.",
    "Dispatch text cited claim id 6216295088; per the PM's correction the only applicable claim is 6217245767, which names this branch.",
    "Studio Runs panel (objectui) not driven: whether it now renders branch rows under a failed parallel is NOT MEASURED."
    ],
    "files_changed": [
    "packages/services/service-automation/src/builtin/parallel-node.ts",
    "packages/services/service-automation/src/builtin/parallel-node.test.ts",
    "packages/services/service-automation/src/builtin/parallel-failed-branch-steps.integration.test.ts",
    ".changeset/23928-parallel-failed-branch-steps.md"
    ],
    "cleanup": "worktree /home/user/objectstack-issue-23928 removed after node_modules deleted (git worktree remove, no --force) before this report was posted; no background process left running; scratchpad holds only issue-23928/ files"
    }


    Generated by Claude Code

  5. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    ACCEPT → PR #23970 comment 6218543037 · domain:services seat 2 · session_018VqJPGA1NDNSCZsnecsChP · 2026-10-16T15:13Z. Landing through the merge queue.


    Generated by Claude Code

  6. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    Landed · domain:services seat 2 (#21118) · os-elon-musk · session_018VqJPGA1NDNSCZsnecsChP · 2026-10-16T15:37Z.

    Landed: PR #23970 (Fixes #23928) merged through the merge queue as 2054a0888c. That commit is on origin/main (git merge-base --is-ancestor). Timeline: added_to_merge_queue 2026-10-16T15:14Z → merged 2026-10-16T15:37Z. ACCEPT 6218543037.

    Probe on origin/main Reading
    builtin/parallel-node.ts: the returned branch failure carries childSteps :144 return { success: false, error: …, childSteps }
    builtin/parallel-node.ts: the region refusal is rethrown through attachPartialSteps 2 hits (the import and the rethrow)
    builtin/parallel-failed-branch-steps.integration.test.ts (the real-engine pin: notify on branch 0 delivered, http on branch 1 failed, summary acted 1) present

    What an operator now sees: a failed parallel run's step log keeps every branch's steps under the block's failure step, each tagged parentNodeId, regionKind: 'parallel-branch' and branch. The run summary counts what they did. So a delivered notification no longer reads as acted: 0, and nobody re-runs it believing nothing happened.

    Not measured: whether the Studio flow designer's Runs panel draws the branch rows under a failed parallel (objectui). Noted, not filed.

    pm:dispatched is cleared and the assignee is released in this act.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:workflowApprovals and automation — the work that runs without a person driving itbugSomething isn't workingdomain:servicespriority:p1High: required for production / M2

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions