feat(service-automation)!: decline to arm a region-nested call to a child that can suspend, and cancel the child run the region refusal leaves behind

.changeset/28897-region-call-suspending-subflow.md

@@ -0,0 +1,20 @@
+---
+'@objectstack/service-automation': patch
+'@objectstack/spec': patch
+---
+
+feat(service-automation)!: a flow whose region body calls a child that can suspend is registered but not armed, and the region refusal cancels the child run it leaves behind (#28897)
+
+Clause-②: yes (narrowing)
+Metadata-protocol: none
+
+**BREAKING** for a record-triggered flow with a `subflow` or `map` node inside a `loop` body, a `parallel` branch or a `try_catch` slot, whose child flow can suspend the run.
+
+- **Before.** A region body cannot carry a durable pause (ruling D on #15646). The flow parse refuses the pausing node types inside a region, but it cannot see into a `subflow` / `map` child. Such a flow armed, and the first time its child suspended, the run failed at the region boundary with the named refusal (`durable pause inside a structured region …`). The child run that node had started was left parked in the suspended-run store: a later resume ran the rest of the child for a caller that had already failed, and then failed to resume that caller.
+- **Now, the first door.** The arming gate in `activateFlowTrigger` declines such a flow, after its disabled-subflow and save-notice halves. "Can suspend" is the save-notice half's reading: the child holds a node of an always-suspending type reachable from its start, or calls, from its top-level graph, a child that can suspend. The flow still registers, is not armed, and warns once, naming the region node (the innermost container), the `subflow` / `map` node and the child's suspending node. `GET /automation/_status` reports it `bound: false` with that reason and the new `unboundCause` value `suspending_subflow_in_region` (added to `FLOW_UNBOUND_CAUSES` in `@objectstack/spec/contracts`). Whenever a flow is registered, every flow whose calls reach it is judged again, so the decline does not depend on load order, and a child republished without its suspension arms the caller again.
+- **Now, the second door.** A run that still meets the region refusal (a direct execute, or a flow with no trigger) fails exactly as before, with the same refusal. Before throwing it, the engine cancels the child run the `subflow` / `map` node parked, and every run that child is parked on, through `cancelRun`. Each cancelled run records `cancelled`, with the refusal as its reason.
+- **Not declined.** A region-nested call whose child cannot suspend, and a top-level call to a child that can (the top-level graph carries the pause). Registration, the save door and `objectstack validate` accept the same flows as before.
+- **The fix.** Move the `subflow` / `map` node the reason names onto the top-level graph and route the region's exit to it, or republish the child without the node that suspends. The flow arms on its own once the shape is gone.
+- **Who moves.** No shipped flow has the shape. HotCRM's `campaign_enrollment` calls two children from loop bodies, and neither child suspends. The showcase and dogfood calls are all top-level.
+
+<!-- adr-0087: not-required (no-migration-prescription) no metadata key is added, removed, renamed or re-shaped and the save door accepts the same flows: a flow that registered still registers and is only left unarmed by the runtime, with its own reason naming what to move, a run that failed at the region still fails there with the same refusal, and the one contract change is a new value in the runtime status cause list, which no author writes and no migration could rewrite -->

packages/services/service-automation/src/dag-traversal.ts

@@ -34,6 +34,7 @@ import {
     FlowSuspendSignal,
     isRefusalSignal,
     isSuspendSignal,
+    nestedChildRunId,
     type ExecutionLogEntry,
     type NodeExecutionResult,
     type StepLogEntry,
@@ -141,6 +142,8 @@ export interface DagTraversalHost {
     readonly persistSuspendedRun: AutomationEngine['persistSuspendedRun'];
     readonly flowChecksumOf: AutomationEngine['flowChecksumOf'];
     readonly renderFlowTerminalMessage: AutomationEngine['renderFlowTerminalMessage'];
+    readonly resolveSuspensionChain: AutomationEngine['resolveSuspensionChain'];
+    readonly cancelRun: AutomationEngine['cancelRun'];
 }
 
 /** The body of {@link AutomationEngine.executeNode}, moved unedited; that method's docblock is its contract. */
@@ -746,9 +749,9 @@ export async function traverseNext(
     }
 }
 
-/** The body of {@link AutomationEngine.runRegion}, moved unedited; that method's docblock is its contract. */
+/** The body of {@link AutomationEngine.runRegion}; that method's docblock is its contract. */
 export async function runRegion(
-    this: Pick<DagTraversalHost, 'executeNode'>,
+    this: Pick<DagTraversalHost, 'executeNode' | 'resolveSuspensionChain' | 'cancelRun' | 'logger'>,
     region: FlowRegionParsed,
     variables: Map<string, unknown>,
     context: AutomationContext,
@@ -835,12 +838,42 @@ export async function runRegion(
         if (isSuspendSignal(err)) {
             const suspended = region.nodes.find(n => n.id === err.nodeId);
             const flowName = (suspended?.config as { flowName?: unknown } | undefined)?.flowName;
-            throw refuseRegionSuspension({
+            const refusal = refuseRegionSuspension({
                 regionNodeId: grouping?.parentNodeId ?? entryId,
                 regionKind: grouping?.regionKind ?? 'region',
                 suspendedNodeId: err.nodeId,
                 ...(typeof flowName === 'string' && flowName ? { subFlowName: flowName } : {}),
             });
+            // [#28897] A `subflow` / `map` node suspends by parking the CHILD
+            // run it started and linking it through its correlation
+            // (`subflow:<childRunId>` / `map:<childRunId>`). This run fails
+            // here, so nothing will ever resume through that link: measured,
+            // the child stayed parked in the durable store, and a later
+            // resume ran the rest of the child for a caller that had already
+            // failed, then failed to resume that caller. So the child run is
+            // cancelled, with every run it is itself parked on, deepest
+            // first, through the engine's own `cancelRun`. ⛔ The refusal is
+            // thrown exactly as before: a failed cancellation is logged, never
+            // put in its place.
+            const childRunId = nestedChildRunId(err.correlation);
+            if (childRunId !== undefined) {
+                try {
+                    const chain = await this.resolveSuspensionChain(childRunId);
+                    const reason = `the run that started it failed at a region boundary: ${refusal.message}`;
+                    for (const run of [...chain].reverse()) await this.cancelRun(run.runId, reason);
+                } catch (cancelErr) {
+                    // #4632 verdict: DURABILITY, `error`, as `cancelRun`'s own
+                    // store failure: the child may still be parked.
+                    this.logger.error(
+                        `[automation] node '${err.nodeId}' suspended inside region '${refusal.regionNodeId}', and ` +
+                            `cancelling the child run '${childRunId}' it parked failed, so that run may still be ` +
+                            `parked. Cancel it with cancelRun('${childRunId}'). The failure is in this record's meta.`,
+                        undefined,
+                        describeThrownForLog(cancelErr),
+                    );
+                }
+            }
+            throw refusal;
         }
         // [#15788] The refusing `end` node's signal, converted at exactly
         // the same boundary and for the same reason: a control signal must

packages/services/service-automation/src/engine.ts

@@ -222,6 +222,7 @@ import {
     refuseCustomerAuthoredToggle,
 } from './subflow-guards.js';
 import { describeSubflowNoticeTrap, subflowNoticeTrap } from './subflow-notice-guard.js';
+import { describeRegionSuspendingCall, regionSuspendingCall } from './subflow-region-guard.js';
 // [#25222] The consumed-suspension restore's bodies: journaling a consumed
 // suspension, resolving its witnesses, and restoring, peeking at or inspecting
 // one. Each `AutomationEngine` method of the same name calls its body.
@@ -2915,6 +2916,16 @@ export class AutomationEngine implements IAutomationService {
      * registered, and dropped with the flow by {@link withdrawFlow}.
      */
     private readonly subflowNoticeDeclinedFlows = new Map<string, string>();
+    /**
+     * [#28897] Flows {@link activateFlowTrigger} declined to arm because a
+     * `subflow` / `map` node inside one of their region bodies calls a child
+     * flow that can suspend, each with the reason `/_status` and the binding
+     * audit read. A record for {@link subflowNoticeDeclinedFlows}'s reasons,
+     * kept the same way: deleted the moment the flow gets past the gate,
+     * re-asked by {@link rejudgeSubflowCallers}, dropped by
+     * {@link withdrawFlow}.
+     */
+    private readonly subflowRegionDeclinedFlows = new Map<string, string>();
     /** Connectors registered by integration plugins, keyed by connector name (ADR-0018 §Addendum). */
     private connectors = new Map<string, RegisteredConnector>();
     /** Connector provider factories keyed by provider name (ADR-0097 §2 — `openapi`/`mcp`/`rest`/…). */
@@ -3899,6 +3910,15 @@ export class AutomationEngine implements IAutomationService {
         // so the gate asks here and {@link rejudgeSubflowCallers} asks again
         // when the child arrives or changes.
         if (this.declineOntoSuspendingSubflowNotice(flowName, resolved.triggerType)) return;
+        // [#28897] The region half, a decline for the same reasons: a region
+        // body cannot carry a durable pause, so a region-nested `subflow` /
+        // `map` whose child suspends fails the run at the region boundary
+        // (`region-suspension-refusal.ts`), the first time that child
+        // suspends. That refusal stays as the second door; this is the first.
+        // Last of the halves, and {@link describeUnboundReason} reads the
+        // records in the same order, so a record an earlier half's decline
+        // left unasked is never the one reported.
+        if (this.declineOntoRegionSuspendingCall(flowName, resolved.triggerType)) return;
         const trigger = this.triggers.get(resolved.triggerType);
         if (!trigger) return;
         try {
@@ -4649,6 +4669,7 @@ export class AutomationEngine implements IAutomationService {
             policyDisabledFlows: this.policyDisabledFlows,
             subflowDeclinedFlows: this.subflowDeclinedFlows,
             subflowNoticeDeclinedFlows: this.subflowNoticeDeclinedFlows,
+            subflowRegionDeclinedFlows: this.subflowRegionDeclinedFlows,
             flowRefusals: this.flowRefusals,
             logger: this.logger,
         };
@@ -4780,6 +4801,7 @@ export class AutomationEngine implements IAutomationService {
             policyDisabledFlows: this.policyDisabledFlows,
             subflowDeclinedFlows: this.subflowDeclinedFlows,
             subflowNoticeDeclinedFlows: this.subflowNoticeDeclinedFlows,
+            subflowRegionDeclinedFlows: this.subflowRegionDeclinedFlows,
             triggers: this.triggers,
         };
         return describeUnboundReason.call(host, name, resolved);
@@ -5222,6 +5244,11 @@ export class AutomationEngine implements IAutomationService {
      * child be switched off under it. The regions are read through
      * `collectFlowGraphs`, the spec's one region walk, never a second walker.
      *
+     * [#28897] That reading is `subflowCalls` (`subflow-region-guard.ts`),
+     * which also keeps each call's node and the container holding it: the
+     * arming gate's region half ({@link declineOntoRegionSuspendingCall})
+     * reads the same calls, never a walk of its own.
+     *
      * ⛔ The historical `flow` alias for `config.flowName` is NOT read here.
      * ADR-0087 D2's conversion `flow-node-subflow-flow-alias` canonicalizes it
      * at load, so only the canonical key reaches a registered definition;
@@ -5559,6 +5586,48 @@ export class AutomationEngine implements IAutomationService {
         return true;
     }
 
+    /**
+     * [#28897] The arming gate's region half, asked by
+     * {@link activateFlowTrigger} of an enabled flow it is about to arm: does a
+     * `subflow` / `map` node inside one of its region bodies (a `loop` body, a
+     * `parallel` branch, a `try_catch` slot, at any depth) call a child flow
+     * that can suspend ({@link regionSuspendingCall}, which reads the calls
+     * through {@link subflowTargets}' reading and where the child suspends
+     * through the save-notice half's)? A region body cannot carry a durable
+     * pause, so that run would fail at the region boundary the first time the
+     * child suspends (`region-suspension-refusal.ts`). Then it is declined:
+     * recorded on {@link subflowRegionDeclinedFlows} for `/_status` and the
+     * binding audit, and said once as a `warn` naming the region node, the call
+     * node and where the child suspends.
+     *
+     * Any flow, packaged or authored here, as the save-notice half: the failed
+     * run is the same. ⛔ A decline, never a refusal, for that half's reasons.
+     *
+     * #4632 verdict: FUNCTIONAL, `warn`, as the other halves. Said once per
+     * decline reason.
+     *
+     * @returns `true` when the flow is declined and must not be armed.
+     */
+    private declineOntoRegionSuspendingCall(flowName: string, triggerType: string): boolean {
+        const trap = regionSuspendingCall(this.flows, flowName);
+        if (!trap) {
+            this.subflowRegionDeclinedFlows.delete(flowName);
+            return false;
+        }
+        const why = describeRegionSuspendingCall(trap);
+        const reason =
+            `not armed — ${why}, and it is armed; it is also armed once '${trap.child}' is republished without a ` +
+            `node that suspends`;
+        const previous = this.subflowRegionDeclinedFlows.get(flowName);
+        this.subflowRegionDeclinedFlows.set(flowName, reason);
+        if (previous === reason) return true;
+        this.logger.warn(
+            `Flow '${flowName}' is registered but NOT armed on trigger '${triggerType}' — ${why}. It is armed the ` +
+                `moment '${trap.callNode.id}' moves, or '${trap.child}' is republished without a node that suspends.`,
+        );
+        return true;
+    }
+
     /**
      * [#20725, ADR-0126 §7.3] `subflow` just changed state — (re)registered,
      * switched on or off, or read off the ledger at boot — so ask the arming
@@ -5582,13 +5651,17 @@ export class AutomationEngine implements IAutomationService {
      * change two calls away moves the answer. A caller that half declined is
      * re-offered, and an armed caller it would now decline is unbound and
      * offered again, both through the one gate.
+     *
+     * [#28897] The region half reads the same children through the same
+     * calls, so the same callers are re-asked for it, the same way.
      */
     private rejudgeSubflowCallers(subflow: string): void {
         // The body lives in ./subflow-guards.ts (#25222).
         const host: ThisParameterType<typeof rejudgeSubflowCallers> = {
             boundFlowTriggers: this.boundFlowTriggers,
             subflowDeclinedFlows: this.subflowDeclinedFlows,
             subflowNoticeDeclinedFlows: this.subflowNoticeDeclinedFlows,
+            subflowRegionDeclinedFlows: this.subflowRegionDeclinedFlows,
             flows: this.flows,
             subflowTargets: (...a) => this.subflowTargets(...a),
             packagedSubflowCallers: (...a) => this.packagedSubflowCallers(...a),
@@ -9267,6 +9340,12 @@ export class AutomationEngine implements IAutomationService {
      * failure, ran its catch handler, and the run reported success over a sweep
      * that had processed nothing. The container executors test for the named
      * type and re-throw, so no region can contain it.
+     *
+     * [#28897] When the suspending node is a `subflow` / `map`, the child run
+     * it parked (and any run that child is parked on) is cancelled before the
+     * refusal is thrown, so no run is left parked under a caller that failed.
+     * The arming gate declines such a caller first
+     * ({@link declineOntoRegionSuspendingCall}); this is the second door.
      */
     runRegion(
         region: FlowRegionParsed,
@@ -9278,6 +9357,9 @@ export class AutomationEngine implements IAutomationService {
         // The body lives in ./dag-traversal.ts (#25222).
         const host: ThisParameterType<typeof runRegion> = {
             executeNode: (...a) => this.executeNode(...a),
+            resolveSuspensionChain: (...a) => this.resolveSuspensionChain(...a),
+            cancelRun: (...a) => this.cancelRun(...a),
+            logger: this.logger,
         };
         return runRegion.call(host, region, variables, context, grouping, partialSteps);
     }