feat(runner): delegate whole workflow to session-capable environment via exec_session (REQ-052) - #35
Open
Shooksie wants to merge 6 commits into
Open
feat(runner): delegate whole workflow to session-capable environment via exec_session (REQ-052)#35Shooksie wants to merge 6 commits into
Shooksie wants to merge 6 commits into
Conversation
…via exec_session (REQ-052) When a workflow's resolved `environment` is a FULL standalone animus that advertises `environment/exec_session`, hand it the ENTIRE workflow in ONE `EnvironmentClient::exec_session` call instead of running phases locally or routing them per-phase (REQUIREMENT-048). The node runs every phase on its own animus and streams `environment/journal` events home; the runner forwards them through its `WorkflowEventEmitter` and drives the run to terminal from the session's terminal `ExecSessionResponse`. New `src/workflow_session.rs`: - capabilities_advertise_exec_session / environment_is_session_capable — detect a session-capable environment from its plugin manifest `capabilities` list. - map_journal_event_kind / session_status_to_workflow_status — pure, unit-tested mappings from node journal events + terminal status to the home runtime types. - delegate_workflow_via_session — prepare bare node, exec_session with a journal-forwarding closure, teardown, synthesize WorkflowExecuteInternalResult. Branch point added in execute_workflow_with_hub before the REQUIREMENT-048 per-phase environment resolution; only full runs delegate (a `--phase` run keeps the per-phase/local path). Non-session-capable / unset environments fall through UNCHANGED. Gated behind the `remote-animus-session` cargo feature (default OFF): the parent-side exec_session / EnvironmentJournalEvent surface (orchestrator-core REQ-052, commit bcce5012, one commit past the pinned rev d03cd174) and the environment/exec_session protocol types (animus-environment-protocol, not in the pinned tag v0.7.0-rc.6) are not in the current pins. With the feature off the module is inert; the pure detection/mapping logic compiles + is unit-tested unconditionally. needs node/CI build+test — not built locally per policy.
…ion by default - all animus-protocol crates v0.7.0-rc.6 -> v0.7.0-rc.8 (rc.8 = rc.6 + the environment/exec_session commit only — clean, minimal bump) - orchestrator-* animus-cli rev d03cd174 -> 748f684d (env-protocol repinned to rc.8, local exec_session [patch] dropped) - default = [remote-animus-session]: flip the whole-workflow exec_session delegation ON. Needs node/CI build+test (per ao-cli policy).
…on) ON Validated green on GH Actions (all 3 targets). Pins: animus-protocol v0.7.0-rc.8, orchestrator-* animus-cli rev 748f684d.
…on delegation (v0.4.47) REQ-052 exact-once: when the parent runner delegates a whole workflow via exec_session and it completes, delegate_workflow_via_session emitted a terminal journal event but never drove the parent's persisted workflow state machine to terminal. The journal_runs row stayed status=running, so the daemon's journal-resume sweep (resumable_orphans_for_redispatch, past the 90s grace) re-dispatched it as a resumable orphan until a re-dispatch happened to terminalize it -- ~3 runs per dispatch instead of exactly 1. Thread the ServiceHub into delegate_workflow_via_session and, after the session returns, drive the parent state machine terminal best-effort: bounded complete_current_phase_with_decision loop for Completed, mark_completed_failed for Failed/Escalated, cancel for Cancelled. Pure state-machine transitions (no agents/post-success); the terminal event + synthesized result are unchanged.
Whole-workflow exec_session delegation forwarded only lifecycle journal events; the agent-run transcript (output_chunk/tool_call/...) was dropped by map_journal_event_kind and never reached the parent log store, so the portal showed 'No output captured for this run' for remote runs. on_journal now mirrors transcript events into the PARENT run dir first (before the lifecycle map), re-keying the node run id onto the parent workflow id (rekey_transcript_run_id) so the portal's /logs groups them, and persists via ipc::persist_run_event -> the daemon log_storage supervisor ships them to the store the portal reads. Lifecycle stays journal-only (reaches the parent via the upstream proxy). Paired with animus-env-transport bridge change (streams journal/event). Bump v0.4.47 -> v0.4.48.
Contributor
Author
|
Added commit for TASK-722 (in-node agent transcript relay), bumping v0.4.47 → v0.4.48.
Paired with the node-side relay in |
…(REQ-052 slice) (#37) Co-authored-by: animus-launchapp-gitprovider[bot] <4147602+animus-launchapp-gitprovider[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Make a workflow whose
environmentis a session-capable plugin delegate the whole workflow to that environment's own animus via a singleEnvironmentClient::exec_session(REQ-052), instead of running phases locally or routing them per-phase.When the resolved environment advertises
environment/exec_session, the runner prepares a bare node, callsexec_session(subject_id, workflow_ref, dispatch_input, on_journal)once, forwards each streamedenvironment/journalevent home through itsWorkflowEventEmitter, tears the node down, and drives the run to terminal from the session's terminalExecSessionResponse. It executes no phase itself and prepares no per-phase held node.Investigation (answers to the design questions)
1. How does the runner currently handle a workflow with
environmentset?(a) per-phase routing — not local, not nothing. This is REQUIREMENT-048, already on
main.execute_workflow_with_hub(src/workflow_execute.rs) resolves the environment once per run just before the phase loop:src/workflow_execute.rs~L350:BrokeredEnvironment::acquire_from_env(...)(daemon owns one node per run over a private socket) takes precedence; elsePreparedEnvironment::prepare_off_runtime(...)(runner owns one bare node) viaphase_environment::resolve_workflow_environment(...);held_environment: &dyn HeldEnvironment, threaded into everyrun_workflow_phase(PhaseRunParams { held_environment, .. })call (~L458, ~L687).Each phase's harness command is then run inside that held node via
HeldEnvironment::exec_session→EnvironmentClient::exec_stream(src/phase_environment.rsL408–423, L891). Note the name collision: the existingHeldEnvironment::exec_sessionis a per-phase harness exec, not the REQ-052 whole-workflowEnvironmentClient::exec_sessionthis PR adds. In REQ-048 the home runner still drives the phase loop; the node is a shared workspace/sandbox, not a standalone animus.2. Where is the branch point, and was exec_session already wired?
Branch point: in
execute_workflow_with_hub, immediately before the REQ-048 resolution block (right afterworkflow_refis resolved). Inserted atsrc/workflow_execute.rsL326–367 — if the resolved environment is session-capable, delegate andreturn; otherwise fall through to the unchanged REQ-048 per-phase path.The REQ-052
EnvironmentClient::exec_session+forward_journal+EnvironmentJournalEventare defined but not called by the runner anywhere — and, importantly, not reachable from the runner's current pins:orchestrator-coreto revd03cd174(Cargo.tomlL46). REQ-052exec_sessionlanded inbcce5012("orchestrator-core: EnvironmentClient::exec_session + journal forwarding (REQ-052)"), which is one commit pastd03cd174on the unmerged branchagent/v0.7.0/env-exec-session(not onorigin/main).animus-environment-protocoltag (incl. the pinnedv0.7.0-rc.6) definesExecSessionRequest/Response,METHOD_ENVIRONMENT_EXEC_SESSION, orNOTIFICATION_ENVIRONMENT_JOURNAL.So this PR wires the runner side against the REQ-052 API (exact signatures/types taken from
bcce5012), and the actual call is compiled under a feature until those two pins are advanced (see Build gating).3. How does the runner detect a "session-capable" environment?
Via the plugin manifest capability list.
PluginManifest.capabilities: Vec<String>is documented as "Methods implemented by the plugin." A session-capable environment advertises"environment/exec_session"there.environment_is_session_capable(src/workflow_session.rs) discovers environment plugins withorchestrator_plugin_host::discover_by_kind(project_root, "environment"), selects by exactname(else the sole installed one — mirroringEnvironmentClient::resolve), and checkscapabilities_advertise_exec_session(&manifest.capabilities). Any discovery error / missing plugin fails safe tofalse(keeps the per-phase path).Change
src/workflow_session.rs(REQ-052 remote-animus session):capabilities_advertise_exec_session/environment_is_session_capable— detection.map_journal_event_kind— node journal kind → coarseRuntimeWorkflowEventKind(phase lifecycle only; the single terminal event is emitted by the driver so it isn't double-emitted).session_status_to_workflow_status— terminalExecSessionResponse.status→WorkflowStatus(fails closed toFailedon unknown).delegate_workflow_via_session— prepare bare node (repo onmetadata.github_repo),exec_sessionwith aSend + Syncjournal-forwarding closure, best-effort teardown, synthesizeWorkflowExecuteInternalResult. BlockingEnvironmentClientwork runs on a dedicated multi-thread runtime so the resident-host I/O driver outlivesprepare → exec_session → teardown(same hazardPreparedEnvironment::prepare_off_runtimeguards).src/workflow_execute.rs— the delegation branch (full runs only;--phaseruns keep the per-phase path).src/lib.rs— registerpub mod workflow_session;.Cargo.toml— add theremote-animus-sessionfeature.Build gating —
remote-animus-session(default OFF)Because REQ-052
exec_session/EnvironmentJournalEvent(orchestrator-core, past the pinned rev) and theenvironment/exec_sessionprotocol types (env-protocol, past the pinned tag) are not in the current pins, the delegation call is behind theremote-animus-sessioncargo feature:environment_is_session_capablereturnsfalse, so the branch never delegates and behavior is unchanged. The pure detection/mapping logic still compiles and is unit-tested.orchestrator-corerev + ananimus-environment-protocoltag that shipexec_session.To land end-to-end, three pins must advance together (a coordinated protocol release, outside this repo):
animus-environment-protocol— publishExecSessionRequest/Response,METHOD_ENVIRONMENT_EXEC_SESSION,NOTIFICATION_ENVIRONMENT_JOURNALunder a tag.orchestrator-core— mergebcce5012(REQ-052) repinned to that env-protocol tag; cut a rev.orchestrator-core/siblings to that rev +animus-environment-protocolto that tag, and build--features remote-animus-session.Testing
needs node/CI build+test — not built locally per policy (ao-cli builds/tests run on ephemeral nodes/CI). Unit tests added for the pure logic; the
exec_sessionround-trip needs the feature + advanced pins.