Feature: loop runtime — config, orchestration, validators, and state machine

[hpieris-dm]

Problem

The existing dmx workflow requires a developer to know the right sequence and run each /dmx/* skill manually. They are the orchestrator — checking output at each step, deciding when to proceed, and remembering what comes next.

There is no declarative way to formalize a trusted skill sequence, encode validation judgment in code, or resume a multi-step workflow from persisted state. Foreground skills and background orchestration are the same thing today: the developer in the IDE.

Proposal

Introduce the loop runtime — a declarative execution engine that takes a loop config and runs skills sequentially with automated validators, policy-driven proceed/pause decisions, and durable state.

A loop has five properties:

1. Skills — ordered sequence executed within the loop
2. Trigger — what starts it (manual in M1; interface accepts notify_and_wait and auto for M3)
3. Memory — read persistent context before running; write learnings back on complete
4. Goal state — plain-language condition validators check against
5. Validators — automated checks that verify output against the goal state

Loop config

Declarative YAML. Default configs ship with dmx (src/dmx/loops/); teams override via .dmx/loops/.

name: dev
skills:
  - implement-next-phase
  - commit
trigger:
  type: manual
goal_state: "All phases implemented and committed, no unchecked tasks remaining"
repeat_until: all_phases_complete
validators:
  - tool: run_tests
    checks:
      - name: tests_pass
        required: true
      - name: coverage_threshold
        required: false
on_optional_failure: warn   # proceed | warn | pause
failure_handling: pause       # retry | pause | fail
human_gate: true
on_complete:
  on_success:
    trigger_loop: validate

Orchestrator (MCP tools)

Expose three tools on the dmx MCP server:

• run_loop(name) — load config, init state, return first skill instruction
• loop_advance(output) — persist skill output, apply human gate / advance / run validators on completion
• loop_continue() — resume a paused loop

The coding agent executes skills (IDE context required). Validators run via deterministic subprocess — the agent is not involved in validator execution.

Validators

Plain Python functions at repo-root validators/{name}.py, invoked by subprocess with JSON input:

• skill_outputs — outputs keyed by skill name
• goal_state — from loop config
• loop_context — job ID, task ID, loop name, branch, ticket reference

Output contract:

{
    "pass": bool,
    "message": str,
    "checks": [{"name": str, "pass": bool}]
}

Required vs optional checks are a loop config concern. The orchestrator applies on_optional_failure and failure_handling policies based on check results — not fuzzy judgment.

State machine

running → paused → running → ... → complete
                                 → failed
                → iterating → running   (repeat_until not yet met)

State serialized to .dmx/loop-state.json (active pointer) and .dmx/jobs/{job_id}/{loop_name}-{task_id}.json after each skill. Includes loop name, job ID, task ID, skill position, skills completed, validator results, timestamp.

Job ID resolved from .dmx/spec.md frontmatter ticket_id, falling back to git branch name.

Bundled loops

Ship default loop configs for the SDLC pipeline: spec, plan, dev, validate, release.

Foreground vs background

Foreground = developer runs /dmx/* skills manually (unchanged). Background = loop runtime formalizes a trusted sequence with validators. Graduation is writing validators that encode what the developer was checking by hand — not a config flag flip.

Out of scope

• Event triggers (notify_and_wait, auto, CI failure, cron) — M3
• True headless background worker without agent involvement for skill execution — skills always need IDE context; the runtime orchestrates via MCP
• Parallel skill execution or branching within a loop
• HTTP endpoints or MCP wrappers for validators

Acceptance criteria

Config and schema

• Pydantic schema validates all loop config fields (skills, trigger, goal_state, repeat_until, validators, policies, human_gate, on_complete)
• Bundled loops in src/dmx/loops/; app repo overrides via .dmx/loops/
• Skill names in bundled configs match actual skill slugs (e.g. create-ticket, not dmx-create-ticket)

State persistence

• .dmx/loop-state.json active pointer + per-run state under .dmx/jobs/{job_id}/
• Job ID from spec frontmatter → branch → fallback
• State tracks status, skill index, completed skills, skill outputs, validator results, outcome, timestamps
• State machine supports pending, running, paused, iterating, complete, failed

MCP orchestration

• run_loop, loop_advance, loop_continue registered on dmx MCP server
• Sequential skill execution with human gate pause/resume
• human_gate: false runs skills to completion without pausing between them
• System prompt enforces loop mode when .dmx/loop-state.json exists
• /run-loop and /loop-continue slash-command skills

Validators

• Orchestrator resolves validators/{name}.py deterministically and invokes via subprocess with JSON input contract
• Bundled validators ship with dmx; app repos override at repo root
• Required check failures apply failure_handling policy
• Optional check failures apply on_optional_failure policy
• Bundled validators implemented for loops that reference them (minimum: check_spec_complete; others as needed per bundled loop configs)

Iteration and chaining

• repeat_until: all_phases_complete transitions to iterating and re-triggers without recording failure
• on_complete.trigger_loop chains to the next loop on success (and per policy on failure/warning)

Memory

• Loop reads persistent context (memory bank) before running
• Loop writes learnings back on completion

Tests

• Unit tests for schema, state I/O, validator runner, policy decisions
• Integration coverage for a full loop run (skills → validators → outcome)

Context

Design reference: loop runtime spec (M1). Foreground skills remain the starting point; the loop runtime is the graduation path when validators encode developer judgment.

Related files (in progress on feature branch):

• src/dmx/loop_schema.py — config schema
• src/dmx/loop_state.py — state persistence
• src/dmx/loop_tools.py — MCP tools (validator execution currently stubbed)
• src/dmx/loops/*.yaml — bundled loop configs
• src/dmx/validators/ — bundled validators
• src/dmx/skills/loop/ — run-loop and loop-continue skills