[codex] Add Pipelex API orchestration and tools endpoints

[lchoquel]

Summary

• Adds request-level orchestration mode handling for execute/start/validate flows and related API config/error handling.
• Adds Pipe Tools endpoints for linting and formatting single .mthds files, backed by pipelex-tools-py.
• Updates docs, OpenAPI, and unit coverage for the new API contract.

Validation

• .venv/bin/python -m pytest tests/unit/test_tools_routes.py -q
• .venv/bin/ruff format --check api/routes/pipelex/tools.py tests/unit/test_tools_routes.py
• .venv/bin/ruff check api/routes/pipelex/tools.py tests/unit/test_tools_routes.py
• .venv/bin/pyright api/routes/pipelex/tools.py tests/unit/test_tools_routes.py
• .venv/bin/python scripts/export_openapi.py --check docs/openapi/pipelex-api.openapi.yaml

Notes

• Draft PR opened against dev as requested.
• Focused validation was run for the tools endpoint change; broader branch validation should run in CI.

---

Summary by cubic

Make the API orchestrator-agnostic and add lightweight MTHDS Tools. Routes now dispatch by orchestration_mode via a new api.toml, /start refuses non-async backends, and /validate cleanly separates invalid bundles (200) from faults (problem+json).

• New Features

  • Orchestration mode: new api/api.toml with orchestration_mode (default direct) and allow_request_orchestration_mode_override. /v1/execute, /v1/start, and /v1/validate honor it.
  • Honest async: /v1/start returns 400 (StartRequiresAsyncOrchestration) when the resolved backend can’t do fire-and-forget.
  • MTHDS Tools: /v1/lint and /v1/format for single .mthds files, backed by pipelex-tools-py and capped by MAX_MTHDS_FILE_BYTES. Docs page: “MTHDS Tools”.
  • Error handling: RFC 7807 across the board with plugin-contributed transport mappers; /validate returns 200 for user-fixable invalid bundles and uses problem+json for backend/runner faults. OpenAPI and docs updated; broad unit test coverage.
  • Deps: pin pipelex to a git revision for CI; no behavior change.

• Migration

  • Pick an orchestrator flavor and set orchestration_mode in ~/.pipelex/api_override.toml or .pipelex/api_{env}.toml (the base image runs direct in-process).
  • To let callers switch backends per request, set allow_request_orchestration_mode_override = true.
  • If you use direct, call /execute for runs; /start requires an async-capable orchestrator.

^{Written for commit 82f3537. Summary will update on new commits.}

[greptile-apps]

Greptile Summary

This PR makes the API choose orchestration backends per request and adds editor tooling endpoints. The main changes are:

• New api.toml orchestration-mode config and override policy.
• /execute, /start, and /validate dispatch through plugin registries.
• Plugin-supplied HTTP error mappers replace direct Temporal handling.
• New /lint and /format routes backed by pipelex-tools-py.
• Updated OpenAPI, docs, dependencies, and focused unit tests.

Confidence Score: 4/5

This is close, but I would fix the packaging mismatch before merging.

• Standard package installs can resolve an older Pipelex core than the code imports.
• Validation dispatch has a conditional path where backend fault reports can be shaped as invalid content verdicts.
• The main orchestration and tools paths have focused test coverage.

pyproject.toml and api/routes/pipelex/validate.py

Important Files Changed

Filename	Overview
api/routes/pipelex/pipeline.py	Adds registry-based orchestration dispatch for run, start, and validation flows.
api/routes/pipelex/validate.py	Maps registry validation verdicts to valid or invalid 200 responses, with one conditional fault-classification issue.
api/routes/pipelex/tools.py	Adds lint and format endpoints with bounded request bodies and response models.
api/main.py	Registers plugin-provided transport error mappers during app setup.
pyproject.toml	Adds the tools dependency and git-pins Pipelex for uv, but leaves package metadata pointing at an older released core.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[HTTP request] --> B{Endpoint}
  B -->|/execute| C[Resolve orchestration_mode]
  C --> D[OrchestratorRegistry]
  D --> E[BLOCKING run]
  E --> F[Rehydrate PipeOutput]
  B -->|/start| G[Resolve orchestration_mode]
  G --> H{supports fire-and-forget?}
  H -->|no| I[400 start requires async]
  H -->|yes| J[FIRE_AND_FORGET run]
  J --> K[202 start ack]
  B -->|/validate| L[Resolve orchestration_mode]
  L --> M[BundleValidatorRegistry]
  M --> N{Verdict type}
  N -->|PipelexValidationReport| O[200 valid report]
  N -->|ErrorReport| P[200 invalid report]
  B -->|/lint or /format| Q[pipelex-tools-py]
  Q --> R[200 diagnostics or formatted output]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
  A[HTTP request] --> B{Endpoint}
  B -->|/execute| C[Resolve orchestration_mode]
  C --> D[OrchestratorRegistry]
  D --> E[BLOCKING run]
  E --> F[Rehydrate PipeOutput]
  B -->|/start| G[Resolve orchestration_mode]
  G --> H{supports fire-and-forget?}
  H -->|no| I[400 start requires async]
  H -->|yes| J[FIRE_AND_FORGET run]
  J --> K[202 start ack]
  B -->|/validate| L[Resolve orchestration_mode]
  L --> M[BundleValidatorRegistry]
  M --> N{Verdict type}
  N -->|PipelexValidationReport| O[200 valid report]
  N -->|ErrorReport| P[200 invalid report]
  B -->|/lint or /format| Q[pipelex-tools-py]
  Q --> R[200 diagnostics or formatted output]

Loading

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
pyproject.toml:19
**Package installs wrong core** The new API code imports the orchestration registry and runtime-bridge modules from the git-pinned Pipelex core, but the published dependency still says `pipelex==0.35.0`. `tool.uv.sources` is only used by uv, so a wheel, Docker layer, or pip install can resolve the released 0.35.0 package and then crash on import when `api.main` or `api.routes.pipelex.pipeline` loads the new symbols. Please make the runtime dependency point at the same commit, or bump it to the released Pipelex version that contains these APIs.

### Issue 2 of 2
api/routes/pipelex/validate.py:190-191
**Separate fault verdicts** This branch treats every non-`PipelexValidationReport` returned by a validator as a 200 invalid-bundle verdict. That is correct for a `ValidateBundleError` report with `validation_errors`, but a registry-backed validator can also return an `ErrorReport` for a backend/config/transport failure. In that case this route would drop the report's server-fault classification and return `is_valid: false` with an empty validation list, so clients see a content error instead of a problem response. Please check that the returned report is actually a validation verdict before building the 200 invalid response, and let non-verdict faults reach the global error handler.

_{Reviews (1): Last reviewed commit: "fix(dependencies): update pipelex depend..." | Re-trigger Greptile}