[bbednarski9]

What

Adds escalation-mode llm_classifier and signal-driven stage_router support to the external nvidia.switchyard NeMo Relay plugin.

The version-2 plugin configuration remains backward compatible: an omitted classifier mode still selects capability routing. Escalation exposes the trajectory-judge settings, while stage routing exposes picker policy, signal windows, tier prompts, handoff notes, and an optional capability-classifier fallback. Both routers use the Relay-owned provider HTTP clients introduced by the base branch.

This is a stacked follow-up to NVIDIA-NeMo#270 and intentionally targets feat/nemo-relay-plugin-owned-http-client in this fork. It does not modify PR NVIDIA-NeMo#270.

The plugin README includes an algorithm compatibility matrix and a separate inventory of partial or not-yet-built integration components. The inventory documents validation and platform gaps without widening this PR's implementation scope.

Why

PR NVIDIA-NeMo#270 establishes the native plugin runtime and initially wires random and capability-classifier routing. Switchyard already implements escalation and stage routing in switchyard-libsy; exposing those algorithms through the plugin completes the planned in-process router integration without adding a Switchyard service or a new Relay API.

The follow-up patches also close issues found in an exact Relay 0.7.1 smoke: same-protocol prompt rewrites now reach providers, raw gateway placeholder sessions cannot share escalation latch state, stage default decisions retain tier/source metadata, hard overrides retain full confidence, and judge targets can set non-secret provider defaults such as think = false.

Relates to NVIDIA-NeMo#270.

How tested

• cargo test -p switchyard-nemo-relay-plugin — 45 tests
• cargo test --workspace --all-targets
• cargo clippy --workspace --all-targets -- -D warnings
• cargo fmt --all -- --check
• cd docs && env UV_CACHE_DIR=/private/tmp/switchyard-docs-uv-cache make publish with strict MkDocs validation
• git diff --check
• All GitHub Actions checks, including the required CI Success gate, passed on the compatibility-matrix commit
• Manual smoke with the official NeMo Relay 0.7.1 macOS arm64 release and Ollama 0.23.0
  • same-protocol capable system prompt and escalation handoff note reached the recorded provider request
  • critical override mark reported routing_tier = strong, decision_source = override, and confidence 1.000
  • picker-default mark reported routing_tier = strong and decision_source = fall_open
  • two positive no-session escalation verdicts both served weak; an explicit stable session served weak then strong
  • the recorded judge request contained the configured think = false provider default
• uv run ruff check . clean (N/A: no Python changes)
• uv run mypy switchyard clean (N/A: no Python changes)
• uv run pytest tests/ green (N/A: no Python changes)

Checklist

• One class per file; filename = snake_case of the primary class. (N/A: no Python classes added.)
• New public symbols exported from switchyard/__init__.py.__all__ if intended for downstream use. (N/A: no Python public symbols added.)
• Unit tests added for new components / bug fixes.
• README / --help updated if customer-facing surface changed.
• Commits signed off (Signed-off-by: Your Name <email>) per the DCO.

Notes for reviewers

• Start with crates/switchyard-nemo-relay-plugin/src/config.rs for schema-to-router construction, then crates/libsy/src/algorithms/stage.rs for cascade assembly and decision attribution.
• The compatibility matrix in crates/switchyard-nemo-relay-plugin/README.md distinguishes supported runtime behavior from partial validation and components that are not built yet.
• Escalation intentionally aggregates an unlatched weak stream before judging it. This delays first-token delivery and reconstructs a declined weak response from the aggregate, so its provider-event preservation envelope is not retained.
• Escalation with confirmations > 1 requires stable session identity. Raw Relay gateway placeholder identity is treated as absent; callers should send x-switchyard-session-id to retain a streak and latch.
• Structured-output judge targets remain limited to OpenAI Chat and OpenAI Responses; Anthropic Messages judge targets fail during static configuration validation.
• Stage decision marks add decision_source; non-stage fall-through decisions emit it as null.
• After PR feat(relay): add Switchyard-owned HTTP dynamic plugin NVIDIA-NeMo/Switchyard#270 changes or merges, this branch should be rebased onto its updated head and the draft retargeted without modifying feat(relay): add Switchyard-owned HTTP dynamic plugin NVIDIA-NeMo/Switchyard#270 from this branch.

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://bbednarski9.github.io/Switchyard/pr-preview/pr-2/
Built to branch gh-pages at 2026-08-08 20:54 UTC. Preview will be ready when the GitHub Pages deployment is complete.