[sejori]

Third of five stacked PRs for the worker validation gates DEP (ai-dynamo/dynamo#15897). Based on #2. #4 is stacked on this PR so that #5's end-to-end test covers both.

Why

Under etcd discovery the frontend routes to endpoint records that have an admitted model card, whatever the pod's readiness. Holding /health alone doesn't keep traffic off a held worker, so the card's discovery write has to be held too.

What

• New lib/llm/src/validation_hold.rs:
  • With DYN_WORKER_VALIDATION_HOLD=card (the default) and the worker not yet released, register_model_card parks the card in a per-process map keyed by (connection_id, namespace, component, endpoint, suffix), then returns Ok.
  • Each held card adds a release hook (from feat(runtime): worker validation gates #2), so cards are written in registration order. Each write is retried up to 3 times; a final failure fails the publish gate and the worker stays held.
  • While a card is held, update_model_taints edits the held spec, using the new DiscoverySpec::with_model_taints so the rules match Discovery::update_model_taints (reserved topology prefix and so on). detach_from_endpoint drops the held card and skips the discovery unregister.
  • The map uses an async lock, so a release write and a concurrent taint update or unregister are ordered.
• register_model_card still calls record_engine_capacity, which sizes the process rather than routing. Without gates, the order is unchanged: register, then record.
• hold=health writes the card immediately.
• Virtual planner: _get_actual_worker_count counts only instances whose model card is published, using the existing FpmEventSubscriber.get_model_cards() snapshot. When cards can't be tracked, it falls back to the endpoint-instance count with a warning.

Tests

Discovery-level tests on the in-memory store (--features integration):

• the card is held, then written on release;
• hold=health writes it immediately;
• a base card and a LoRA card are written in registration order;
• taints set during the hold are in the written card, and go straight to discovery after release;
• reserved taints are rejected during the hold;
• unregister during the hold drops the card;
• after release, cards are written directly;
• a failing gate never writes the card;
• without gates, behaviour is unchanged.

Planner unit tests: workers without a card aren't counted and are counted once the card is published; untracked cards fall back to the endpoint count.

cargo clippy -p dynamo-llm -p dynamo-runtime --all-targets -D warnings is clean, and all 12 virtual-connector tests pass.

Known differences

• Discovery::register's different-model conflict check runs at write time. Under the hold, a conflicting registration returns Ok and fails at release as a publish gate failure. That's fail-closed, but later than today.
• Cards written outside register_model_card aren't held. That covers the vLLM sidecar LoRA path (sidecar mode is rejected at startup) and lib/rl.

Stack

1. refactor(runtime): serve /live and /health from separate handlers #1 Split /live from /health
2. feat(runtime): worker validation gates #2 Runtime validation gates
3. This PR: hold the model card until release
4. feat(llm): loopback OpenAI endpoint for validation checks #4 Loopback OpenAI endpoint
5. feat: reference check runner, docs and example DGD for validation gates #5 Reference check runner, docs and an example DGD

🤖 Generated with Claude Code

[cubic-dev-ai]

Running ultrareview automatically — this PR appears to change concurrency-sensitive code such as locks, queues, or retries, where a missed race may only surface under production load, so a deeper multi-pass review is worth running. I'll post findings when complete.

[cubic-dev-ai]

Ultrareview ran automatically — this PR appears to change concurrency-sensitive code such as locks, queues, or retries, where a missed race may only surface under production load, so a deeper multi-pass review is worth running.

All reported issues were addressed across 6 files

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[sejori]

Superseded by the cut-down stack #6 → #7 → #8 → #9 → #10 → #11 (~4,950 → ~2,000 lines): #2 is split into exec gates (#7) and sidecar url gates (#8), the url contract is now 200 = pass, and metrics, the fixed loopback port, the virtual-planner change and the parity harness are dropped. The review threads here stay for reference.

🤖 Generated with Claude Code