[tedzhouhk]

Note

Stacked PR, live retune (split from #17562). Merge order: #17483 → #17733 (this PR) → #17734 → #17735 → #17736 → #17562

This PR targets hzhou/dgdr-trace-pvc, the branch of #17483 (the PR below it), so its diff shows only this layer. #17562 was split into these layers for review. Together they carry #17562's original diff exactly, plus a 40-line replay check in #17734 that rejects executor: dgdr in offline replay.

Overview

Live planner retune (#17562) compares Candidates that different DGDR Runs of one window scored. That comparison only works if a Candidate's score depends on its own configuration and nothing else. This PR makes DynamoReplayRunner replay every trace workload with common random numbers (CRN): ordinal request ids and the canonical router seed. DynamoReplayRunner is the dynamo replay runner that AISimulate's Sweeper loads (entry point in pyproject.toml), and it is what a DGDR trace Run executes.

run_trace_replay already has the internal _deterministic option for this; offline replay retune uses it for its baseline and candidates. This PR only passes it from the Sweeper runner. The synthetic-workload path is unchanged.

This layer changes no file that #17483 touches. It sits on #17483 for merge order, because DGDR trace Runs (spec.workload.trace) come from #17483.

Key Changes

• components/src/dynamo/replay/simulation.py: passes _deterministic=True to both run_trace_replay calls of DynamoReplayRunner (agg and disagg). run_synthetic_trace_replay has no such parameter and does not receive it.
• components/src/dynamo/replay/tests/test_simulation_crn.py (new): replaces the replay entry points with fakes that bind their kwargs to the real signatures. It checks that the flag is set for agg and disagg with dynamo and mooncake trace workloads, and that synthetic replays never receive it.

Behavior note: this applies to every trace replay through the Sweeper's dynamo runner, including every DGDR trace Run, not only the Planner's. A Run's scores no longer depend on the other candidates or on the Run's parallelism.

Validation

Ran on this layer (e9ab6afc24) and compared with the same suites at the stack base 6e8087d359 (#17483's head, with main c6b055d4c7 merged in):

Check	Result
replay/tests/test_simulation_crn.py	6 passed
Planner and replay suites (components/src/dynamo/planner/tests, components/src/dynamo/replay/tests, no xdist)	59 failed, 2137 passed, 2 skipped. At the base: 59 failed, 2131 passed, 2 skipped. The two sets of failing tests are identical; the 6 extra passes are the new tests.
pre-commit run --files on the 2 changed files	All hooks pass, no files changed
CI mypy (the dynamo-pipeline.yml command, MYPYPATH=components/src:lib/bindings/python/src mypy --explicit-package-bases) on dynamo/planner and dynamo/replay	9 errors, the same 9 as at the base: missing yaml/pandas/nvtx stubs, plus existing arg-type errors in common/http/url_validator.py and planner/simulation/provider.py. None is new.

The 59 failures come from this host. Its prebuilt bindings lack conversation_affinity (#16854) and the ais-forward-pass feature, and it has no etcd binary. CI builds the bindings from source.

The development-cluster run in #17562 used a planner image with this change for its Run Jobs, on #17562's pre-split branch. Both windows of that run reached a decision (kept, applied). The executor allows a decision only when the CRN probe's score matches the grid evaluation of the same point within a relative error of 1e-9. This layer was not run on a cluster by itself.

Issues and Ongoing Work

• Part of live retune (feat(planner): enable live retune through DGDR v1beta2 #17562), the retune work in feat(planner): engine-owned retune with replay router control #17095 and the runtime reconfiguration DEP DEP (light): Planner runtime reconfiguration with predictor warmup #16835.
• The next layer is feat(planner): add the live retune configuration and deployment premises #17734.

Pull Request Checklist

• I used Dynamo's /pr-checklist skill.

🤖 Generated with Claude Code

[devin-ai-integration]

✅ Dynamo PR CI passed — run 41146042861 (attempt 1) on e9ab6afc24

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

_{Posted automatically by Devin for run 41146042861. Updated on every full-CI run of this PR.}

[jthomson04]

Reviewed at e9ab6af. The change is fine: the canonical seed is the same for every candidate and window, no user seed is overridden, and the offline-only guard can't trip. Two nits inline.

[jthomson04]

140 lines of tests for a 9-line change. test_simulation.py::test_trace_runner_preserves_current_replay_arguments already captures these kwargs; add assert seen["_deterministic"] is True and one disagg row there and drop this file. The synthetic test checks a function this PR doesn't touch.

[jthomson04]

Cut this to the invariant: trace replays use ordinal ids and the canonical router seed. Naming the live-retune consumer here will go stale. replay/api.py:100 still says _deterministic is internal to retune sweeps, which this PR makes untrue.