Skip to content

fix(mocker): adopt AISimulate nightly 82 - #16945

Merged
tedzhouhk merged 6 commits into
mainfrom
hzhou-codex/fix-agentic-snapshot-constructor
Oct 26, 2026
Merged

tedzhouhk merged 6 commits into
mainfrom
hzhou-codex/fix-agentic-snapshot-constructor

Conversation

@tedzhouhk

@tedzhouhk tedzhouhk commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Upgrade the Python package and Rust core together from nightly 73 to the published AISimulate nightly 82 pair, including all five pins and all three Cargo lockfiles. Synchronize the branch with Dynamo main at 100cce2e48c4, including the earlier 2d17836993b0 and 4b487d890d merges.
  • Adapt the additive request, replay, topology, placement, and lifecycle APIs. Preserve text-only defaults and session identity. Keep the four AgentX snapshot fixtures on the seed-only serde contract so new optional fields retain their defaults.
  • Fail explicitly for new AISimulate features not yet supported by Dynamo-owned adapters: image requests and authored KV fetch hints in live/online/custom-KV routing, and SGLang host-loop mode in live schedulers. Reject host-loop mode before spawning tasks; do not silently discard its TTFT milestones. Native offline round-robin behavior remains upstream-owned and unchanged by these guards.
  • Extend dependency consistency coverage to the standalone Python-bindings Cargo manifest.
  • Preserve text-only synthetic benchmark defaults with images: None, and adapt the single_worker_topology() fixture added by main feat(planner): engine-owned retune with replay router control #17095 with router_hint: false.

Published artifacts

Component Version
Python aisimulate 0.13.0.dev202610240000000082
Rust aisimulate-core 0.13.0-dev.202610240000000082
AISimulate source f3a3e3156530574f5e7a10446f19e571b82db871

Published x86_64 wheel and crate bytes were downloaded and their SHA-256 digests verified against nightly run 82 provenance. This source includes AISimulate #463, #465, #476, #477, #492, and #574. Validation uses the published artifacts, not an unpublished source override.

Validation

Review fix and latest main synchronization

  • Merged main 100cce2e48c4 without conflicts and added router_hint: false to the new single_worker_topology() fixture. Independent review confirmed the merge differs from the clean merge tree only by that compatibility fix.
  • cargo test --locked -p dynamo-mocker --lib on the updated merge tree: 283 passed, including the new replay-router control tests from main.
  • cargo tree --locked succeeds in the root, Python-bindings, and KVBM workspaces, all using nightly 82. Dependency-consistency tests: 5 passed. Changed-file pre-commit, cargo fmt --all --check, and git diff --check passed.
  • Before this main synchronization, all nine CI feature-gated benchmark Clippy commands passed locally and full CI attempt 2 passed on d38d790c89. New-head CI must run again; those earlier results do not qualify the newer main merge.

Initial nightly 82 integration qualification

  • cargo test --locked -p dynamo-mocker --lib --no-default-features on Rust 1.96.1: 277 passed, including AgentX replay, grouped scheduling, and unsupported-field/host-loop regression tests.
  • Built fresh Dynamo Python bindings from this branch with maturin build --manifest-path lib/bindings/python/Cargo.toml --features ais-forward-pass --profile dev, then installed that runtime wheel with the editable Dynamo Python package and the actual published AISimulate 82 wheel. uv pip check passed.
  • Actual native-binding Python suites: 58 passed across test_ais_capacity.py (34), test_replay_ais_integration.py (9), test_replay_ais_capacity.py (7), and test_replay_agentic_timing.py (8). These exercise real AIS callbacks, aggregated/disaggregated replay, capacity, and AgentX timing; two live tests use the actual in-memory runtime.
  • Dependency pin/lockfile checks: 5 passed in tests/dependencies/test_aisimulate_consistency.py.
  • Published-artifact/public-API probes: 5 passed, covering provenance, seed-only and expanded snapshot serialization, three-draft-token EAGLE3 profiles, and the canonical best_available factory. An additional real AIS session/native static-oracle matrix passed 16 cases (nextn 0/2, attention DP 1/2, prefill/decode); vLLM and SGLang capacity probes passed.
  • Changed-file pre-commit hooks, workspace and standalone-bindings Rust formatting, and git diff --check passed. Independent review completed with its host-loop early-rejection finding fixed and retested.

Python suites were run offline (HF_HUB_OFFLINE=1, TRANSFORMERS_OFFLINE=1), one module per process, with --noconftest and the required pytest plugins explicitly enabled. The selected network-free suites do not require the parent conftest's blanket NATS/etcd autouse fixture. Separate module processes avoid a pytest-forked parent fixture-state teardown error seen in a combined invocation; no test assertions or repository warning filters were relaxed.

Validation boundaries

  • Local qualification used a CPU/x86_64 development build. The linked full-CI result covers the prior d38d790c89 head; qualification of the latest main synchronization is limited to the explicitly listed local checks until its new-head CI completes.
  • This upgrades the dependency and preserves existing integration boundaries. It does not add Dynamo callback support for canonical MTP/speculative FPM interpolation, multimodal replay, or authored peer-KV hints. Existing nextn/ngram support is unchanged.
  • Ready for review after local validation; CI on the pushed head must still complete before merge.

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
@tedzhouhk
tedzhouhk requested a review from a team as a code owner October 20, 2026 20:54
@github-actions github-actions Bot added the test label Oct 20, 2026
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: ai-dynamo/dynamo/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: babea6da-b634-47e6-bcca-0e9a6215bd48
📥 Commits

Reviewing files that changed from the base of the PR and between 682e0bd and f021648.

📒 Files selected for processing: 1
  • lib/mocker/src/replay/agentic.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 7 remain after this review.

Walkthrough

The agentic replay tests now deserialize AgenticSnapshotOptions from JSON in four test setups. Runtime behavior and public declarations are unchanged.

Changes

Agentic replay tests

Layer / File(s) Summary
Snapshot options in tests
lib/mocker/src/replay/agentic.rs
Four test setups now create snapshot options through JSON deserialization instead of direct struct literals.

Priority: Low

Estimated code review effort: 1 (Trivial) | ~3 minutes

Merge Risk: Minimal · up to f0216

No actionable merge-blocking risk was identified.

Pre-merge checks: Passed 3 · Failed 2

Failed Failed checks: 2 warnings

Check name Status Explanation Resolution
Title check Warning The title claims that the PR adopts AISimulate nightly 82, but the changes only update Agentic snapshot tests to use seed-only deserialization. The title is misleading about the primary change. Use a title that describes the test compatibility change, such as "fix(mocker): use seed-only AgenticSnapshotOptions deserialization in replay tests".
Description check Warning The description is detailed and relevant, and it includes implementation and validation information. However, it does not follow the required template headings and omits the required Related Issues se… Add the required sections: Overview, Details, Where should the reviewer start?, and Related Issues. Replace Closes #XXXX with the actual issue reference, or use the documented Linear issue format when applicable. Ensure the reviewer-start…
Passed Passed checks: 3 passed
Check name Status Explanation
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description is detailed and relevant, and it includes implementation and validation information. However, it does not follow the required template headings and omits the required Related Issues section with a real issue reference.

Resolution

Add the required sections: Overview, Details, Where should the reviewer start?, and Related Issues. Replace Closes #XXXX with the actual issue reference, or use the documented Linear issue format when applicable. Ensure the reviewer-start section identifies the files or areas to inspect first.

✨ Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@tedzhouhk
tedzhouhk marked this pull request as draft October 20, 2026 22:17
@devin-ai-integration

devin-ai-integration Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ Dynamo PR CI passed — run 40380859564 (attempt 2) on d38d790c89

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

Posted automatically by Devin for run 40380859564. Updated on every full-CI run of this PR.

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
@tedzhouhk tedzhouhk changed the title test(mocker): keep AgentX snapshot fixtures compatible with AISimulate fix(mocker): adopt AISimulate nightly 82 Oct 26, 2026
@tedzhouhk
tedzhouhk marked this pull request as ready for review October 26, 2026 17:55
@tedzhouhk
tedzhouhk requested review from a team as code owners October 26, 2026 17:55
@github-actions github-actions Bot added fix and removed test labels Oct 26, 2026

@dreamtalen dreamtalen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, approving once CI is all green. I'll add KVCR router hint support in a follow-up PR.

@dmitry-tokarev-nv dmitry-tokarev-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No P0 or P1 found. One P2, inline. This review read head d38d790c89.

  • [P2] A merge with current main (dea508e1c1, #17095) does not compile the dynamo-mocker tests. composition.rs:635 needs router_hint: false. Inline on composition.rs:576. The newest main, 8222ff098c, has the same literal and merges with this head without a conflict.

Open items:

  • In the PR run 40380859564, planner / Build multi-arch cpu was cancelled at the 60-minute job limit, and backend-status-check failed because of that. The image build with the nightly 82 pins passed for both architectures. The Compliance extract step then printed nothing for 21 minutes after the amd64 export. The same job was also cancelled on run 40382487453 of #17229. This review does not relate it to this change.
  • The description says that the branch merged main at 2d17836993b0. The head also merged 4b487d890d later. Both merge commits match a fresh git merge-tree and add nothing to the diff of this PR.
What this review measured, on the GPU box and on macOS, with base `4b487d890d` as the control.

Pins and locks:

  • All five pins name 0.13.0.dev202610240000000082 or 0.13.0-dev.202610240000000082. No other file names nightly 73.
  • Each of the three Cargo.lock files changes 4 lines, for aisimulate-core only.
  • cargo tree --locked passes in ., lib/bindings/python and lib/bindings/kvbm, at base and at head. The only dependency change is aisimulate-core 73 to 82.

Published artifacts:

  • The crate sha256 equals the lock checksum fa3e351f.... The x86_64 wheel sha256 equals the PyPI digest 7f985fff....
  • The crate names f3a3e3156530574f5e7a10446f19e571b82db871 in .cargo_vcs_info.json. 266 of its 270 files are byte-equal to crates/core at that commit. The other 4 are the packaging files and the version stamp.
  • Every Python and data file in the wheel is byte-equal to python/aisimulate at that commit. Only the compiled .so is not in git.
  • AISimulate #463, #465, #476, #477, #492 and #574 are in the history of that commit. The nightly run 40076769417 built that commit.

Rust (cargo test --locked -p dynamo-mocker --all-targets):

tree result
base 272 passed
head 277 passed

A deliberate break of each new check fails exactly one new test:

break failing test
remove the KV-hint check in native_request unhandled_request_fields_do_not_mutate_compatibility_state
remove the image check in native_request unhandled_request_fields_do_not_mutate_compatibility_state
remove the sglang.host_loop check host_loop_is_rejected_before_live_tasks_start
change the TTFT milestone message host_aware_ttft_milestones_are_rejected_explicitly
remove the KV-hint check in the offline KV router unhandled_request_fields_are_rejected_instead_of_dropped
remove the image check in the offline KV router unhandled_request_fields_are_rejected_instead_of_dropped
remove the KV-hint check in online replay online_replay_rejects_unhandled_request_fields_before_placement
remove the image check in online replay online_replay_rejects_unhandled_request_fields_before_placement

Where each new error goes: online replay returns it from the whole replay (joined?? in live_runtime.rs). The offline KV router returns it from on_request_arrival. The grouped live engine sends it on the command reply. Outside the tests, the DirectRequest and synthetic trace builders in this tree set no images and no KV hints. So these checks do not stop a text-only request.

Python, with a _core built from each tree (maturin build --features ais-forward-pass --profile dev):

suite base, nightly 73 head, nightly 82
AISimulate tests in tests/dependencies 9 passed 9 passed
the 4 binding modules that the description names 58 passed 58 passed
the 30 test files that use aisimulate, with -n 8 594 passed, 7 failed, 6 skipped 595 passed, 6 failed, 6 skipped

Each head failure also fails at base. Three router end-to-end tests fail on both trees, also in a run of their own. In each, a server process of the test exits with code 1 before it opens its port. The other failures are smoke tests in profiler/tests/test_replay_optimize.py under -n 8. In a run of their own, the three that failed at head pass at head and at base.

tests/dependencies/test_aisimulate_consistency.py: 5 passed at head. A changed pin in lib/bindings/python/Cargo.toml or in container/deps/requirements.aisimulate.txt makes it fail. At base, the same change in lib/bindings/python/Cargo.toml still passes (5 passed), so the new check in this PR catches it.

@dmitry-tokarev-nv dmitry-tokarev-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head d38d790c89. No P0 or P1 found.

All five pins and the three Cargo.lock files move to AISimulate nightly 82, and cargo tree --locked passes in all three workspaces. The published crate and wheel match source commit f3a3e315. A removal of each new request check fails a test, and dynamo-mocker passes 277 tests. The Python suites that use aisimulate show no new failures against base.

Open item: one P2. A merge with current main (dea508e1c1, #17095) needs router_hint: false in single_worker_topology() in composition.rs, or the dynamo-mocker tests do not compile. Please merge main with that line before this lands.

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
@tedzhouhk
tedzhouhk enabled auto-merge (squash) October 26, 2026 21:54
@tedzhouhk
tedzhouhk merged commit b5313e7 into main Oct 26, 2026
129 checks passed
@tedzhouhk
tedzhouhk deleted the hzhou-codex/fix-agentic-snapshot-constructor branch October 26, 2026 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants