Skip to content

feat(replay): consume canonical AIS speculative decoding - #15627

Draft
Harrilee wants to merge 11 commits into
mainfrom
harrli/agentx-mtp-adapter
Draft

Harrilee wants to merge 11 commits into
mainfrom
harrli/agentx-mtp-adapter

Conversation

@Harrilee

@Harrilee Harrilee commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Forward canonical AISimulate speculative decoding through Dynamo's current EngineLaunchConfig and replay adapter. Preserve the selected scheme and draft checkpoint, use its resolved replay depth for native timing queries and scheduler progress, and advertise generic speculation / expected-acceptance support to the unified CLI and Sweeper.

Update the Python/Rust dependency pair together and adapt the newer native request, handoff, and replay descriptors. Keep the existing model-family, linear-depth, acceptance, fixed-capacity, and host-offload constraints enforced by AISimulate. The planner CI image includes the new tests and their self-contained fixture.

Release dependency: blocked on ai-dynamo/aisimulate#505 and a published wheel/crate pair containing that fix. The currently pinned nightly contains #382 but its shared launch adapter rejects canonical speculative depth. This PR remains a draft until the fixed pair is published and the pins are updated; the local validation below uses an explicit local patch, not an already published fix.

Closes #16722.

Validation

  • Rebuilt and installed the actual native Dynamo wheel against the current published AIS pair: 70 Python baseline tests and 263 Rust Mocker tests pass.
  • Rebuilt native Dynamo and AISimulate with the shared launch fix: all 108 focused Python tests pass, including all 22 speculative integration tests. They execute MTP, EAGLE3, ngram, DFlash, DSpark, and standalone draft-model configurations across aggregated/disaggregated deployments and supported vLLM/SGLang combinations; check that acceptance changes completion time; and reject missing capacity/acceptance, tree drafts, excessive depth, and conflicting controls.
  • Public aisimulate predict --stack dynamo CLI smoke with EAGLE3 completes all three requests and its agentic trajectory.
  • Focused pre-commit hooks, Rust formatting, and installed dependency consistency pass.
  • Local patched builds are validation artifacts only. No local path or Git dependency is committed, and published-package CI is not yet green for generic speculation.

@Harrilee Harrilee changed the title fix: preserve MTP assumptions through replay adapters feat: preserve MTP assumptions through replay adapters Oct 5, 2026
@github-actions github-actions Bot added feat and removed fix labels Oct 5, 2026
@Harrilee
Harrilee force-pushed the harrli/agentx-mtp-adapter branch 2 times, most recently from 073daa1 to 624f0f2 Compare October 5, 2026 06:56
@Harrilee
Harrilee changed the base branch from harrli/aic-1817-existing-adapter to main October 5, 2026 06:56
@devin-ai-integration

devin-ai-integration Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

❌ Dynamo PR CI failed — run 39027565114 (attempt 1) on 40cf7deb6b

Gate checks: ❌ backend-status-check · ✅ deploy-status-check · ❌ dynamo-status-check

Framework 1-GPU amd64
vLLM ❌ 1
Other Jobs
dynamo-runtime ❌ 3 ✅ 5
dynamo-sidecar ❌ 1
planner ❌ 2 ✅ 3

Failure details

7 jobs failed. In 4 test jobs (planner CPU amd64/arm64, dynamo-runtime sequential amd64/arm64), the same 22 replay speculation tests in test_speculation.py raise ValueError: canonical nextn conflicts with scheduler aic_nextn (or does not implement ngram draft scheduling) from aisimulate. dynamo-runtime / rust-gpu fails to compile lib/llm/src/mocker/handoff_tests.rs (missing state_bytes). The vLLM GPU test job (exit 137 kill) and the sidecar build (BuildKit connection timeout) look like infrastructure failures. No jobs were cancelled.

❌ planner / CPU Test cpu, amd64: 22 replay speculation tests: ValueError: canonical nextn conflicts with scheduler aic_nextn

Job: planner / CPU Test cpu, amd64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 117384705246 -R ai-dynamo/dynamo --log-failed

FAILED components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-agg] - ValueError: canonical nextn conflicts with scheduler aic_nextn
FAILED components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-agg] - ValueError: engine launch does not implement ngram draft scheduling
...
/opt/dynamo/venv/lib/python3.13/site-packages/dynamo/replay/simulation.py:355: in _engine_args
    materialize_engine_launch_config(
        payload    = {'block_size': 4, 'engine_type': 'vllm', 'num_gpu_blocks': 256, 'speculation': {'expected_accepted_tokens': 2.4, 'kind': 'mtp', 'params': {'depth': 3}, 'seed': 73}, ...}
/opt/dynamo/venv/lib/python3.13/site-packages/aisimulate/runner.py:1392: in materialize_engine_launch_config
    _runtime.engine_launch_from_replay_role_json(
E   ValueError: canonical nextn conflicts with scheduler aic_nextn
=========== 22 failed, 2263 passed, 22 skipped in 376.45s (0:06:16) ============

The same 22 tests in components/src/dynamo/replay/tests/test_speculation.py fail: all 16 test_agentic_replay_preserves_scheme_and_accepted_progress cases and all 6 test_native_boundary_rejects_invalid_speculative_controls cases. dynamo/replay/simulation.py:_engine_args calls aisimulate.runner.materialize_engine_launch_config, and that call raises ValueError: canonical nextn conflicts with scheduler aic_nextn (20 tests) or ValueError: engine launch does not implement ngram draft scheduling (the 2 ngram cases).

❌ planner / CPU Test cpu, arm64: same 22 replay speculation tests (canonical nextn conflict)

Job: planner / CPU Test cpu, arm64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 117384705238 -R ai-dynamo/dynamo --log-failed

Same 22 test_speculation.py failures with the same ValueErrors as the amd64 planner job (22 failed, 2263 passed, 22 skipped).

❌ dynamo-runtime / test / sequential cuda13.0, amd64: same 22 replay speculation tests (canonical nextn conflict)

Job: dynamo-runtime / test / sequential cuda13.0, amd64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 117385138456 -R ai-dynamo/dynamo --log-failed

Same 22 test_speculation.py failures with the same ValueErrors (22 failed, 5389 passed, 153 skipped, 2519 deselected). No other tests failed.

❌ dynamo-runtime / test / sequential cuda13.0, arm64: same 22 replay speculation tests (canonical nextn conflict)

Job: dynamo-runtime / test / sequential cuda13.0, arm64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 117385138459 -R ai-dynamo/dynamo --log-failed

Same 22 test_speculation.py failures with the same ValueErrors (22 failed, 5389 passed, 153 skipped, 2519 deselected). No other tests failed.

❌ dynamo-runtime / rust-gpu: error[E0063]: missing field `state_bytes` in HandoffTransferTiming (lib/llm/src/mocker/handoff_tests.rs)

Job: dynamo-runtime / rust-gpu · Failed step: Run Rust checks (block-manager + media-ffmpeg + MM routing + integration tests) · Logs: gh run view --job 117385138399 -R ai-dynamo/dynamo --log-failed

error[E0063]: missing field `state_bytes` in initializer of `dynamo_mocker::common::handoff::HandoffTransferTiming`
  --> lib/llm/src/mocker/handoff_tests.rs:59:5
   |
59 |     HandoffTransferTiming {
   |     ^^^^^^^^^^^^^^^^^^^^^ missing `state_bytes`

error[E0063]: missing field `state_bytes` in initializer of `dynamo_mocker::common::handoff::HandoffTransferTiming`
  --> lib/llm/src/mocker/handoff_tests.rs:69:16
   |
69 |     let full = HandoffTransferTiming {
   |                ^^^^^^^^^^^^^^^^^^^^^ missing `state_bytes`

error: could not compile `dynamo-llm` (lib test) due to 2 previous errors

The dynamo-llm lib tests do not compile. Both HandoffTransferTiming initializers in lib/llm/src/mocker/handoff_tests.rs (lines 59 and 69) are missing the state_bytes field.

❌ vllm-runtime / Test cuda13.0, amd64: GPU test step killed with exit code 137 (infra)

Job: vllm-runtime / Test cuda13.0, amd64 · Failed step: Run GPU tests (parallel) · Logs: gh run view --job 117387289140 -R ai-dynamo/dynamo --log-failed

[w21] tests/router/test_router_e2e_with_vllm.py::test_router_decisions_vllm_multiple_workers[tcp] PASSED [76s]
[elapsed 809s] GPU0: 1.4/22 GiB [w22(44s)] [queued: w6, w7, w10, w11, w18, w25]
[w6] tests/serve/test_vllm.py::test_serve_deployment[mm_agg_gemma4-e2b-it-3] (GPU0, profiled=12.0 GiB, req_kv=0.86 GiB) RUNNING
[elapsed 819s] GPU0: 2.9/22 GiB [w6(10s), w22(54s)] [queued: w7, w10, w11, w18, w25]
##[error]Error: failed to run script step: Error: command terminated with non-zero exit code: command terminated with exit code 137
##[error]Executing the custom container implementation failed. Please contact your self hosted runner administrator.
##[error]Error: Internal error occurred: unable to upgrade connection: container not found ("job")

No test failed before this. The earlier unit stages passed (2877 passed, 10 skipped) and every GPU test that finished passed. About 820s into Run GPU tests (parallel), the step was killed with exit code 137 (SIGKILL) while tests/serve/test_vllm.py::test_serve_deployment[mm_agg_gemma4-e2b-it-3] was running, and the job container then disappeared (container not found). That looks like a runner or pod kill (possibly OOM), not a test assertion, so rerunning the failed jobs is likely enough.

❌ dynamo-sidecar / Build multi-arch cpu: buildkit remote driver: context deadline exceeded (infra)

Job: dynamo-sidecar / Build multi-arch cpu · Failed step: Extract sidecar binaries (amd64) · Logs: gh run view --job 117381509033 -R ai-dynamo/dynamo --log-failed

#0 building with "b-39027565114-1" instance using remote driver

#1 [internal] waiting for connection
#1 ERROR: context deadline exceeded
------
 > [internal] waiting for connection:
------
ERROR: failed to build: context deadline exceeded
##[error]Process completed with exit code 1.

Infrastructure: in the binary-extraction step, the remote BuildKit builder never accepted a connection (context deadline exceeded), so no code was built. Rerunning the failed jobs is likely enough.

For agents
{"pr": 15627, "run_id": 39027565114, "run_attempt": 1, "head_sha": "40cf7deb6b9fb5267e847c317246c5148e189c59", "failures": [{"job": "planner / CPU Test cpu, amd64", "job_id": 117384705246, "failed_step": "Run CPU-only tests (parallelized)", "signature": "22 replay speculation tests: ValueError: canonical nextn conflicts with scheduler aic_nextn", "tests": ["components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dflash-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dflash-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dspark-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dspark-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[draft_model-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[draft_model-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[capacity]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[inferred]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[rates]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[depth]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[tree]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[conflict]"], "log_cmd": "gh run view --job 117384705246 -R ai-dynamo/dynamo --log-failed"}, {"job": "planner / CPU Test cpu, arm64", "job_id": 117384705238, "failed_step": "Run CPU-only tests (parallelized)", "signature": "same 22 replay speculation tests (canonical nextn conflict)", "tests": ["components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dflash-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dflash-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dspark-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dspark-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[draft_model-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[draft_model-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[capacity]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[inferred]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[rates]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[depth]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[tree]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[conflict]"], "log_cmd": "gh run view --job 117384705238 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-runtime / test / sequential cuda13.0, amd64", "job_id": 117385138456, "failed_step": "Run CPU-only tests (parallelized)", "signature": "same 22 replay speculation tests (canonical nextn conflict)", "tests": ["components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dflash-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dflash-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dspark-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dspark-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[draft_model-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[draft_model-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[capacity]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[inferred]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[rates]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[depth]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[tree]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[conflict]"], "log_cmd": "gh run view --job 117385138456 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-runtime / test / sequential cuda13.0, arm64", "job_id": 117385138459, "failed_step": "Run CPU-only tests (parallelized)", "signature": "same 22 replay speculation tests (canonical nextn conflict)", "tests": ["components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[mtp-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[eagle3-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[ngram-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dflash-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dflash-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dspark-sglang-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[dspark-sglang-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[draft_model-vllm-agg]", "components/src/dynamo/replay/tests/test_speculation.py::test_agentic_replay_preserves_scheme_and_accepted_progress[draft_model-vllm-disagg]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[capacity]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[inferred]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[rates]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[depth]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[tree]", "components/src/dynamo/replay/tests/test_speculation.py::test_native_boundary_rejects_invalid_speculative_controls[conflict]"], "log_cmd": "gh run view --job 117385138459 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-runtime / rust-gpu", "job_id": 117385138399, "failed_step": "Run Rust checks (block-manager + media-ffmpeg + MM routing + integration tests)", "signature": "error[E0063]: missing field `state_bytes` in HandoffTransferTiming (lib/llm/src/mocker/handoff_tests.rs)", "tests": [], "log_cmd": "gh run view --job 117385138399 -R ai-dynamo/dynamo --log-failed"}, {"job": "vllm-runtime / Test cuda13.0, amd64", "job_id": 117387289140, "failed_step": "Run GPU tests (parallel)", "signature": "GPU test step killed with exit code 137 (infra)", "tests": [], "log_cmd": "gh run view --job 117387289140 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-sidecar / Build multi-arch cpu", "job_id": 117381509033, "failed_step": "Extract sidecar binaries (amd64)", "signature": "buildkit remote driver: context deadline exceeded (infra)", "tests": [], "log_cmd": "gh run view --job 117381509033 -R ai-dynamo/dynamo --log-failed"}]}

Posted automatically by Devin for run 39027565114. Updated on every full-CI run of this PR.

@Harrilee
Harrilee force-pushed the harrli/agentx-mtp-adapter branch from 3564c12 to ff7babf Compare October 6, 2026 02:25
@Harrilee Harrilee changed the title feat: preserve MTP assumptions through replay adapters feat: forward generic speculative decoding in replay Oct 6, 2026
Normalize public and legacy MTP controls through the shared AIS contract, preserve canonical cost identity and authored capacity, and validate Agentic assumptions before execution. Retain acceptance and qualification in ordinary RR/KV reports. Pin matching AIS Python and Rust sources and defer backend context-length enforcement to the shared core.

Signed-off-by: Harry Lee <harrli@nvidia.com>
Signed-off-by: Harry Lee <harrli@nvidia.com>
Signed-off-by: Harry Lee <harrli@nvidia.com>
Signed-off-by: Harry Lee <harrli@nvidia.com>
Signed-off-by: Harry Lee <harrli@nvidia.com>
Signed-off-by: Harry Lee <harrli@nvidia.com>
Signed-off-by: Harry Lee <harrli@nvidia.com>
Signed-off-by: Harry Lee <harrli@nvidia.com>
Signed-off-by: Harry Lee <harrli@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 17, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@Harrilee Harrilee changed the title feat: forward generic speculative decoding in replay feat(replay): consume canonical AIS speculative decoding Oct 17, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation backend::vllm Relates to the vllm backend deployment::k8s Relates to dynamo deployment in kubernetes backend::sglang Relates to the sglang backend backend::trtllm Relates to the trtllm backend planner frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` multimodal router Relates to routing, KV-aware routing, etc. actions xpu labels Oct 17, 2026
Signed-off-by: Harry Lee <harrli@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

actions backend::sglang Relates to the sglang backend backend::trtllm Relates to the trtllm backend backend::vllm Relates to the vllm backend container deployment::k8s Relates to dynamo deployment in kubernetes documentation Improvements or additions to documentation feat frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` multimodal planner router Relates to routing, KV-aware routing, etc. size/L xpu

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(replay): consume canonical AISimulate speculative decoding

1 participant