Repository navigation
Conversation
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Contributor
Contributor
✅ Dynamo PR CI passed — run 38660904985 (attempt 1) on
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Depends on #16348 (
hzhou-codex/sd-selfbench-eagle3). That PR provides native async EAGLE3 decode self-benchmarking; this stacked follow-up completes the native prefill path and artifact contract without duplicating the decode implementation.--benchmark-mode prefilland both phases ofagg. Match timing and workload snapshots by native iteration ID and keep the point alive until its timing is received; never substitute scheduler wall time.single_stepas one exact retained observation (shape, timing, queue, native ID), preserve provenance/capacity, and reject CPU/CUDA relabeling. Existing AR artifacts and explicitly phase-specific legacy SD artifacts remain readable.Scope
Sequential EAGLE3, tensor parallelism, DP1/PP1/CP1, no EP/offload/KV transfer, native-FPM-capable asynchronous vLLM. No additional speculative algorithms are enabled. Real-KV cached-prefill collection requires
DYN_BENCH_PREFILL_REAL_SEED=1; synthetic prefixes remain labelled and are not qualified for strict AISimulate import.The execution binding matches the profile contract in ai-dynamo/aisimulate#476 and ai-dynamo/aisimulate#477. A resolved identity is not an assertion of coverage, import readiness, or serving accuracy. Local/unknown checkpoint identities remain explicitly unresolved; no network lookup or fabricated checkpoint revision is introduced. AISimulate's initial K=1–5 consumer restriction does not silently narrow the collector's existing positive-depth support.
Validation
Qualification boundary
GPU tests used cached Qwen3-0.6B + EAGLE3 on one RTX 6000 Ada, with a retained vLLM 0.24 runtime and the previously validated native-FPM lifecycle backport from
b3563fc65ae0f5359802593d78e7ea097e1fed31. Prefill cases additionally used a test-only adapter for the older two-element cache-lookup return: it reads the same cached-token element that the newer three-element API exposes, without changing the coordinator or production code. Decode cases did not need this adapter. Both adaptations remain outside this PR.Peak measured VRAM stays within the existing 3.6 GiB test marker; all test GPU processes were reaped. No checkpoint/image downloads or cluster allocations were performed. This is functional qualification, not current-nightly/container qualification or MiniMax/B200 performance evidence. The PR remains draft pending target-runtime CI and its dependency.
Documentation:
fern checkpasses after generating standard ignored API-reference pages.fern docs broken-linksreports one pre-existing link in the unchanged DeepSeek-V4-Pro-0813 recipe; the changed page has no reported broken links. No generated API pages are included in this diff.