Repository navigation
[Doc] DFlash2: probabilistic drafting is the recommended setting in 0.1.4 - #41
Merged
Merged
Conversation
….1.4 The 0.1.4 release gate validated probabilistic drafting for DFlash2 at one to eight concurrent requests, at temperature 1 with thinking, including tool calls and structured output, and the reference deployment uses it. The documentation still told every method to use greedy drafting. Configuration now keeps greedy as a requirement for MTP only (upstream 1Cat-vLLM issue #656) and names probabilistic drafting as the DFlash2 recommendation. The DFlash2 page uses it in the launch example, gives the replay cost with probabilistic drafting (about 0.8 ms per step) and the GPU KV pool it measures (1,366,121 tokens, 2.61x at 512K; the greedy value 1,379,634 is unchanged). The KV capacity table gains the 0.1.4 row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
The 0.1.4 release gate validated probabilistic drafting for DFlash2, and the reference deployment now uses it, but the documentation still told every method to use greedy drafting. This updates the public docs to match 0.1.4:
docs/configuration.md: greedy drafting stays a requirement for MTP (upstream 1Cat-vLLM issue #656); for DFlash2, probabilistic drafting is the recommended setting from 0.1.4 and greedy still works.docs/dflash2.md: the launch example uses"draft_sample_method": "probabilistic"; a short paragraph explains the setting; the state-replay cost is given for both drafting methods (about 0.44 ms per step with greedy drafting, about 0.8 ms with probabilistic drafting); the GPU KV pool with probabilistic drafting is 1,366,121 tokens (2.61× at 512K), while the greedy value 1,379,634 is unchanged.docs/performance.md: the KV capacity table gains the 0.1.4 probabilistic-drafting row.No code changes. No effect on the support matrix in the README or
.github/SUPPORT.md. The release itself (release/0.1.4) is unchanged; these documents are not part of the wheel or the image.Validation
GPU KV cache size: 1,366,121 tokensandMaximum concurrency for 524,288 tokens per request: 2.61xwith probabilistic drafting,1,379,634/2.63xwith greedy drafting; the replay cost with probabilistic drafting is the gate's six-session replay measurement.pre-commit run --files docs/configuration.md docs/dflash2.md docs/performance.md(pre-commit 4.6.2): all hooks pass, including markdownlint-cli2 and typos.Attribution
AI-assisted: drafted with Claude Code. The commit is signed off under the Developer Certificate of Origin. A human review of every changed line is pending before merge.
🤖 Generated with Claude Code