[Doc] DFlash2: probabilistic drafting is the recommended setting in 0.1.4

docs/configuration.md

@@ -23,10 +23,13 @@ or disable custom all-reduce with `--disable-custom-all-reduce`.
 
 ## Launch options
 
-- **`draft_sample_method` must be `greedy`.** A probabilistic draft corrupts output
-  above `max_num_seqs=2` at temperature > 0 (upstream 1Cat-vLLM issue #656). This
-  setting is about draft verification only; clients can still use any sampling
-  parameters.
+- **`draft_sample_method`: `greedy` for MTP, `probabilistic` for DFlash2.** With MTP, a
+  probabilistic draft corrupts output above `max_num_seqs=2` at temperature > 0
+  (upstream 1Cat-vLLM issue #656), so MTP must stay `greedy`. For DFlash2, `probabilistic`
+  is the recommended setting from 0.1.4; it was validated in the 0.1.4 release gate at one
+  to eight concurrent requests, at temperature 1 with thinking, including tool calls and
+  structured output. `greedy` still works with DFlash2. This setting is about drafting and
+  draft verification only; clients can still use any sampling parameters.
 - **`--gpu-memory-utilization 0.95`** passed the full gate, including the multimodal
   stress test with 8 concurrent requests. Higher values ran out of memory in earlier
   multimodal stress tests. If you change concurrency or media limits, measure again.
@@ -73,8 +76,8 @@ client settings. Qwen recommends:
 - `top_k` and `min_p` are not standard OpenAI fields. With the OpenAI Python SDK, pass them
   in `extra_body`.
 - These are the model vendor's values. The [speed tests](performance.md) use
-  `temperature=0`. The greedy `draft_sample_method` in the launch command applies only to
-  draft verification, not to these request settings.
+  `temperature=0`. The `draft_sample_method` in the launch command applies only to
+  drafting and draft verification, not to these request settings.
 
 ## YaRN and native context
 

docs/dflash2.md

@@ -63,7 +63,7 @@ export VLLM_SM70_DFLASH2_FUSED_QKV_PACK=1
       "model": "/data/revolta/views/dflash2-512k",
       "num_speculative_tokens": 7,
       "attention_backend": "FLASH_ATTN_V100",
-      "draft_sample_method": "greedy",
+      "draft_sample_method": "probabilistic",
       "dflash_concurrency_drafts": {"4": 3, "5": 4},
       "sm70_dflash2": {"context_kv_graph": true, "context_pipeline": true,
         "verify_fastpath": true, "fused_gdn_metadata": true,
@@ -74,6 +74,11 @@ export VLLM_SM70_DFLASH2_FUSED_QKV_PACK=1
 In the startup log, the AWQ MoE line should read `grouped QPN decode admitted for 2..48
 rows (split=8, pack_rows=24, up to 2 row chunks)`.
 
+From 0.1.4, `"draft_sample_method": "probabilistic"` is the recommended setting for DFlash2:
+the drafter samples its proposals and the target accepts them with the standard rejection
+rule. `"greedy"` still works. MTP must stay `"greedy"`; see
+[Launch options](configuration.md#launch-options).
+
 ## Settings
 
 - **Enable the six `sm70_dflash2` options above.** They are off unless set.
@@ -122,7 +127,7 @@ GDN speculative state replay keeps two recurrent-state blocks per running reques
 ```
 
 - **Gain:** the GPU KV pool grew from 1,379,634 to 1,542,558 tokens (+11.8%; 2.63× to 2.94× maximum concurrency at 512K) on our 4 × V100 PCIe deployment. At eight concurrent 96K-token requests, peak KV use fell from 55% to 26%.
-- **Cost (0.1.4, measured in the same session):** about +0.44 ms per decoding step when the target samples (30.05 to 30.49 ms per step at one request, 30 replayed agent turns); with greedy decoding about 1–4% lower throughput at one to four requests and level at eight. Outputs were identical with replay on and off (24 of 24 greedy streams, 30 of 30 sampled turns).
+- **Cost (0.1.4, measured in the same session):** about +0.44 ms per decoding step with greedy drafting when the target samples (30.05 to 30.49 ms per step at one request, 30 replayed agent turns); with greedy decoding about 1–4% lower throughput at one to four requests and level at eight. Outputs were identical with replay on and off (24 of 24 greedy streams, 30 of 30 sampled turns). With probabilistic drafting, recommended from 0.1.4, the replay costs about 0.8 ms per step (six real agent sessions at one request, 250 W), with outputs identical to replay off.
 - **Who should use it:** deployments that run long contexts or several long requests at once and are limited by KV capacity. For single short requests, leave it off.
 - **CUDA graph:** with replay on, the replay after sampling runs as a CUDA graph (`"spec_state_replay_graph": true`, the default). Without it, the replay's launches cost about 5 ms per step with sampling, because sampling has already drained the GPU queue. One graph is captured per verify shape, the second time the shape runs; with the `{"4": 3, "5": 4}` width policy that is eight graphs (one to eight requests). Each capture takes 35–90 ms once, at the step that first meets the shape. The graphs added no measurable GPU memory (peak 31,564 MiB per GPU with them, 31,580 MiB without, in the same benchmark). With all eight captured, eight concurrent 96K-token requests completed without preemption at 27% peak KV use, and GPU memory peaked at 31,884 of 32,768 MiB per GPU.
 - Replay needs Model Runner V2, `--mamba-cache-mode align`, synchronous scheduling and no pipeline parallelism; elsewhere it stays off with a log line.
@@ -134,7 +139,7 @@ GDN speculative state replay keeps two recurrent-state blocks per running reques
   batch sizes.
 - English code and reasoning gain the most. Chinese is slower than MTP4 in almost every
   measured case: the drafter was trained on English. Very long prompts also favour MTP4.
-- With the recommended 0.1.3 settings (sharded selector codebooks, the new graph settings, state replay off), the measured GPU KV pool is 1,379,634 tokens, with a reported maximum concurrency of 2.63× at 512K; with state replay on it is 1,542,558 tokens (2.94×). The 0.1.2 DFlash2 configuration measured 1,268,831 tokens (2.42×); see [GPU KV capacity by method](performance.md#gpu-kv-capacity-by-method).
+- With the recommended 0.1.3 settings (sharded selector codebooks, the new graph settings, state replay off), the measured GPU KV pool is 1,379,634 tokens, with a reported maximum concurrency of 2.63× at 512K; with state replay on it is 1,542,558 tokens (2.94×). The 0.1.2 DFlash2 configuration measured 1,268,831 tokens (2.42×). In 0.1.4 with probabilistic drafting, the recommended setting, the pool is 1,366,121 tokens (2.61×); with greedy drafting it is unchanged at 1,379,634. See [GPU KV capacity by method](performance.md#gpu-kv-capacity-by-method).
 - On a replay of real agent sessions (one request at a time, sampling, long reasoning),
   DFlash2 and MTP4 were on par: 0.97× the MTP4 speed (95% interval 0.90–1.01). DFlash2
   was ahead on tool calls, answers and contexts up to 64K, and behind in reasoning, above

docs/performance.md

@@ -65,7 +65,8 @@ Measured on our 4x V100 PCIe deployment with TP4 + DCP2, `--gpu-memory-utilizati
 
 | Format | Measured version | Method | GPU KV capacity, tokens | Maximum concurrency at 512K |
 | --- | --- | --- | ---: | ---: |
-| Uncensored AWQ g32 + E4M3 KV | 0.1.3 | DFlash2 recommended (state replay off) | **1,379,634** | **2.63×** |
+| Uncensored AWQ g32 + E4M3 KV | 0.1.4 | DFlash2 recommended (probabilistic drafting, state replay off) | **1,366,121** | **2.61×** |
+| Uncensored AWQ g32 + E4M3 KV | 0.1.3 / 0.1.4 | DFlash2, greedy drafting, state replay off | 1,379,634 | 2.63× |
 | Uncensored AWQ g32 + E4M3 KV | 0.1.3 | DFlash2 with state replay | 1,542,558 | 2.94× |
 | Uncensored AWQ g32 + E4M3 KV | 0.1.2 | DFlash2 | 1,268,831 | 2.42× |
 | Uncensored AWQ g32 + E4M3 KV | 0.1.2 / 0.1.3 | MTP4 | 1,153,143 | 2.20× |