feat(qwen3): add DFlash2 Phase 1 path selector

[RicardoMin]

Phase1 of #930

Summary

This PR implements Phase 1 of DFlash2 for Qwen3: a proposer-side top-16 path selector layered on the existing batched DFlash backbone.

The selector keeps the existing draft/verify boundary unchanged. For each draft position, it extracts the deterministic top-16 candidates and performs a bounded greedy path walk using the selector's low-rank predecessor/successor scoring.

Changes

• Normalize legacy DFlash, DSpark, and native DFlash2 configuration schemas.
• Detect the DFlash2 top-16 selector capability and validate its configuration.
• Load and validate the BF16 hidden projection, predecessor codebook, and successor codebook tensors.
• Add CUDA top-16 candidate extraction with deterministic token-id tie breaking.
• Add the request-local bounded path walk over the candidate sets.
• Add persistent selector scratch buffers and account for selector memory reservation.
• Preserve the existing anchor-drop draft layout and reconstruct the unchanged draft result contract.
• Dispatch the selector only for selector-enabled drafters.
• Keep legacy DFlash and DSpark dispatch paths intact.
• Fail closed for capabilities that are outside Phase 1.

The target verification path, KV commit/rollback transaction, acceptance logic, and CUDA Graph shapes are unchanged.

Verification

Environment:

• Linux GPU host
• CUDA target: sm_89
• Target model: Qwen3-4B
• Legacy regression drafter: Qwen3-4B-DFlash-b16

Check	Result
cargo fmt --all -- --check	Passed
git diff --check	Passed
cargo check --release -p pegainfer-qwen3 --tests	Passed
cargo build --release -p pegainfer-server --bin pegainfer	Passed
DFlash configuration compatibility test	1 passed, 0 failed; no skipped configuration test
cargo test --release -p pegainfer-qwen3 --lib	88 passed, 0 failed
dflash_speculative_gate	4 passed, 0 failed
Legacy DFlash single-stream A/B, batch size 1	Plain decode: 29.7 tok/s; DFlash: 59.7 tok/s; observed speedup: 2.01x

The greedy gate covers existing DFlash losslessness and regression cases including graph-shape transitions, heterogeneous batches, and insufficient draft headroom.

Supplementary shared-path checks

These checks use the existing legacy DFlash path and are included as supplementary regression evidence only:

Check	Result
Sampled smoke, 8 runs, batch size 2	1024 cells; BH rejections: 0
Sampled equivalence, 64 runs, batch size 4	1024 cells; BH rejections: 0
Sampled null check	1024 cells; BH rejections: 0
Seeded determinism	1 passed, 0 failed

For the sampled checks, the gate is zero Benjamini-Hochberg rejections at the configured FDR threshold. The reported maximum total-variation values are diagnostic effect sizes from sparse high-vocabulary samples, not pass/fail criteria.

Scope and Limitations

The only available native Qwen3-4B-DFlash2 checkpoint also declares Phase 2 capabilities:

• conv_kernel_size=2
• conv_group_size=16
• sliding-window attention

Phase 1 detects these capabilities and rejects the checkpoint before draft-weight upload with:

DFlash2 dynamic convolution (kernel_size=2, group_size=16) is not supported in Phase 1.
This is an expected fail-closed boundary check, not a native DFlash2 end-to-end success.

Therefore, this PR does not claim native DFlash2 throughput or acceptance improvements. 
Direct native DFlash2 end-to-end validation requires a Phase 1-compatible selector-only checkpoint.
Dynamic convolution and sliding-window execution remain Phase 2 work, while sampled rejection sampling remains Phase 3 work.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 29ab0bef35

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[RicardoMin]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9ce2357568

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Verify selector outputs, not only final losslessness

When the selector returns any incorrect but in-vocabulary sequence—for example from broken row mapping or codebook scoring—the target verifier rejects those drafts and still produces the plain greedy continuation, so this check_lossless assertion passes while the new selector has zero useful acceptance. Fresh evidence since the earlier review is that the newly added native gate inspects only final verified tokens and never compares selected IDs with a reference implementation or checks acceptance; add such a comparison so the core feature is actually validated.

AGENTS.md reference: AGENTS.md:L121-L123

Useful? React with 👍 / 👎.