feat(pflash): Qwen3.5-0.8B PFlash runtime: scoring head, segment and request probes, strict budget selection, chat views, cache-aware admission, fallback

optimizations/kvflash/DESIGN.md

@@ -106,7 +106,7 @@ FA span traffic is bandwidth-realistic:
 ## Full LSA loop (drafter as Memory Indexer) — measured
 
 Test run F implements the paper's complete inference paradigm with the
-pflash drafter (Qwen3-0.6B, `/opt/lucebox/models/drafter/`) standing in
+pflash drafter (Qwen3.5-0.8B, `/opt/lucebox/models/drafter/`) standing in
 for the trained indexer: prompt (2048) larger than the pool (1024) so
 prefill itself evicts, then every τ=64 decoded tokens the drafter
 rescores the full sequence (tail attention = indexer query, chunk means
@@ -176,7 +176,7 @@ The pool is wired into the qwen35 backend behind `--kvflash <tokens>`
    pool (live LRU eviction mid-request). Coherent story end to end,
    36.9 tok/s, clean finish. Second request (per-request pager reset) ok.
 2. WITH pflash: `--kvflash 2048 --prefill-compression always
-   --prefill-threshold 256 --prefill-drafter <Qwen3-0.6B>`. Compression
+   --prefill-threshold 256 --prefill-drafter <Qwen3.5-0.8B>`. Compression
    1468 -> 60 tokens, then `[kvflash] drafter scorer attached (tau=64)`
    automatically; 400 coherent tokens answering from the compressed
    context. Same binary, zero pflash-specific configuration on the pool.
@@ -246,7 +246,7 @@ and masks through it. What differs per arch:
   3.09, identical text).
 
 Policy: drafter-scored residency is the default on all four archs. The
-server probes for the Qwen3-0.6B next to the model (or --prefill-drafter)
+server probes for the Qwen3.5-0.8B next to the model (or --prefill-drafter)
 and lazy-loads it at the first reselect; `--kvflash-policy lru` opts out.
 qwen35/qwen35moe feed the drafter target ids directly; laguna/gemma4 use
 KvFlashCrossTokScorer (detokenize -> re-tokenize -> score -> map back by

optimizations/kvflash/README.md

@@ -41,7 +41,7 @@ does not fit at all.)
 # recommended: drafter-scored residency, pool auto-sized from VRAM.
 # pass --prefill-drafter so the drafter is guaranteed (no silent LRU fallback).
 luce_server model.gguf --max-ctx 32768 --kvflash auto \
-    --prefill-drafter /opt/lucebox/models/drafter/Qwen3-0.6B-BF16.gguf
+    --prefill-drafter /opt/lucebox/models/drafter/Qwen3.5-0.8B-BF16.gguf
 
 # drop the path to auto-probe (model dir, drafter/, draft/, /opt/lucebox/models/drafter/);
 # falls back to LRU if none is found, so check the banner reads policy=drafter
@@ -52,7 +52,7 @@ luce_server model.gguf --max-ctx 32768 --kvflash 8192 --kvflash-policy lru
 ```
 
 Drafter-scored residency is the DEFAULT policy on every model family:
-the server probes for `Qwen3-0.6B-BF16.gguf` next to the model (same
+the server probes for `Qwen3.5-0.8B-BF16.gguf` next to the model (same
 dir, `drafter/`, `draft/`, then `/opt/lucebox/models/drafter/`) and
 lazy-loads it on the first reselect; `--prefill-drafter` overrides the
 location, prefill compression can stay off either way. Qwen-family
@@ -147,7 +147,7 @@ Env: `LUCE_KVFLASH_POLICY=qk`. Bench: `test_kvflash --qkbench`.
 - `server/src/common/kvflash_qk.h` — target-QK scorer: pure scoring math
   (unit-tested in `server/test/unit/test_kvflash_qk.cpp`), seal-time key pooling,
   `KvFlashTargetQkScorer`
-- `server/src/qwen3/qwen3_kvflash_scorer.{h,cpp}` — pflash-drafter scorer
+- `server/src/pflash/kvflash_drafter_scorer.{h,cpp}` — pflash-drafter scorer
   (tail attention; bisects on allocation pressure)
 - `server/src/qwen35/*` — cache `ctx_alloc`, masked pooled decode, slot-mapped
   spec verify, daemon flags

optimizations/pflash/README.md

@@ -39,7 +39,7 @@ Long-context prefill is O(S²): vanilla llama.cpp on a single RTX 3090 takes **~
 **What was missing:** no implementation that sits in front of a quantized GGUF target on a 24 GB card without dragging Python+Triton into the runtime path. PFlash is that:
 
 - C++/CUDA daemon-resident drafter + scoring + target generation, all in one process, one ggml allocator.
-- Custom Qwen3-0.6B BF16 forward (`qwen3_0p6b_loader.cpp` + `qwen3_0p6b_graph.cpp`) — no libllama.
+- Custom Qwen3.5-0.8B BF16 forward (`src/pflash/qwen35_loader.cpp` + the qwen35 target graph) — no libllama.
 - 4 CUDA kernels for the FlashPrefill `mean_K → score → select → sparse_fwd` algorithm (`flashprefill_kernels.cu`).
 - BSA ([mit-han-lab/Block-Sparse-Attention](https://github.com/mit-han-lab/Block-Sparse-Attention), FA-2 derived, sm_80+) for the long-context drafter forward, wired without `libtorch` via 3 ATen/c10 header stubs (`server/deps/bsa_stubs/`).
 - 128K → 2.6K span selection at `keep_ratio=0.05`, NIAH retrieved at every measured context, decode ~74 tok/s downstream.
@@ -72,21 +72,21 @@ cmake --build server/build --target test_dflash test_flashprefill_kernels -j
 
 # 2. fetch weights (target + spec-decode draft + drafter scorer)
 uv run hf download unsloth/Qwen3.6-27B-GGUF Qwen3.6-27B-Q4_K_M.gguf --local-dir server/models/
-uv run hf download Qwen/Qwen3-0.6B model.safetensors tokenizer.json --local-dir server/models/drafter/
+uv run hf download Qwen/Qwen3.5-0.8B model.safetensors tokenizer.json --local-dir server/models/drafter/
 uv run hf download z-lab/Qwen3.6-27B-DFlash model.safetensors --local-dir server/models/draft/
 
-# 2b. convert the drafter (Qwen3-0.6B HF) to a BF16 GGUF for the C++ scorer.
+# 2b. convert the drafter (Qwen3.5-0.8B HF) to a BF16 GGUF for the C++ scorer.
 #     The submodule already vendors llama.cpp at deps/llama.cpp.
 uv run python server/deps/llama.cpp/convert_hf_to_gguf.py server/models/drafter \
-       --outtype bf16 --outfile server/models/Qwen3-0.6B-BF16.gguf
+       --outtype bf16 --outfile server/models/Qwen3.5-0.8B-BF16.gguf
 
 # 3. generate NIAH cases + run head-to-head bench against the C++ daemon
 uv run --directory pflash python tests/niah_gen.py --n 1 --ctx 131072 --out /tmp/niah_128k.jsonl
 uv run --directory pflash python tests/bench_niah_cpp.py \
   --bin    ../server/build/test_dflash \
   --target ../server/models/Qwen3.6-27B-Q4_K_M.gguf \
   --draft-spec ../server/models/draft/model.safetensors \
-  --drafter-gguf ../server/models/Qwen3-0.6B-BF16.gguf \
+  --drafter-gguf ../server/models/Qwen3.5-0.8B-BF16.gguf \
   --cases  /tmp/niah_128k.jsonl --keep-ratio 0.05 --n-gen 256
 ```
 
@@ -97,10 +97,12 @@ For an OpenAI-compatible server with transparent compression on long prompts, ru
 | Flag | Choices / type | Default | Effect |
 |---|---|:---:|---|
 | `--prefill-compression` | `off` / `auto` / `always` | `off` | When to run pflash. `auto` compresses when total prompt ≥ threshold; `always` compresses every request. |
-| `--prefill-threshold` | int (tokens) | `32000` | Token threshold for `auto` mode. |
-| `--prefill-keep-ratio` | float `(0, 1]` | `0.05` | Fraction of source tokens to keep after compression. `0.02` for 128K, `0.10` for 32K. |
-| `--prefill-drafter` | path to `.gguf` | required when not `off` | Drafter weights (Qwen3-0.6B BF16 GGUF). |
-| `--prefill-drafter-tokenizer` | HF repo id | `Qwen/Qwen3-0.6B` | HF tokenizer for the drafter vocab. |
+| `--prefill-threshold` | int (tokens) | `32000` | Token threshold for `auto` mode. `16384` with a trained scoring head. |
+| `--prefill-keep-ratio` | float `(0, 1]` | `0.05` | Fraction of source tokens to keep after compression. `0.02` for 128K, `0.10` for 32K. With a trained scoring head and no ratio flag, the release curve `16384:0.315 32768:0.270 65536:0.270 120000:0.225` applies. |
+| `--prefill-drafter` | path to `.gguf` | required when not `off` | Drafter weights (Qwen3.5-0.8B BF16 GGUF). |
+| `--prefill-drafter-tokenizer` | HF repo id | `Qwen/Qwen3.5-0.8B` | HF tokenizer for the drafter vocab. |
+
+With `PFLASH_SCORING_HEAD_GGUF` set, every unset PFlash setting takes the measured release profile; see [PFlash configuration](../../server/docs/PFLASH_CONFIGURATION.md#release-profile).
 
 When `--prefill-compression != off`, the server auto-sets `LUCE_LM_HEAD_FIX=0` and `LUCE_FA_WINDOW=0` (matching the bench harness — needed so the post-compress draft graph fits on a 24 GB card without OOM).
 
@@ -111,7 +113,7 @@ When `--prefill-compression != off`, the server auto-sets `LUCE_LM_HEAD_FIX=0` a
   --prefill-compression auto \
   --prefill-threshold 4096 \
   --prefill-keep-ratio 0.02 \
-  --prefill-drafter server/models/Qwen3-0.6B-BF16.gguf
+  --prefill-drafter server/models/Qwen3.5-0.8B-BF16.gguf
 ```
 
 Below the threshold the server runs the standard target generate (no compression). Above it, the server transparently runs `compress` on the daemon, swaps the prompt for the compressed text, and continues the normal `/v1/chat/completions` flow. Tool-calling requests (`req.tools` non-empty) skip compression so JSON tool definitions stay intact.
@@ -155,7 +157,7 @@ prompt (≤ 128K tokens)
    ▼
 ┌──────────────────────────────────────────────┐
 │  drafter (in-process)                        │
-│   custom Qwen3-0.6B BF16 forward in ggml     │
+│   custom Qwen3.5-0.8B BF16 forward in ggml   │
 │   FlashPrefill block-sparse via BSA (≥ 32K)  │
 │   tail-attention scoring → score [S]         │
 │   chunk(128) + alpha-threshold → top blocks  │
@@ -177,7 +179,7 @@ prompt (≤ 128K tokens)
 └──────────────────────────────────────────────┘
 ```
 
-**Drafter forward.** Custom Qwen3-0.6B graph (`qwen3_0p6b_graph.cpp`) per-layer A/FP/B blocks: dense attention up to ~32K source, FlashPrefill sparse attention at and above. The 4 FP kernels live in `flashprefill_kernels.cu`; BSA dispatch is in `bsa_launcher.cu` + `bsa_fwd_inst.cu`.
+**Drafter forward.** The Qwen3.5-0.8B drafter (`src/pflash/qwen35_drafter.cpp` on the qwen35 target graph) runs the model's first fifteen blocks and scores the context with block 15's NoPE Q/K attention-mass head; `PFLASH_QWEN35_LEGACY_SCORER=1` selects the all-layer running-max scorer instead.
 
 **Scoring + selection.** Tail attention `Q[-N:] @ K^T / sqrt(d)` per layer/head, max over (L, H), mean over the tail window. Block-level threshold by `alpha * mean(scores)` selects which K-blocks each Q-block attends to. Configurable via `LUCE_FP_ALPHA`.
 
@@ -205,14 +207,14 @@ What we built:
 
 - C++/CUDA port of the FlashPrefill algorithm: 4 kernels (`mean_K / score / select / sparse_fwd`), no Triton dependency.
 - BSA ([mit-han-lab/Block-Sparse-Attention](https://github.com/mit-han-lab/Block-Sparse-Attention)) wired without `libtorch` via 3 ATen/c10 header stubs (`server/deps/bsa_stubs/`).
-- Custom Qwen3-0.6B BF16 forward so the drafter runs through the same ggml allocator as the 27B target.
+- Custom Qwen3.5-0.8B BF16 forward so the drafter runs through the same ggml allocator as the 27B target.
 - Daemon stdin protocol (`compress` / `generate` / `park` / `unpark` / `free drafter`) so target + drafter coexist on a 24 GB card.
 - NIAH harness against `llama-bench` for end-to-end validation.
 
 ## Scope and limits
 
 - **Single 24 GB GPU** target (RTX 3090 reference). On 32+ GB cards, drafter + target can coexist and the park/unpark dance disappears.
-- **Qwen3.6-27B Q4_K_M target + Qwen3-0.6B drafter** is the validated pair. Other targets/drafters need keep_ratio + alpha re-calibration.
+- **Qwen3.6-27B Q4_K_M target + Qwen3.5-0.8B drafter** is the validated pair. Other targets/drafters need keep_ratio + alpha re-calibration.
 - **NIAH single-needle** is the only retrieval task validated end-to-end. Multi-doc QA, long-form code retrieval, etc. still TBD.
 - **sm_80+** required for BSA (RTX 3090 sm_86 is the reference). On sm_75 (Turing) the build auto-disables BSA and falls back to the WMMA path; expect a slower drafter forward at long ctx.
 
@@ -242,16 +244,16 @@ These are operator-side flags on the launcher; they do not change
 PFlash semantics. A short prompt lane should keep the original
 defaults.
 
-### Drafter selection: BF16 Qwen3-0.6B for compress
+### Drafter selection: BF16 Qwen3.5-0.8B for compress
 
 PFlash compress benefits from a small, fast drafter. The validated
-choice is **Qwen3-0.6B** in **BF16 safetensors** with ~5 attention
+choice is **Qwen3.5-0.8B** in **BF16 safetensors** with ~5 attention
 layers. The DFlash drafter for the same target works correctly
 during decode-after-unpark but is heavier than ideal for compress.
 
 Practical guidance:
 
-- Use Qwen3-0.6B BF16 for `compress` (PFlash side).
+- Use Qwen3.5-0.8B BF16 for `compress` (PFlash side).
 - Reuse the larger DFlash drafter for `decode` after unpark
   (DFlash side).
 
@@ -262,7 +264,7 @@ simultaneously on a 24 GB GPU.
 
 Reproducible comparison vs Ollama native `/api/chat` on the same
 64K unique-prompt summary task, RTX 6000 Ada sm_89,
-Qwen3.6-27B-Q4_K_M, FA_WINDOW=0. Drafter setup: Qwen3-0.6B BF16
+Qwen3.6-27B-Q4_K_M, FA_WINDOW=0. Drafter setup: Qwen3.5-0.8B BF16
 GGUF for the PFlash compress path (see "Drafter selection" above);
 the larger DFlash drafter on the luce daemon side ran as FP16
 safetensors during decode-after-unpark on this run. Feel free to

optimizations/pflash/pflash/dflash_client.py

@@ -230,7 +230,7 @@ def park_target(self):   self._send("park target\n")
     def unpark_target(self): self._send("unpark target\n")
 
     def compress(self, prompt_ids: list[int], keep_ratio: float, drafter_gguf: str,
-                 drafter_arch: str = "qwen3-0.6b") -> list[int]:
+                 drafter_arch: str = "qwen35-0.8b") -> list[int]:
         """C++ drafter score+compress via daemon. Returns compressed token ids.
 
         Daemon command: compress <bin> <keep_x1000> <drafter_gguf> <drafter_arch>

optimizations/pflash/tests/bench_niah_cpp.py

@@ -27,12 +27,12 @@ def main():
     ap.add_argument("--target", default="/opt/lucebox/models/Qwen3.6-27B-Q4_K_M.gguf")
     ap.add_argument("--draft-spec", default="/home/lucebox/lucebox-hub/server/models/draft/model.safetensors",
                     help="draft model used for spec decoding (NOT drafter scorer)")
-    ap.add_argument("--drafter-gguf", default="/home/lucebox/lucebox-hub/server/models/Qwen3-0.6B-BF16.gguf",
-                    help="C++ drafter scorer GGUF (Qwen3-0.6B BF16)")
-    ap.add_argument("--drafter-arch", default="qwen3-0.6b", choices=["qwen3-0.6b", "qwen35-0.8b"],
+    ap.add_argument("--drafter-gguf", default="/home/lucebox/lucebox-hub/server/models/Qwen3.5-0.8B-BF16.gguf",
+                    help="C++ drafter scorer GGUF (Qwen3.5-0.8B BF16)")
+    ap.add_argument("--drafter-arch", default="qwen35-0.8b", choices=["qwen35-0.8b"],
                     help="C++ drafter architecture selector")
     ap.add_argument("--target-tokenizer", default="Qwen/Qwen3.6-27B")
-    ap.add_argument("--drafter-tokenizer", default="Qwen/Qwen3-0.6B")
+    ap.add_argument("--drafter-tokenizer", default="Qwen/Qwen3.5-0.8B")
     ap.add_argument("--max-ctx", type=int, default=16384,
                     help="daemon KV cache max ctx; sized for compressed prompt+gen, NOT source")
     ap.add_argument("--keep-ratio", type=float, default=0.020)

optimizations/pflash/tests/niah_gen.py

@@ -110,7 +110,7 @@ def main():
     # Default matches bench_niah_cpp.py's --drafter-tokenizer, since that is
     # the tokenizer the downstream NIAH bench uses to size case["prompt"]
     # for the drafter forward. Override for any other harness.
-    ap.add_argument("--tokenizer", default="Qwen/Qwen3-0.6B")
+    ap.add_argument("--tokenizer", default="Qwen/Qwen3.5-0.8B")
     args = ap.parse_args()
     tok = AutoTokenizer.from_pretrained(args.tokenizer)