feat(MODEL-MM-dots3-note): W4b-2 — the sliding arm on the decode path (#699)

docs/FEATURES.md

@@ -150,7 +150,7 @@ speed-pending, which [BENCHMARKS.md](BENCHMARKS.md) tracks.
 | `LagunaForCausalLM` | poolside/Laguna-S-2.1-NVFP4, GGUF-Q4_K, Laguna-XS | byte-exact near-tie (distributional vs vLLM) | vLLM parity+ 1.03x, default on, via the `laguna-gen` CLI; the registered engine forward VT_CHECKs non-bf16 (`ARCH-ONE-SURFACE` fold) |
 | `KimiLinearForCausalLM` | Kimi-Linear-48B-A3B (KDA + NoPE-MLA + MoE) | **Folded onto the shared paged runner (ROW 7 §21, #122): engine==CLI 128/128 byte-identical; vs golden 122/128 (the intrinsic near-tie profile); FA2 paged MLA default-ON; SACRED post-fold green** | Served via `vllm_engine_load` + `vllm_complete_tokens` (ABI v13); server 19.0 tok/s wall vs vLLM ~21 (~0.90×), speed residual open |
 | `KimiK3ForConditionalGeneration` | Kimi-K3 (2.8T MoE) | scaffold: registry+config+enumeration gated, forward refuses | HW-infeasible (~1.56 TB); no run |
-| `Dots3NoteForCausalLM` | `dots-studio/dots3-note-prev` @`1e1e7b0c` (280B-A16B multimodal MoE, ~576 GB bf16; the `-fp8` sibling is ~290 GB). Headers only — no tensor byte downloaded | W1+W2 scaffold: registry + config gated off the REAL released `config.json`, with one assertion per §4 config trap (ungrouped 1/1 router, GPT-J indexer RoPE, one nextn layer, the two LoRA rescales, the sliding theta); name map accounted **38006/38006** over the WHOLE released index — 35381 language, 2195 vision, 430 audio, with the two tower files carried as named W6/W7 deferrals rather than dropped; load, GGUF and forward all REFUSE BY NAME | **No oracle, on any host we own** (~290 GB fp8 against a 122 GiB ceiling), so NO number is claimable on any axis and the e2e gate is an open gap by construction ([spec](../.agents/specs/dots3-note.md) §6.4, #699) |
+| `Dots3NoteForCausalLM` | `dots-studio/dots3-note-prev` @`1e1e7b0c` (280B-A16B multimodal MoE, ~576 GB bf16; the `-fp8` sibling is ~290 GB). Headers only — no tensor byte downloaded | W1+W2 scaffold: registry + config gated off the REAL released `config.json`, with one assertion per §4 config trap (ungrouped 1/1 router, GPT-J indexer RoPE, one nextn layer, the two LoRA rescales, the sliding theta); name map accounted **38006/38006** over the WHOLE released index — 35381 language, 2195 vision, 430 audio, with the two tower files carried as named W6/W7 deferrals rather than dropped; W4a+W4b-2 put BOTH attention geometries on the DECODE PATH, reached through `ModelRegistry::Forward`: the 13 full-attention layers with the two LoRA rescales, `k_rope_only_layernorm` and the headwise gate, and the 33 sliding-window layers over a PADDED 1088-wide MLA cache row that each layer narrows to its own logical width on read; `vt::MlaDecodeAttention` and `vt::MlaPrefillAttention` grew an optional window whose absent state is bit-identical to no window. The RELEASED checkpoint still REFUSES BY NAME at its first MoE layer (W5), and so do GGUF, the nextn tail (W10) and both towers (W6/W7) | **No oracle, on any host we own** (~290 GB fp8 against a 122 GiB ceiling), so NO number is claimable on any axis and the e2e gate is an open gap by construction ([spec](../.agents/specs/dots3-note.md) §6.4, #699) |
 | `NemotronHForCausalLM` | Nemotron-3.5-Lightning-30B-A3B-NVFP4 (`nvidia` @`29f2d174`) | config+enumeration+KV-shape gated; hybrid forward COMPUTES; loader materializes 18487/18487 as SHIPPED; A3 e2e gate 96/96 `STRICT PASS` on GB10 at `0ea5d249f` (#1221); NO run against current `main` | **PAGED (#810 A2-P): K/V go to the runner's pages; conv+SSM rows carry at the metadata's state indices.** G-SAFE: `num_reqs <= 1`. Device `lm_head` (A2-Q2b), UNMEASURED. Owed: FP8 mamba (A2-Q1), MTP, GGUF |
 | `MuseGlimmerForCausalLM` | real tensors, **bf16 depth 4/52 only**: 5 prefill argmax positions match a torch transcription of vllm#51655 and HF. GGUF full depth generates coherently (#347, #359) but is **NOT token-exact** | text forward + loader vs an fp32 reference, per-mechanism property tests, scaffold 11/11, GGUF gate 17/17. An ABSENT config key now takes the architecture's constant (#412): GGUF post-norms ran at 1e-5, not 1e-8 | no vLLM denominator (pin cannot load it); SECONDARY llama.cpp, same GGUF, GB10 CPU: prefill tie **0.997x**, decode 0.232x, RSS 1.92x (#333) |
 | `MuseGlimmerForConditionalGeneration` | vision: **no reference run of any kind**; enumeration gated vs the released 30B index (1436/1436). Image/video need bf16 safetensors: `mmproj-kquant.gguf` is refused by name | perception encoder loaded and wired, so an image or video prompt runs; `perception_emb_norm` now armed by default (#405). Reachability plus placeholder scatter only, no image or video correctness | not measurable; anchored to open vllm#51655 |

include/vllm/model_executor/layers/attention/mla_chunked_context.h

@@ -320,6 +320,13 @@ inline void ComputeMlaPrefillContext(vt::Queue& q, const vt::Tensor& query,
 // The final merge is PREFIX = context, SUFFIX = new tokens (:2413-2420) — the
 // order matters, and it carries `prefill_tokens_with_context` so query rows
 // belonging to context-free requests take the suffix verbatim.
+//
+// `sliding_window` (dots3-note W4b-2, #699) is 0 for every DeepSeek / MiniCPM3 /
+// Kimi-Linear caller, which leaves `window_size` at `std::nullopt` and this
+// function byte-identical. > 0 is `sliding_window_size`, and it reaches the
+// new-tokens call as the `AttentionWindow{W - 1, 0}` pair upstream hands
+// FlashAttention (`vllm/models/dots3_note/nvidia/attention.py:300` @
+// `bc2d63e650`).
 inline void ForwardMlaPrefillMha(vt::Queue& q, vt::Tensor& output, const vt::Tensor& query,
                                  const vt::Tensor& key, const vt::Tensor& value,
                                  const vt::Tensor& kv_cache, const vt::Tensor& block_table,
@@ -328,8 +335,22 @@ inline void ForwardMlaPrefillMha(vt::Queue& q, vt::Tensor& output, const vt::Ten
                                  const MlaUpProjectFn& up_project, float scale,
                                  int32_t max_query_len, int32_t prefill_tokens_with_context,
                                  MlaPrefillContextBuffers& bufs, vt::Tensor& suffix_output,
-                                 vt::Tensor& suffix_lse) {
+                                 vt::Tensor& suffix_lse, int64_t sliding_window = 0) {
   const bool has_context = !chunks.empty();
+  // A windowed prefill that ALSO has chunked context has no upstream form to
+  // mirror: a sliding layer gathers only `min(seq_len, query_len + W - 1)` keys
+  // and runs one varlen call per request group (attention.py:206, :594-654), so
+  // the LSE merge below never runs windowed upstream. Refuse rather than merge
+  // an unwindowed context into a windowed suffix, which is a silently wrong
+  // answer and exactly the class this row keeps naming.
+  if (sliding_window > 0 && has_context) {
+    throw std::invalid_argument(
+        "MLA prefill: a SLIDING-WINDOW layer with chunked CONTEXT is not ported. "
+        "Upstream's windowed prefill caps the gather at the window instead of "
+        "merging context chunks (dots3-note attention.py:206, :594-654), so there "
+        "is no windowed form of this merge to mirror. See "
+        ".agents/specs/dots3-note.md `## Owed` and issue #699.");
+  }
 
   // ":2381-2392" — the causal pass over the new tokens. `return_softmax_lse` is
   // True exactly when there is context to merge with (:2385).
@@ -338,6 +359,9 @@ inline void ForwardMlaPrefillMha(vt::Queue& q, vt::Tensor& output, const vt::Ten
   args.causal = true;
   args.max_seqlen_q = max_query_len;
   args.max_seqlen_k = max_query_len;
+  if (sliding_window > 0) {
+    args.window_size = vt::AttentionWindow{static_cast<int32_t>(sliding_window - 1), 0};
+  }
   vt::Tensor& new_out = has_context ? suffix_output : output;
   vt::MlaPrefillAttention(q, new_out, has_context ? &suffix_lse : nullptr, query, key, value,
                           cu_seqlens_q, cu_seqlens_q, args);

include/vllm/model_executor/models/mla_attention.h

@@ -155,6 +155,39 @@ struct MlaBlockDims {
   // when `has_q_lora()` is false rather than dropping it silently.
   double kv_lora_scale = 1.0;
 
+  // ─── dots3-note's SLIDING WINDOW (W4b-2, #699) ────────────────────────────
+  // 0 is the ABSENT state and it is a NOT-TAKEN branch, not a wide window: at
+  // 0 no `window_size` reaches `vt::MlaDecodeAttention` or
+  // `vt::MlaPrefillAttention` at all, both keep `std::nullopt`, and their
+  // loops keep the full-context bounds they had. Every DeepSeek / MiniCPM3 /
+  // Kimi-Linear registration leaves it 0.
+  //
+  // > 0 is `sliding_window_size` — 513 on dots3-note's 33 `sliding_attention`
+  // layers (`vllm/models/dots3_note/nvidia/model.py:456` passes it to
+  // `MLAAttention`; `attention.py:439-468` @ `bc2d63e650` is the impl subclass
+  // that keeps it). It reaches the two ops as `AttentionWindow{W - 1, 0}`,
+  // which is literally the pair upstream hands FlashAttention on the prefill
+  // half (`attention.py:300`) and exactly the key set its decode mask keeps
+  // (`:151-152`).
+  //
+  // WHAT IT DOES NOT COVER, and the seam refuses it BY NAME rather than
+  // serving a wrong answer: a windowed prefill that also has CHUNKED CONTEXT.
+  // Upstream never builds one — a sliding layer's prefill gathers only
+  // `min(seq_len, query_len + W - 1)` keys and runs ONE varlen call per chunk
+  // of requests (`attention.py:206, :594-654`), so the chunked-context merge
+  // this seam inherits from `DeepseekV2` has no windowed counterpart upstream
+  // to mirror. Owed to the row; see `.agents/specs/dots3-note.md` `## Owed`.
+  //
+  // There is deliberately NO `has_sliding_window()` accessor beside this field,
+  // unlike `has_q_lora()` below. Every consumer wants the VALUE, not the
+  // predicate: `ForwardMlaAttentionBlock` passes `dims.sliding_window` to
+  // `ForwardMlaPrefillMha` and assigns it to `impl.sliding_window`
+  // unconditionally, precisely so a 0 cannot be skipped and leave a previous
+  // layer's 513 in place. W4b-2 shipped the accessor with no caller in `src`,
+  // `include` or `tests`, and its review removed it under `## Nothing lands
+  // dead` rather than inventing a call site for it.
+  int64_t sliding_window = 0;
+
   // `self.qk_head_dim = qk_nope_head_dim + qk_rope_head_dim` (:969) — 192.
   int64_t qk_head_dim() const { return qk_nope_head_dim + qk_rope_head_dim; }
   // The MLA cache head_size `kv_lora_rank + qk_rope_head_dim`

include/vllm/v1/attention/backend.h

@@ -644,6 +644,24 @@ class TritonMLAImpl final : public AttentionImpl {
   // runner's queue in when the DeepSeek-V2 forward lands.
   vt::Queue* queue = nullptr;
 
+  // ─── the SLIDING-WINDOW arm (dots3-note W4b-2, #699) ──────────────────────
+  // Upstream expresses this as a SUBCLASS: `Dots3NoteTritonMLAImpl(
+  // TritonMLAImpl)` passes `sliding_window=None` up to the base — so the base's
+  // own rejection at `triton_mla.py:165-171` still fires for everyone else —
+  // and keeps the value on itself as `self.sliding_window`
+  // (`vllm/models/dots3_note/nvidia/attention.py:439-468` @ `bc2d63e650`),
+  // which `_forward_swa_mqa` (`:470-563`) then reads. This tree has one MLA
+  // impl and a registry that hands it out, so the subclass is expressed as a
+  // FIELD whose absent state is 0 — the additive shape used for every other
+  // family-specific MLA extension here.
+  //
+  // 0 leaves `MlaDecodeAttentionArgs::window_size` at `std::nullopt`, i.e. the
+  // full-context decode every DeepSeek / MiniCPM3 / Kimi-Linear caller gets.
+  // `layer.window_size` — the per-LAYER window on `AttentionLayer` — stays
+  // REFUSED in `forward_mqa`, because that is the base class's rejection and
+  // dots3-note does not set it either.
+  int64_t sliding_window = 0;
+
   // The DECODE entry point — the 1:1 counterpart of `forward_mqa`
   // (triton_mla.py:189-260). `q` is the already-concatenated
   // [num_reqs, num_heads, kv_lora_rank + qk_rope_head_dim] query (upstream

include/vt/ops.h

@@ -1401,6 +1401,34 @@ struct MlaDecodeAttentionArgs {
   // used to derive `num_kv_splits` when that is 0; an upper bound is safe. When
   // both are 0 the impl falls back to 1 split.
   int32_t max_seq_len = 0;
+  // ─── the SLIDING-WINDOW decode arm (dots3-note W4b-2, #699) ───────────────
+  // OPTIONAL local-attention bounds, the same `AttentionWindow` convention
+  // `PagedAttentionArgs::window_size` already uses: for the bottom-right
+  // aligned absolute query position `p = seq_len - 1` (MLA decode is ONE query
+  // per row), visible keys are `[p - left, p + right]` intersected with
+  // `[0, seq_len)`. `std::nullopt` — every DeepSeek / MiniCPM3 / Kimi-Linear
+  // registration — leaves the full-context loop byte-identical: the window is
+  // not a mask applied afterwards, it is the loop's START BOUND, so an absent
+  // window is a NOT-TAKEN branch rather than a no-op.
+  //
+  // UPSTREAM. `TritonMLAImpl` itself REJECTS a sliding window
+  // (triton_mla.py:165-171); dots3-note SUBCLASSES it —
+  // `Dots3NoteTritonMLAImpl.__init__` passes `sliding_window=None` to super and
+  // keeps the value on itself (`vllm/models/dots3_note/nvidia/attention.py`
+  // :439-468 @ `bc2d63e650`), then `_forward_swa_mqa` (`:470-563`) gathers a
+  // window-bounded slice of the paged latent and masks the scores with
+  // `kv_positions >= query_position - WINDOW_SIZE + 1` (`:152`) and
+  // `kv_positions <= query_position` (`:151`). `WINDOW_SIZE` is
+  // `sliding_window_size` = 513, so `left == sliding_window - 1`, matching the
+  // `window_size=(sliding_window - 1, 0)` upstream hands FlashAttention on the
+  // PREFILL half (`:300`). The gather is upstream's Triton WORKSPACE strategy;
+  // the paged kernels here read the block table directly over the same key
+  // range, which is the same function with no gather and no mask.
+  //
+  // `right` must be 0: an MLA decode query IS the last position of its own
+  // sequence, so a positive right bound could only admit keys that do not
+  // exist. Anything else is refused BY NAME in ops.cpp rather than ignored.
+  std::optional<AttentionWindow> window_size = std::nullopt;
 };
 
 // Arguments for vt::MlaPrefillAttention (MLA campaign W5). Mirrors the scalar
@@ -1423,6 +1451,26 @@ struct MlaPrefillAttentionArgs {
   // same fallback the FA-2 paged prefill launcher uses.
   int32_t max_seqlen_q = 0;
   int32_t max_seqlen_k = 0;
+  // ─── the SLIDING-WINDOW prefill arm (dots3-note W4b-2, #699) ──────────────
+  // OPTIONAL local-attention bounds, the `AttentionWindow` convention. Query
+  // `i` of a request whose query length is `Lq` and key length is `Lk` sits at
+  // the bottom-right aligned position `p = i + (Lk - Lq)`; visible keys are
+  // `[p - left, p + right]` intersected with `[0, Lk)`.
+  //
+  // UPSTREAM is literally this pair: `Dots3NoteFlashAttnPrefillBackend.
+  // run_sliding_window` calls `_flash_attn_varlen_diff_headdims(..., causal=
+  // True, window_size=(sliding_window - 1, 0))`
+  // (`vllm/models/dots3_note/nvidia/attention.py:279-305` @ `bc2d63e650`, the
+  // window at `:300`), so `left == sliding_window - 1` and `right == 0`.
+  //
+  // `causal` must be TRUE whenever this is set, and `right` must be 0. Both are
+  // refused BY NAME in ops.cpp rather than approximated: FlashAttention's local
+  // mask REPLACES the causal specialization (the adapter normalizes
+  // `is_causal = causal && !is_local`, cuda_flash_attn_fa2.cu:472-476), so a
+  // non-causal window would have to be spelled with an infinite right bound,
+  // which this struct cannot say. Upstream never asks for one — every windowed
+  // call it makes is the causal `(W-1, 0)` pair above.
+  std::optional<AttentionWindow> window_size = std::nullopt;
 };
 
 // Router SCORING function. softmax over all E is the Qwen3.6 / DeepSeek-V2

src/vllm/model_executor/layers/attention/mla_attention.cpp

@@ -124,6 +124,15 @@ void MlaBlockDims::Validate() const {
         "q_a_layernorm output to rescale on the DIRECT q_proj branch "
         "(deepseek_v2.py:1028-1034)");
   }
+  // dots3-note's sliding window (#699 W4b-2). 0 is ABSENT; a negative value is
+  // a caller that computed `sliding_window - 1` one layer too early, which
+  // would otherwise reach the ops as a window that admits nothing.
+  if (sliding_window < 0) {
+    throw std::invalid_argument(
+        "MlaBlockDims: sliding_window must be >= 0 (0 means ABSENT — the full "
+        "context; dots3-note's sliding layers set `sliding_window_size` 513, "
+        "model.py:456)");
+  }
 }
 
 // mla_attention.py:880-900 + :959-962. Upstream's chain is
@@ -546,11 +555,16 @@ void ForwardMlaAttentionBlock(Dev d, const MlaBlockDims& dims, const MlaBlockWei
     MlaUpProjectFn up = MakeMlaUpProjectFn(d, dims, w, up_scratch);
     Tensor suffix_out_t = suffix_out.t(), suffix_lse_t = suffix_lse.t();
     Tensor kv_cache_ro = kv_cache;
+    // dots3-note's sliding layers (#699 W4b-2). 0 — every DeepSeek / MiniCPM3 /
+    // Kimi-Linear registration — leaves the call byte-identical; > 0 becomes
+    // the `(W - 1, 0)` FlashAttention pair upstream's `run_sliding_window`
+    // passes (attention.py:300 @ bc2d63e650), and refuses a windowed prefill
+    // that also has chunked context BY NAME.
     ForwardMlaPrefillMha(d.q, prefill_out, q_prefill, key_t, value, kv_cache_ro,
                          meta.prefill_block_table, meta.prefill_cu_seqlens_q, meta.chunks,
                          up, dims.scale, meta.max_query_len,
                          meta.prefill_tokens_with_context, bufs, suffix_out_t,
-                         suffix_lse_t);
+                         suffix_lse_t, dims.sliding_window);
   }
 
   // ─── 5b. DECODE — the ABSORBED MQA form (mla_attention.py:739-830) ───────
@@ -582,6 +596,14 @@ void ForwardMlaAttentionBlock(Dev d, const MlaBlockDims& dims, const MlaBlockWei
     impl.head_size = static_cast<int>(dims.head_size());
     impl.scale = dims.scale;
     impl.queue = &d.q;  // W4 deviation (i), wired here.
+    // dots3-note's windowed decode (#699 W4b-2). Upstream expresses it as the
+    // `Dots3NoteTritonMLAImpl` subclass keeping `self.sliding_window`
+    // (attention.py:439-468 @ bc2d63e650); here it is the impl's field, and 0
+    // is every DeepSeek / MiniCPM3 / Kimi-Linear caller's value. It is assigned
+    // UNCONDITIONALLY rather than under a guard because `impl` is the caller's
+    // object and may be reused across layers of DIFFERENT kinds — a guard would
+    // let a sliding layer's 513 leak into the next full layer.
+    impl.sliding_window = dims.sliding_window;
     v1::AttentionLayer layer{};
     impl.forward_mqa(layer, mqa_q_t, kv_cache, meta.decode, mqa_out_t, nullptr);
     // `self._v_up_proj(attn_out, out=mqa_output_slice)` (:830, :1024-1034):