Fix over-wide KV states from recorded sliding window cache updates

[VaggelisGian]

What does this PR do?

Assisted and speculative decoding crash or silently corrupt output for any model with a sliding-window attention layer whose sequence reaches the sliding window. The regression came from #47447, which let assisted decoding run on real DynamicSlidingWindowLayer caches instead of forcing a full cache: while past-recording is active, update() stored AND returned the unbounded key/value concatenation, but get_mask_sizes() kept advertising only sliding_window - 1 + query_length columns for the attention mask. Draft bursts call update() once per drafted token with no crop in between, so from the second token of a burst onward attention received more key/value columns than the mask has:

• eager attention fails hard: RuntimeError: The size of tensor a (13) must match the size of tensor b (6) at non-singleton dimension 3, deterministically reproducible;
• SDPA skips the mask at query length 1 and silently attends beyond-window states, so assisted output diverges from greedy decoding (7/8 seeds in my probing).

Minimal reproducer (CPU, tiny model):

import torch
from transformers import LlamaConfig, LlamaForCausalLM
cfg = LlamaConfig(vocab_size=64, hidden_size=16, intermediate_size=32, num_hidden_layers=4,
                  num_attention_heads=2, num_key_value_heads=2, head_dim=8,
                  max_position_embeddings=512, sliding_window=6)
model = LlamaForCausalLM._from_config(cfg, attn_implementation="eager").eval()
model.generation_config.assistant_confidence_threshold = 0.0
input_ids = torch.randint(1, 60, (1, 12))   # prompt longer than the 6-token window
model.generate(input_ids=input_ids, attention_mask=torch.ones_like(input_ids), do_sample=False,
               max_new_tokens=8, assistant_model=model, num_assistant_tokens=12)
# before: RuntimeError size mismatch inside the assistant forward

The default confidence gate (0.4) collapses most draft bursts to a single token, which is why the mismatch hides in casual use; any confident drafting model or a lower threshold exposes it.

The fix keeps the unbounded buffer stored internally for crop rollback but returns only the trailing advertised window while recording. Paths without recording are bit-identical; get_mask_sizes, masking, and crop semantics are untouched. After the fix every probed configuration shows zero width mismatches, no crashes on eager, and greedy-equal results where the model is fully windowed.

One honest caveat: assisted-vs-greedy divergence can still occur when the MAIN model has a bare sliding window, because multi-token verify forwards give later query positions extra context through plain causal masks. That is a separate pre-existing mask-content trait this PR deliberately does not touch.

No upstream issue exists for this to my knowledge; I searched open issues mentioning dflash/sliding-window assisted crashes before preparing this.

Code Agent Policy

This PR was prepared with the help of a code agent and is disclosed here per the Agentic contributions section of CONTRIBUTING.md; I reviewed every changed line and re-ran all verification locally. The first-time-contributor confirmation above stays unticked because it would be false. No existing issue or PR covers this fix, so there is no coordination link or competing work to differentiate from.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline and the
  Pull Request checks?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes according to the guidelines?
• Did you write any new necessary tests?

Who can review?

@zucchini-nlp (generation) @Cyrilvallez (generate/caches)

Test plan

python -m pytest tests/utils/test_cache_utils.py -k "recording_returns_advertised" -q --no-header
fail-before (fix reverted): 1 failed, AssertionError: 5 != 4   (returned KV wider than advertised)
pass-after: 1 passed

python -m pytest tests/generation/test_utils.py -k "multi_token_draft" -q --no-header
fail-before: 1 failed, RuntimeError "The size of tensor a (13) must match the size of tensor b (6)"
pass-after: 1 passed (stable across repeated runs)

python -m pytest tests/generation/test_utils.py -k "sliding_window_matches" -q --no-header
1 passed

python -m pytest tests/utils/test_cache_utils.py::CacheCroppingTests tests/generation/test_utils.py -k "assisted" ...
CacheCroppingTests 5 passed; assisted sweep: 22 passed, 3 skipped

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 32824071859
Result: success | Grafana metrics are not available yet.