test(inference): close Gemma 4 F32 attention evidence gaps (ST4A) - #62
Merged
Conversation
Close the remaining ST4 attention-evidence gaps with synthetic F32 CPU
tests driven through the canonical kv_prefill_run / kv_decode_step_run
loops. No attention implementation change.
* larql-kv/tests/st4_shared_kv.rs:
- Canonical prefill/decode numerical equivalence at the 512-window
boundaries 511/512/513/1024 (Route A one-shot prefill vs Route B
prefill-prefix + single decode). Compares the post-layer hidden state,
the local source K/V tail, local/global source cache lengths, shared
consumer absence, and the absolute position pointer.
- Absolute-position-independent-of-clipped-cache-length pin at 1024.
- Qwen2 full-attention regression: intrinsic window None, full causal
prefix retained past 512, decode grows cache +1, no shared-KV routing,
and old-key sensitivity (position-600 decode depends on position-0 K/V
outside a 512-token window).
- Source append-count proof: local/global sources append exactly one row
per decode token (unambiguous deltas below the window cap) and shared
consumers append zero; plus the clipped steady-state companion.
* larql-models/src/test_fixtures.rs:
- make_synthetic_e2b_like_weights_random_window512 + arch json: 4-layer
E2B-like Gemma 4 shape with a 512-token intrinsic window.
- make_qwen2_test_weights: small Qwen2 fixture with Q/K/V biases.
- Refactor make_synthetic_e2b_like_weights_random to share its body.
* larql-inference/src/test_utils.rs: re-export the two new fixtures.
Correct the ST4 (PR #60) report metadata and add the ST4A closeout. * Correct ST4 PR #60 metadata: head SHA 2889157, squash merge b2584c0, CI all 11 workflows green. Remove the prior 'pending PR head' / 'pending PR creation' placeholders. * Replace the Qwen 'N/A' result with the passing Qwen2 full-attention regression result. * Replace the claimed prefill/decode data with actual measured canonical -loop results (max abs <= 1.073e-6 across 511/512/513/1024; bit-exact local K/V tail; local 512 / global target+1 cache lengths). * Add the ST4A closeout section: work-start SHA, tested implementation SHA, target positions, per-position abs/rel diffs, local/global cache lengths, source append deltas, Qwen regression, Q4_K exclusion, GREEN.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the remaining ST4 attention-evidence gaps without reopening or
expanding its implementation scope. Tests + evidence only — no attention
implementation change.
What
PR #60 landed the F32 CPU Gemma 4 local/global attention + shared-KV
routing. This closeout slice adds the missing acceptance evidence:
511 / 512 / 513 / 1024, driven through the production
kv_prefill_run/kv_decode_step_runloops (Route A one-shot prefillvs Route B prefill-prefix + single decode). Real numerical comparison of
the post-layer hidden state, local source K/V tail, local/global cache
lengths, shared-consumer absence, and the absolute position pointer.
(
family == "qwen2", Q/K/V biases): intrinsic windowNone, fullcausal prefix retained past 512, decode grows cache +1, no shared-KV
routing, and old-key sensitivity (position-600 decode depends on
position-0 K/V that a 512-token window would evict).
row per decode token (unambiguous deltas below the window cap) and
shared consumers append zero, across several decode steps.
2889157, mergeb2584c0, CI11/11), Qwen result (was N/A), and measured prefill/decode data (was
claimed), plus a concise ST4A closeout section.
Measured results (canonical loops, abs tolerance 1e-5)
Qwen2 old-key sensitivity: abs
3.679e-4, rel1.334e-2.Scope
In scope: F32 CPU synthetic correctness tests + ST4 evidence/report
correction. Q4_K runs as regression only and remains excluded from the
GREEN claim. No official Gemma 4 checkpoint, no Transformers logits, no
ST5 first-token parity, no GPU paths, no implementation redesign.
Decision
GREEN — all 15 ST4A decision-gate criteria pass.
Next slice:
LARQL-INFERENCE-TRUST-001A-ST5(F32 CPU first-token semanticparity and first-divergence diagnostics) — not started in this PR.