diff --git a/.agents/issue-index.md b/.agents/issue-index.md index ef88b5f90..87a6dc55f 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -748,6 +748,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#526](https://github.com/mudler/vllm.cpp/issues/526) | `SERVE-TOOL-HISTORY-ARGS` | OpenAI multi-turn tool history reaches chat templates with string-valued arguments | bug | | [#1934](https://github.com/mudler/vllm.cpp/issues/1934) | `BACKEND-ROCM` | `RocmPlatform::needs_weight_staging()` is stale-false (a W0-era placeholder never revisited despite #523/#509/#506/ROCM_ATTN/hipGraph landing since), so `CheckDeviceWeightFit` — the #1123/#1870 load-time refusal, including the `policy_forces_full_expand` fix — never runs on ROCm: measured directly, `VT_DEVICE_WEIGHT_BUDGET_BYTES=1` produced no refusal on a real load. The actual device allocation the refusal guards is not gated on this flag, so #1870's crash stays reachable until this closes; owed, not fixed in flow, because flipping the flag also moves `DirectDeviceLoadEligible` and several GDN kernel-dispatch defaults that each need their own correctness check | bug | | [#1978](https://github.com/mudler/vllm.cpp/issues/1978) | `MODEL-MM-QWEN4-EXP` | **`Qwen/Qwen3.8-Flash-Next` declares `Qwen4ExpForConditionalGeneration` / `qwen4_exp`, a new architecture vLLM does not implement, so the port runs on a split oracle: transformers for the ALGORITHM, vLLM ops for the OPTIMIZED PATH.** Released 2026-08-24, 180B total / 6B activated, image-text-to-text. The `Qwen3.8` in the name is marketing continuity: `.agents/specs/qwen38-27b-bf16-gate.md`'s "one config key differs" precedent does NOT extend here. Read live 2026-08-26 at vLLM `origin/main` = `6a5e8f5979`: no `qwen4*` path, no registry entry, and a repository-wide GitHub search for `qwen4` returns ZERO results; `vllm-omni` likewise. That is absence from vLLM `main` rather than staleness in our pin `555967922`, so a pin advance does not reach it. What exists is transformers [#48337](https://github.com/huggingface/transformers/pull/48337) "Add Qwen4Exp model", MERGED 2026-08-26, 5211 lines, and SGLang [#36497](https://github.com/sgl-project/sglang/pull/36497), still OPEN and therefore inadmissible. **Developer direction 2026-08-26, recorded verbatim: "use transformers as oracle for algorithmic side. but use ops from vllm so we account for optimized path."** Justified rather than convenient: `Qwen4ExpTextQSAIndexer.forward` loops in Python over `(batch_idx, query_idx)` and says "we only allow eager and sdpa", so porting it as written yields a correct model at an indefensible speed, while AGENTS.md's mirror-vLLM polarity still binds every primitive vLLM implements. `Qwen4ExpTextModel` inherits from `Qwen3_5MoeTextModel` and leaves rotary, MLP, experts, TopK router and the ENTIRE vision tower unchanged (`class Qwen4ExpVisionModel(Qwen3_5MoeVisionModel): pass`), all of which this tree has; GDN is an exact match for our AOT gate (`K=V=128, Hg=16, Hv=48` against `src/vt/cuda/cuda_gdn.cu`'s `H in {48,32}`). The delta is four things, and **exactly two have no vLLM op at all**: the PLE dilated depthwise conv (kernel 4, dilation 3; `git grep dilation` over vLLM `layers/mamba/` = 0 hits) and the n-gram hashed embedding. **The survey's load-bearing finding, and it REVERSES this row's first reading: QSA's structural twin is DeepSeek-V4's C4 indexer lane, NOT MiniMax-M3.** The original call was that QSA, being plain GQA rather than MLA, had to map onto vLLM's non-MLA block-sparse case; that reasoning rested on treating `MLAAttentionSpec` as an MLA claim, and **it is not one** — M3's own indexer cache uses it while M3 is a plain-GQA model, with the comment "Key-only: MLAAttentionSpec budgets one vector/token (not 2x for K+V)". It is a budget shape. Remove that prop and the GQA-vs-MLA argument collapses. Verified line by line at `6a5e8f5979`: **nine independent structural matches with DSv4**, `compress_ratio == 4` literally the same number — MQA index with 1 key head at dim 128; `relu(q.k)` summed over index heads vs `(score.relu() * weights).sum(dim=0)`; `1/sqrt(head_dim)`; one score set per query token with no head axis vs `topk_indices_buffer[num_tokens, topk]`; pooling boundary `(position+1) % COMPRESS_RATIO == 0`; RMSNorm on the pooled key; **RoPE at the block-start position** vs `compressed_pos = (position // CR) * CR`; candidate count `visible // compress_ratio`; and one stored state per 4 tokens via `MLAAttentionSpec(tokens_per_state=compress_ratio)`, a first-class KV field documented as "Ints > 1 compress multiple tokens into one state (DSv4 sparse MLA)" that has no M3 equivalent. **M3 is a DIFFERENT ALGORITHM**, not a worse fit: its score is `tl.max(qk, axis=1)` over 128 RAW token dots with no pooling, no relu and no head reduction, it asserts `num_idx_heads == num_kv_heads` ("no topk index reduce") so it emits one block set PER KV HEAD, and its `SPARSE_BLOCK_SIZE = 128` is welded to the KV page size ("One sparse block == one KV page") on both the score and the attend side — moving it to 4 forces a page size of 4 and breaks `tl.dot`, whose tile needs >= 16. M3 contributes exactly ONE thing and it is a wiring precedent, not an algorithm: that a plain-GQA model can own a key-only side cache through `MLAAttentionSpec` and a private indexer backend. **The genuinely new work is the CONSUMER and nothing upstream supplies it** — every DSv4 sparse consumer attends to COMPRESSED MLA KV (one state per 4 tokens) and M3's attend to raw tokens only at page granularity, while QSA attends to RAW tokens selected at ratio-4 granularity. Two silent-failure traps follow: wiring QSA's top-k into a DSv4 sparse-MLA consumer attends a POOLED key/value and still emits plausible tokens, and **a short-prompt token gate cannot catch it because at context <= `indexer_budget` 2048 every candidate is selected** — so any QSA gate must run past 2048 tokens of context, which is now a stated `## Gates` requirement; and `SparseAttnCompressNormRopeStoreC4Kernel` does NOT mean-pool despite its name — it is a learned softmax pool over an OVERLAPPING window of 8 using a score channel this checkpoint does not have, and the CuteDSL variant refuses `overlap=False` at compile, so the **Triton** `head_dim=128` variant is the correct starting point. Two structural consequences beyond the module list: the residual stream is `hc_count * hidden_size` = **4 x 2560 = 10240 wide through the whole stack** with a low-rank read gate and per-branch scalar write gate around both attention and MLP, which is a change to the per-layer loop and every residual buffer rather than a drop-in module; and `number_of_conv_states = 3` on a PLE layer (GDN conv, PLE conv, n-gram token history) plus the indexer side cache, adjacent to [#1963](https://github.com/mudler/vllm.cpp/issues/1963) and [#1966](https://github.com/mudler/vllm.cpp/issues/1966). **NOTHING PUBLISHED FITS**, read live from the HF API against ~119 GB usable on GB10: BF16 ~360 GB, official FP8 ~180 GB, `RadixArk/...-NVFP4` ~128 GB (NVFP4 backbone with the n-gram table left at FP8, 51.2 GB) and `unsloth/...-GGUF` is a README with ZERO weight files. No GGUF exists and no tool can make one, because llama.cpp has no `qwen4_exp` either, so the standing k-quant requirement means authoring the arch on our side AND states that the quantized arms have NO llama.cpp oracle. **The chosen arm does NOT load today, and the blocker is neither the offload nor the budget: this tree cannot keep a gather table quantized at all.** `KeepQuantKDim` returns `-1` for `GgufTensorRole::kEmbeddingTable` (`src/vllm/model_executor/model_loader/gguf_keep_quant.cpp`), and `qwen3_5_gguf_weights.cpp` asserts it by name — "the embedding table cannot keep quant blocks" — so a Q4_K or Q8_0 n-gram table EXPANDS to bf16 and 51.2B params become **102.4 GB of anonymous memory**; the arm dies before the first forward. The reason was already sitting in a header comment ("a gather, not a GEMM ... A quantized-gather op is a follow-up row") and **no such row exists**. The only non-expanding gather residency is `kKeepF16`, requiring ggml type 1 exactly (102.4 GB on disk) and CPU-ONLY, because `EmbeddingKernelCuda` refuses anything but f32/bf16. **Second blocker:** `moe_intermediate_size = 640` makes `ffn_down_exps` Q4_K-illegal on its reduction dim (640 % 256 = 128), as does `hc_lowrank = 320`; llama.cpp's substitution is believed to be Q5_0 (**UNVERIFIED, owed against the pinned llama.cpp oracle**) and the dependent fact IS verified in-tree — our reader knows ggml ids `0,1,2,8,10..14,16,18,19,22..28,30,39,40,41,66` and has **no entry for 3, 6, 7 or 20**, so a stock `llama-quantize -Q4_K_M` file fails at header parse. We author the converter, so the fix is Q4_0 (block 32, same 4.5 bpw). **`ENG-WEIGHT-OFFLOAD` will not help** — it moves zero bytes today (`ConsiderWeight` has no production callers, pinned by a test) and is documented inert on GB10; the tier that DOES work already ships and is proven by the 2.4T model serving 369.97 GiB from a 119.631 GiB box at ~62 GiB resident: mmap `MAP_PRIVATE`, borrow in place, alias the host pointer, `prefault: false`. Corrected sizing: backbone ~67.7 GiB, whole process ~73.5 GiB of 119.631 at 32K single-stream, ~46 GiB of headroom for the page cache, so the ~76 GB estimate was right within 10%. The design works because per-token demand is **<= 64 KiB of reads** (16 lookups x 160 dims over at most 16 pages) against the 2.4T expert lane's 6.95 GB/token. The architecture supplies its own lever: the per-token n-gram cost is `(ngram_size-1)*heads_per_ngram` = 16 lookups of 160 dims, so **51 GB of the 180 GB, 28% of the model, is a table touched 16 times per token** and making it non-resident is the intended design point (RadixArk reached the same split independently). Sizing arithmetic, NOT measurement: Q8_0 throughout ~191 GB (no), Q4_K_M throughout ~109 GB (yes, ~10 GB left for KV), Q4_K_M backbone with the table non-resident ~76 GB. GB10 is UNIFIED memory so "offload to host" is not a move there; non-resident means disk-backed, and its cost is unmeasured. **Two decisions were put to the developer as explicit accept-or-reject and BOTH are settled 2026-08-26, recorded in place rather than left open.** (1) `.agents/oracles/transformers.md` pins transformers to 5.14.1, deliberately tied to what the pinned vLLM environment resolves so the environment cannot hold two `transformers` at once, and **5.14.1 does not contain `Qwen4Exp`**; the lane-scoped second pin is **ACCEPTED**, on the argument that the invariant guards a vLLM environment against drifting from its transformers and here there is no vLLM implementation to drift from, and it expires the moment vLLM registers `qwen4_exp`. **The lane pin is a real release, not a branch SHA**, which was not the expected outcome: `Qwen4Exp` merged to `main` at 12:03:40Z on 2026-08-26 and `v5.16.0` published at 12:35:15Z, and this was BOUNDED rather than assumed by fetching `models/qwen4_exp/modeling_qwen4_exp.py` at each tag — `v5.16.0` HTTP **200**, `v5.15.0` HTTP **404** — making 5.16.0 the FIRST release carrying the architecture and therefore the tightest available pin. The version string is UNMEASURED (it is the release proven to contain the model, not a `transformers.__version__` read off a running oracle) and `gateable` stays `no`. (2) The first runnable arm is the **Q4_K_M backbone with the n-gram table NON-RESIDENT** (~76 GB). Q8_0 was raised and does not fit at ~191 GB, and no partial-Q8 split reaches 119 GB with the backbone at 8 bits; Q4_K_M-throughout fits on paper at ~109 GB but leaves ~10 GB for KV and activations on a 262144-native-context model, which is not a margin. This promotes the non-resident table from a note to a first-class W6 deliverable, and it is NOT free: GB10 is UNIFIED memory, so the existing host-pinned offload seam (`ENG-WEIGHT-OFFLOAD`, mirroring vLLM's `cpu_offload_gb`) does not by itself solve it there and the mechanism must be disk-backed or genuinely unloaded — established before it is designed around. Spec: [`specs/qwen4-exp-flash-next.md`](specs/qwen4-exp-flash-next.md). No product code lands under the spec pull request | feature | +| [#1991](https://github.com/mudler/vllm.cpp/issues/1991) | `MODEL-MM-QWEN4-EXP` | **W4, Qwen Sparse Attention: the indexer, its side cache, and a GATHER consumer rather than a mask.** New TU `src/vllm/model_executor/models/qwen4_exp_qsa.{h,cpp}` (`indexer_n_heads` 4, `indexer_kv_heads` 1, `indexer_head_dim` 128, `indexer_budget` 2048 tokens, `indexer_compress_ratio` 4, so `block_topk` 512 and an index buffer 2051 wide). Ported onto DeepSeek-V4's C4 indexer lane per the spec's reversal, with the two things it must NOT inherit reconciled in code: DSv4's `weights_proj` + `head_scale` fold (QSA's weight is the constant `1/sqrt(128)`, applied after the sum over heads) and DSv4's GPT-J RoPE over a TRAILING span (QSA is NeoX `rotate_half` over the LEADING 64 dims with the NoPE dims trailing — the halves are swapped end for end AND the pairing convention differs). `SparseAttnCompressNormRopeStoreC4Kernel` supplies scaffolding only: its `tl.softmax(score, dim=0)` pool over an overlapping window is replaced by an unweighted mean over a non-overlapping window of 4. **The consumer is a GATHER, and that is the wave's point.** llama.cpp [#27739](https://github.com/ggml-org/llama.cpp/pull/27739) records that a sparse mask over a dense cache costs the same as dense attention under CUDA flash attention, because `flash_attn_mask_to_KV_max` only scans back to the first tile that is not all `-inf`; a mask-only QSA is CORRECT and forfeits the lever silently. Proved rather than asserted: mutating `QsaGatherAttention` to delegate to the mask reference leaves **11 of 12 cases green** and fails only `qsa-consumer: the GATHER touches only the selected rows`, so the `keys_visited` counter is the only instrument in the suite that can tell the two apart. Goldens come from RUNNING the unmodified `Qwen4ExpTextQSAIndexer.forward` at the transformers 5.16.0 lane pin (`tests/vllm/models/fixtures/gen_qwen4_exp_qsa_goldens.py`), which resolves the spec's UNMEASURED version string: `transformers.__version__` reads **5.16.0** on a live import, and the oracle runs on CPU with random weights, no checkpoint and no GPU. Both free oracles are used: sub-budget bit-identity with dense attention at kv_len = `budget + compress_ratio - 1`, and index-set equality including the ragged tail above the budget. Side cache is `MLAAttentionSpec(num_kv_heads=1, head_size=128, tokens_per_state=4)` = **64 B/token/layer at bf16**, asserted rather than defaulted, with `StatesForTokens` flooring because a ragged block writes no state. LANDS UNREACHED: `Qwen4ExpTextModel` does not exist (W2 #1987, W3 #1988, W6a #1989 are sibling waves; the registry and runner wiring is W5), row `MODEL-MM-QWEN4-EXP` owns it, [#1978](https://github.com/mudler/vllm.cpp/issues/1978) tracks it, and the spec lists it under `## Owed`. No CUDA arm and no speed claim — the speed axis opens at G4, after W6a | feature | | [#1987](https://github.com/mudler/vllm.cpp/issues/1987) | `MODEL-MM-QWEN4-EXP` | **W2 of [#1978](https://github.com/mudler/vllm.cpp/issues/1978): the hashed n-gram embedding and the PLE dilated depthwise conv, the ONLY two components of `Qwen4ExpForConditionalGeneration` with no vLLM op at all.** The negative is confirmed rather than unfound — at vLLM `origin/main` = `6a5e8f5979`, `git grep -in dilat` returns ZERO lines in `layers/mamba/`, zero in `csrc/` and zero in `tests/`, and `layers/conv.py` defines only `Conv2dLayer`/`Conv3dLayer` — so transformers **v5.16.0**, this row's accepted lane pin, is the sole oracle. Three silent divergence sites, all verified: `_splitmix64` must be `uint64_t` throughout (its `>> 30/27/31` are LOGICAL shifts on a non-negative Python int; on `int64_t` they go arithmetic and the multiplicand is wrong, with the top bit set about half the time), `_splitmix64(value) % half_bound` must be an UNSIGNED modulo (the dividend routinely exceeds 2^63), and shard reassembly is NUMERIC not lexicographic (`shard_0, shard_1, shard_10, ...` silently permutes a 95 GiB table). The forward is int64-exact and needs no bignum because `multiplier_max * vocab_size <= 2^63 - 1` — but **that bound holds only while every token id is below `vocab_size`**, so an out-of-range id overflows and diverges in silence, and this wave refuses one by name. Goldens confirmed a FOURTH way beyond #1987's three: `vocab_size = 248320`, read from the released `config.json`, is the UNIQUE preimage below 2e6 of the published `layer_multipliers = [23703573157769, 20109073645365, 8052911324071]` at the absent-`seed` default 1234. PLE specifics: the conv state is **9 columns** — `(kernel-1) * dilation`, NOT `kernel-1` — and holds the **normed** conv input while the skip term is the **un-normed** copy; the signed-sqrt gate clamps BEFORE the sqrt so the output magnitude floor is `sqrt(1e-6) = 1e-3` and tiny scores are AMPLIFIED, with exactly zero mapping to zero, making the function genuinely discontinuous at an origin that a fully masked row reaches; and the n-gram history must be padded with EOS because upstream's `update_conv_state` pads with 0, a valid token id. **Lands UNREACHED** per AGENTS.md "Nothing lands dead": no production entry point calls it until W5 assembles the model, the wiring is owned by `MODEL-MM-QWEN4-EXP` and tracked by #1978, and the spec lists it under `## Owed`. Host reference only; the batched device arm is owed, and the per-sequence signatures exist so it drops in — the gather is 16 uncoalesced random rows per token PER SEQUENCE, so at batch B it is 16*B independent gathers with no coalescing. No token claim, no speed claim: `## Gates` admits none from this row until an arm runs, and nothing published fits any fleet device | feature | | [#1963](https://github.com/mudler/vllm.cpp/issues/1963) | `ROAD-V1-MEM` | At ctx=32768 `--max-num-seqs 32` our engine consumes ~108 GB during load and never serves; vLLM and SGLang both serve there. Root cause found and fixed by `FIX-KV-GROUP-LAYER-COUNT` ([spec](specs/kv-group-layer-count.md)): thirty-three of thirty-four registries publish ONE placeholder name per KV group, `KVBytesPerBlock` reads `layer_names.size()` as the layer count, and `ResolveNumBlocks` arm 2 therefore divides an absolute `--kv-cache-memory` budget by ONE layer's page while the runner allocates one buffer per layer — measured 8.5 GiB allocated for a 1 GiB budget on the 27B | bug | | [#1966](https://github.com/mudler/vllm.cpp/issues/1966) | `ROAD-V1-MEM` | The #371 recurrent-state OOM guard is 48x under and passes a config that allocates 43.4 GiB, because `recurrent_state_bytes` counts placeholder layer names. Same root defect as [#1963](https://github.com/mudler/vllm.cpp/issues/1963), different code path; both fixed by `FIX-KV-GROUP-LAYER-COUNT` ([spec](specs/kv-group-layer-count.md)) | bug | diff --git a/.agents/oracles/transformers.md b/.agents/oracles/transformers.md index d2d85d055..76c718351 100644 --- a/.agents/oracles/transformers.md +++ b/.agents/oracles/transformers.md @@ -129,5 +129,16 @@ precedent says an oracle is gateable only once it demonstrably builds and runs t model. Resolving the runtime string is owed to the first wave that stands an oracle up. Do not promote this pin to `gateable = yes` by editing the line. +**The version string is no longer unmeasured; `gateable` still is.** W4 +([#1991](https://github.com/mudler/vllm.cpp/issues/1991)) stood the lane oracle up +and `transformers.__version__` reads **5.16.0** on a live import, with +`transformers.models.qwen4_exp.modeling_qwen4_exp.Qwen4ExpTextQSAIndexer` +importable and running on CPU against random weights. +`tests/vllm/models/fixtures/gen_qwen4_exp_qsa_goldens.py` refuses to emit under any +other version string, so that reading is now executable rather than recorded. This +resolves only the half this note said was owed. `gateable` stays `no`: running one +module on random weights is not running the model, no published artifact fits any +fleet device, and the field means what `## Gateability` says it means. + See [`../specs/qwen4-exp-flash-next.md`](../specs/qwen4-exp-flash-next.md) `## Oracles`. diff --git a/.agents/specs/qwen4-exp-flash-next.md b/.agents/specs/qwen4-exp-flash-next.md index b58a12d41..d6117d853 100644 --- a/.agents/specs/qwen4-exp-flash-next.md +++ b/.agents/specs/qwen4-exp-flash-next.md @@ -779,10 +779,57 @@ change that makes any arm reachable, not later. substitution for a ragged-K Q4_K tensor, asserted here as Q5_0. - A K-divisibility assertion in whatever writes our GGUF files. - A speed denominator, once one exists. +- **W4's QSA slice lands UNREACHED**, and this entry is what AGENTS.md "Nothing + lands dead" requires in exchange. + `src/vllm/model_executor/models/qwen4_exp_qsa.{h,cpp}` + ([#1991](https://github.com/mudler/vllm.cpp/issues/1991)) ship the indexer, the + side-cache sizing and the GATHER consumer as host reference math with no + production call site: `Qwen4ExpTextModel` does not exist yet, its PLE + ([#1987](https://github.com/mudler/vllm.cpp/issues/1987)), hyper-connection + stream ([#1988](https://github.com/mudler/vllm.cpp/issues/1988)) and GGUF + reader ([#1989](https://github.com/mudler/vllm.cpp/issues/1989)) are sibling + waves, and the registry entry plus runner wiring belong to W5. Row + `MODEL-MM-QWEN4-EXP` owns that wiring and + [#1978](https://github.com/mudler/vllm.cpp/issues/1978) tracks it. +- **The QSA device arm.** `qwen4_exp_qsa.cpp` is the portable oracle a CUDA + kernel is written against, the way `deepseek_v4_dsa.h` is for + `src/vt/cuda/cuda_deepseek_v4.cu`. Nothing in W4 runs on a GPU, so the gather's + cost advantage over the mask is stated by a `keys_visited` count and NOT by a + measurement. The speed axis opens at G4. +- **`QsaCompressNormRope` assumes a contiguous visible range.** Upstream forms + blocks over `local_visible_indices` of a padded batch; a serving engine's + ragged batch has no interior masking, so the two coincide and the function + asserts `num_keys % compress_ratio == 0` instead of accepting an arbitrary + visibility set. A padded-batch caller would need the general form. +- **The row's lifecycle record is owed the W4 transition, and W5 lands it.** W4 + ([#1991](https://github.com/mudler/vllm.cpp/issues/1991)) is this row's first + product code: `src/vllm/model_executor/models/qwen4_exp_qsa.cpp` joins + `add_library(vllm ...)` at its merge commit. `.agents/model-matrix.md` still + carries the row at `READY` with the note "SPEC ONLY, NO PRODUCT CODE, NO TOKEN, + NO SPEED", which was true at the merge base and is false from W4 onwards. That + cell is NOT edited here: W1 through W3 are live on the same file and the + operator is sequencing those writes, and a per-wave edit to one shared row is + exactly the lock AGENTS.md "Records" forbids. W5, which lands the registry entry + and the runner wiring, moves the row to `ACTIVE`, rewrites that note and updates + `## Now` in the one change. Until then this entry is where the discrepancy is + visible. +- **Nothing gates the interleaved-mRoPE section layout, in W4 or anywhere yet.** + `gen_qwen4_exp_qsa_goldens.py` passes a 2-D `position_ids`, which + `Qwen4ExpTextRotaryEmbedding.forward` expands into three IDENTICAL streams, so + `apply_interleaved_mrope` runs value-blind and the captured `cos`/`sin` are + indistinguishable from plain RoPE. `qwen4_exp_qsa.h` scopes the tables out of W4 + ("this function does not build them") and W4 is honest about that, but no wave + currently owns building them, and a multimodal caller with genuinely different + t/h/w streams would be running an untested section layout. The wave that builds + the cos/sin tables owes a case with three DISTINCT position streams. ## Now -`READY`. Spec committed, no implementation. +`READY` in the matrix, and the state cell is deliberately not moved here — see +`## Owed`, "the row's lifecycle record is owed the W4 transition". The spec is +committed; W4 has landed the QSA host reference math +([#1991](https://github.com/mudler/vllm.cpp/issues/1991)) UNREACHED, and no other +implementation exists. Both decisions this spec was blocked on are **settled** (developer, 2026-08-26) and recorded in place rather than left as proposals: the transformers lane pin is diff --git a/CMakeLists.txt b/CMakeLists.txt index af13d8ed8..8eecaa3a6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -774,6 +774,9 @@ add_library(vllm STATIC src/vllm/model_executor/models/qwen3_5_common.cpp src/vllm/model_executor/models/qwen3_5_dense.cpp src/vllm/model_executor/models/qwen3_5_moe.cpp + # Qwen4-Exp (Qwen3.8-Flash-Next) W4: Qwen Sparse Attention. Host reference + # math for the indexer, its side cache and the GATHER consumer (#1991). + src/vllm/model_executor/models/qwen4_exp_qsa.cpp src/vllm/model_executor/models/qwen3_dense.cpp src/vllm/model_executor/models/qwen3_weights.cpp src/vllm/model_executor/models/qwen3.cpp diff --git a/src/vllm/model_executor/models/qwen4_exp_qsa.cpp b/src/vllm/model_executor/models/qwen4_exp_qsa.cpp new file mode 100644 index 000000000..51e2227ee --- /dev/null +++ b/src/vllm/model_executor/models/qwen4_exp_qsa.cpp @@ -0,0 +1,477 @@ +// Qwen4-Exp W4 — Qwen Sparse Attention. Host reference implementations. +// See qwen4_exp_qsa.h for the full port map, file:line on BOTH sides. +#include "vllm/model_executor/models/qwen4_exp_qsa.h" + +#include +#include +#include +#include + +#include "vt/dtype.h" // VT_CHECK, F32ToBF16, BF16ToF32 + +namespace vllm::qwen4_exp { +namespace { + +constexpr float kNegInf = -std::numeric_limits::infinity(); + +// One bf16 round trip. Upstream's `.type_as(x)` / `.to(raw_keys.dtype)` on a +// bf16 model path, and a bf16 elementwise op's store, both do exactly this. +// `vt::F32ToBF16` rounds to nearest even, which is what torch does. +inline float Bf16(float x) { return vt::BF16ToF32(vt::F32ToBF16(x)); } +inline float MaybeBf16(float x, bool round) { return round ? Bf16(x) : x; } + +} // namespace + +// ── Config ─────────────────────────────────────────────────────────────────── + +void QsaValidateConfig(const QsaConfig& cfg) { + // configuration_qwen4_exp.py:219-220 — every QSA field must be positive. + VT_CHECK(cfg.index_n_heads > 0 && cfg.index_kv_heads > 0 && + cfg.index_head_dim > 0 && cfg.token_budget > 0 && + cfg.compress_ratio > 0, + "QSA config values must be positive"); + // :221-222 — "Qwen4-Exp QSA requires indexer_kv_heads=1." + VT_CHECK(cfg.index_kv_heads == 1, "Qwen4-Exp QSA requires indexer_kv_heads=1"); + // :223-224 — "indexer_budget must be divisible by indexer_compress_ratio." + VT_CHECK(cfg.token_budget % cfg.compress_ratio == 0, + "indexer_budget must be divisible by indexer_compress_ratio"); + // :225-231 — rotary_dim = int(head_dim * partial_rotary_factor) must FIT the + // index head, because the indexer ropes the leading rotary_dim of an + // indexer_head_dim-wide vector. + VT_CHECK(cfg.rotary_dim >= 0 && cfg.rotary_dim <= cfg.index_head_dim, + "Qwen4-Exp attention RoPE dimensions must fit the QSA index head"); + VT_CHECK(cfg.rotary_dim % 2 == 0, "rotary_dim must be even (rotate_half)"); +} + +// ── The side cache ─────────────────────────────────────────────────────────── + +int64_t QsaSideCacheSpec::StatesForTokens(int64_t num_tokens) const { + VT_CHECK(tokens_per_state > 0, "tokens_per_state must be positive"); + VT_CHECK(num_tokens >= 0, "num_tokens must be non-negative"); + // FLOOR, not ceil: `(position + 1) % COMPRESS_RATIO != 0` early-exits the + // compressor (fused_compress_quant_cache.py:729-731), so an incomplete + // trailing block writes no state. It is attended from the raw KV instead. + return num_tokens / tokens_per_state; +} + +int64_t QsaSideCacheSpec::BytesPerTokenPerLayer() const { + VT_CHECK(tokens_per_state > 0 && head_size > 0 && elem_bytes > 0 && + num_kv_heads > 0, + "side cache spec must be positive"); + // KEY-ONLY: one vector per state, not 2x for K+V. That is what + // MLAAttentionSpec buys here, and it is why this is a quarter of the cost of + // a per-token index cache. + const int64_t per_state = elem_bytes * num_kv_heads * head_size; + VT_CHECK(per_state % tokens_per_state == 0, + "side cache state size must divide evenly across its tokens"); + return per_state / tokens_per_state; +} + +int64_t QsaSideCacheSpec::BytesForTokens(int64_t num_tokens) const { + VT_CHECK(num_tokens >= 0, "num_tokens must be non-negative"); + return BytesPerTokenPerLayer() * num_tokens; +} + +QsaSideCacheSpec QsaMakeSideCacheSpec(const QsaConfig& cfg, int64_t elem_bytes) { + QsaValidateConfig(cfg); + VT_CHECK(elem_bytes > 0, "elem_bytes must be positive"); + QsaSideCacheSpec spec; + spec.num_kv_heads = cfg.index_kv_heads; // 1, and the config gate enforces it + spec.head_size = cfg.index_head_dim; + spec.tokens_per_state = cfg.compress_ratio; + spec.elem_bytes = elem_bytes; + return spec; +} + +int64_t QsaCompressedSlot(int64_t position, int64_t compress_ratio) { + VT_CHECK(compress_ratio > 0, "compress_ratio must be positive"); + VT_CHECK(position >= 0, "position must be non-negative"); + // compressor_utils.py:50,:61 — `is_valid = (pos + 1) % COMPRESS_RATIO == 0`, + // and PAD_ID (-1) everywhere else. + if ((position + 1) % compress_ratio != 0) return -1; + return position / compress_ratio; // :51 `pos_after_compress` +} + +// ── Primitives ─────────────────────────────────────────────────────────────── + +std::vector QsaRmsNorm(const std::vector& x, int64_t rows, + int64_t dim, const std::vector& weight, + float eps, bool round_to_bf16) { + VT_CHECK(rows >= 0 && dim > 0, "bad rmsnorm shape"); + VT_CHECK(static_cast(x.size()) == rows * dim, "x size mismatch"); + VT_CHECK(static_cast(weight.size()) == dim, "weight size mismatch"); + std::vector out(x.size()); + for (int64_t r = 0; r < rows; ++r) { + const float* src = &x[r * dim]; + // `x.pow(2).mean(-1)`, in float32 as `_norm(x.float())` demands. + float acc = 0.0f; + for (int64_t d = 0; d < dim; ++d) acc += src[d] * src[d]; + const float rrms = 1.0f / std::sqrt(acc / static_cast(dim) + eps); + for (int64_t d = 0; d < dim; ++d) { + // `out * (1.0 + weight)`, NOT vLLM's `out * weight`. The weight is + // zero-initialised upstream, so dropping the +1 turns the norm off + // entirely and is invisible on a freshly built module. + const float v = src[d] * rrms * (1.0f + weight[d]); + // A single rounding, at the `.type_as(x)` on the way out. + out[r * dim + d] = MaybeBf16(v, round_to_bf16); + } + } + return out; +} + +std::vector QsaApplyRotaryLeadingHalf(const std::vector& x, + int64_t rows, int64_t head_dim, + const std::vector& cos, + const std::vector& sin, + int64_t rotary_dim, + bool round_to_bf16) { + VT_CHECK(rows >= 0 && head_dim > 0, "bad rope shape"); + VT_CHECK(rotary_dim >= 0 && rotary_dim <= head_dim && rotary_dim % 2 == 0, + "rotary_dim must be even and fit head_dim"); + VT_CHECK(static_cast(x.size()) == rows * head_dim, "x size mismatch"); + VT_CHECK(static_cast(cos.size()) == rows * rotary_dim && + static_cast(sin.size()) == rows * rotary_dim, + "cos/sin size mismatch"); + const int64_t half = rotary_dim / 2; + std::vector out(x.size()); + for (int64_t r = 0; r < rows; ++r) { + const float* src = &x[r * head_dim]; + const float* c = &cos[r * rotary_dim]; + const float* s = &sin[r * rotary_dim]; + for (int64_t d = 0; d < rotary_dim; ++d) { + // `rotate_half`: cat(-x2, x1) over the ROTARY span only. + const float rot = d < half ? -src[d + half] : src[d - half]; + // Two products then a sum. On a bf16 tensor each of those three ops + // stores a bf16, so each rounds; folding them into one float32 + // expression would drift from the oracle. + const float a = MaybeBf16(src[d] * c[d], round_to_bf16); + const float b = MaybeBf16(rot * s[d], round_to_bf16); + out[r * head_dim + d] = MaybeBf16(a + b, round_to_bf16); + } + // The NoPE dims trail and are concatenated back untouched. This is the half + // that DeepSeek-V4's indexer puts FIRST — see the header note. + for (int64_t d = rotary_dim; d < head_dim; ++d) { + out[r * head_dim + d] = src[d]; + } + } + return out; +} + +std::vector QsaCompressNormRope(const std::vector& raw_keys, + int64_t num_keys, + const std::vector& k_norm_weight, + const std::vector& cos, + const std::vector& sin, + const QsaConfig& cfg, + bool round_to_bf16) { + QsaValidateConfig(cfg); + const int64_t D = cfg.index_head_dim; + const int64_t CR = cfg.compress_ratio; + VT_CHECK(num_keys >= 0, "num_keys must be non-negative"); + VT_CHECK(num_keys % CR == 0, + "QsaCompressNormRope takes COMPLETE blocks only; the caller trims " + "the ragged tail, which is attended from the raw KV instead"); + VT_CHECK(static_cast(raw_keys.size()) >= num_keys * D, + "raw_keys size mismatch"); + VT_CHECK(static_cast(k_norm_weight.size()) == D, + "k_layernorm weight size mismatch"); + const int64_t nb = num_keys / CR; + VT_CHECK(static_cast(cos.size()) >= num_keys * cfg.rotary_dim && + static_cast(sin.size()) >= num_keys * cfg.rotary_dim, + "cos/sin must cover every key position"); + + // 1. Unweighted mean over a NON-OVERLAPPING window of compress_ratio, in + // float, then rounded back to the cache dtype. This is the one place the + // DeepSeek-V4 compressor must NOT be copied: its pool is a LEARNED softmax + // over an overlapping window, in the Triton + // `_fused_kv_compress_norm_rope_insert_indexer_attn` + // (fused_compress_quant_cache.py:677, softmax at :769) and again in the + // CuteDSL `SparseAttnCompressNormRopeStoreC4Kernel` + // (nvidia/ops/sparse_attn_compress_cutedsl.py:75, :1121-1130). + std::vector pooled(static_cast(nb) * D); + for (int64_t b = 0; b < nb; ++b) { + for (int64_t d = 0; d < D; ++d) { + float acc = 0.0f; + for (int64_t i = 0; i < CR; ++i) acc += raw_keys[(b * CR + i) * D + d]; + pooled[b * D + d] = + MaybeBf16(acc / static_cast(CR), round_to_bf16); + } + } + + // 2. k_layernorm on the POOLED key, not on the raw ones. + const std::vector normed = + QsaRmsNorm(pooled, nb, D, k_norm_weight, cfg.rms_norm_eps, round_to_bf16); + + // 3. RoPE at the position of the block's FIRST token. Upstream reads it as + // `group_starts = block_token_indices[:, 0]`; the Triton kernel computes + // the same value as `(position // CR) * CR`. Taking the block's LAST + // position instead is a silent one-block phase error. + std::vector bcos(static_cast(nb) * cfg.rotary_dim); + std::vector bsin(bcos.size()); + for (int64_t b = 0; b < nb; ++b) { + const int64_t start = b * CR; + for (int64_t d = 0; d < cfg.rotary_dim; ++d) { + bcos[b * cfg.rotary_dim + d] = cos[start * cfg.rotary_dim + d]; + bsin[b * cfg.rotary_dim + d] = sin[start * cfg.rotary_dim + d]; + } + } + return QsaApplyRotaryLeadingHalf(normed, nb, D, bcos, bsin, cfg.rotary_dim, + round_to_bf16); +} + +std::vector QsaBlockScores(const std::vector& q, + const std::vector& block_keys, + int64_t num_blocks, const QsaConfig& cfg) { + QsaValidateConfig(cfg); + const int64_t H = cfg.index_n_heads, D = cfg.index_head_dim; + VT_CHECK(static_cast(q.size()) == H * D, "q size mismatch"); + VT_CHECK(num_blocks >= 0, "num_blocks must be non-negative"); + VT_CHECK(static_cast(block_keys.size()) >= num_blocks * D, + "block_keys size mismatch"); + // The scale is a CONSTANT. DeepSeek-V4 folds a learned per-(token,head) + // `weights_proj` and an extra head_scale of n_heads**-0.5 in here; QSA has + // neither tensor. Inheriting that fold would silently rescale every logit. + const float inv_sqrt_d = 1.0f / std::sqrt(static_cast(D)); + std::vector scores(static_cast(num_blocks)); + for (int64_t b = 0; b < num_blocks; ++b) { + const float* kp = &block_keys[b * D]; + float acc = 0.0f; + for (int64_t h = 0; h < H; ++h) { + const float* qp = &q[h * D]; + float dot = 0.0f; + for (int64_t d = 0; d < D; ++d) dot += qp[d] * kp[d]; + // The ReLU is load-bearing: it lets a negatively-correlated head abstain + // rather than veto. + acc += dot > 0.0f ? dot : 0.0f; + } + // Divided AFTER the sum over heads, as upstream writes it. + scores[b] = acc * inv_sqrt_d; + } + return scores; +} + +std::vector QsaTopkBlocks(const std::vector& scores, + int64_t num_blocks, int64_t k) { + VT_CHECK(num_blocks >= 0, "num_blocks must be non-negative"); + VT_CHECK(static_cast(scores.size()) >= num_blocks, + "scores size mismatch"); + VT_CHECK(k >= 0 && k <= num_blocks, "k must be in [0, num_blocks]"); + + std::vector idx(static_cast(num_blocks)); + std::iota(idx.begin(), idx.end(), 0); + if (num_blocks <= k) { + // sampler.cu:391-402 — the all-select shortcut emits ASCENDING candidate + // order, not score order. Inherited for consistency with vLLM's op, NOT + // load-bearing: `QsaSelectedTokenIndices` sorts the expanded tokens anyway, + // so reversing this order changes nothing observable and the suite stays + // green. What makes the bit-identity oracle hold is that `std::sort`, not + // this shortcut, sets the gather's reduction order. + return idx; + } + // sampler.cu:515 — `logit < otherLogit || (logit == otherLogit && i < j)`: + // a tie goes to the LOWER index. `std::stable_sort` on a strict + // greater-than gives exactly that. Inherited for consistency and UNEXERCISED: + // the golden fixtures produce no exact score tie, so flipping this rule leaves + // the suite green. It is here so a real tie resolves the way the op it mirrors + // resolves one, not because anything below can see it. + std::stable_sort(idx.begin(), idx.end(), + [&](int64_t a, int64_t b) { return scores[a] > scores[b]; }); + idx.resize(static_cast(k)); + return idx; +} + +std::vector QsaSelectedTokenIndices( + const std::vector& selected_blocks, int64_t num_complete_blocks, + int64_t kv_len, const QsaConfig& cfg) { + QsaValidateConfig(cfg); + const int64_t CR = cfg.compress_ratio; + const int64_t width = cfg.index_width(); + VT_CHECK(kv_len >= 0, "kv_len must be non-negative"); + VT_CHECK(num_complete_blocks == kv_len / CR, + "num_complete_blocks must be kv_len / compress_ratio"); + VT_CHECK(static_cast(selected_blocks.size()) <= cfg.block_topk(), + "more blocks selected than the budget allows"); + + std::vector tokens; + tokens.reserve(static_cast(width)); + for (int64_t b : selected_blocks) { + VT_CHECK(b >= 0 && b < num_complete_blocks, "selected block out of range"); + // Block b IS tokens [CR*b, CR*b + CR). This expansion is what a mask-only + // implementation never performs, and the reason the consumer can be a + // gather at all. + for (int64_t i = 0; i < CR; ++i) { + tokens.push_back(static_cast(b * CR + i)); + } + } + // The incomplete trailing block is ALWAYS attended, whatever the scores said. + // It is why the buffer is `budget + compress_ratio - 1` wide, not `budget`. + for (int64_t t = num_complete_blocks * CR; t < kv_len; ++t) { + tokens.push_back(static_cast(t)); + } + VT_CHECK(static_cast(tokens.size()) <= width, + "selected token count exceeds the index buffer width"); + + // ASCENDING. Upstream's buffer is in score-rank order because it is only ever + // scattered into a mask; a gather's order IS the softmax's reduction order, + // and ascending is what makes a sub-budget gather reduce over exactly the + // dense sequence. + std::sort(tokens.begin(), tokens.end()); + std::vector out(static_cast(width), -1); + std::copy(tokens.begin(), tokens.end(), out.begin()); + return out; +} + +// ── Consumers ──────────────────────────────────────────────────────────────── + +namespace { + +// Shared shape check for both consumers, so they cannot disagree about what +// they were handed. +void CheckConsumerShapes(const std::vector& q, const std::vector& k, + const std::vector& v, int64_t kv_len, + int64_t num_q_heads, int64_t num_kv_heads, + int64_t head_dim) { + VT_CHECK(num_q_heads > 0 && num_kv_heads > 0 && head_dim > 0, + "bad attention shape"); + VT_CHECK(num_q_heads % num_kv_heads == 0, + "GQA needs num_q_heads divisible by num_kv_heads"); + VT_CHECK(kv_len >= 0, "kv_len must be non-negative"); + VT_CHECK(static_cast(q.size()) == num_q_heads * head_dim, + "q size mismatch (one query token)"); + VT_CHECK(static_cast(k.size()) >= kv_len * num_kv_heads * head_dim, + "k size mismatch"); + VT_CHECK(static_cast(v.size()) >= kv_len * num_kv_heads * head_dim, + "v size mismatch"); +} + +// The selected positions, ascending, with the -1 padding stripped. +std::vector Gathered(const std::vector& indices, + int64_t kv_len) { + std::vector out; + out.reserve(indices.size()); + for (int32_t idx : indices) { + if (idx < 0) break; // -1 terminates; the padding is not a position + VT_CHECK(idx < kv_len, "selected index past the end of the cache"); + out.push_back(static_cast(idx)); + } + return out; +} + +float Dot(const float* a, const float* b, int64_t n) { + float acc = 0.0f; + for (int64_t i = 0; i < n; ++i) acc += a[i] * b[i]; + return acc; +} + +} // namespace + +std::vector QsaGatherAttention(const std::vector& q, + const std::vector& k, + const std::vector& v, + const std::vector& indices, + int64_t kv_len, int64_t num_q_heads, + int64_t num_kv_heads, int64_t head_dim, + int64_t* keys_visited) { + CheckConsumerShapes(q, k, v, kv_len, num_q_heads, num_kv_heads, head_dim); + const std::vector sel = Gathered(indices, kv_len); + VT_CHECK(!sel.empty() || kv_len == 0, "a causal query attends at least itself"); + + const float scale = 1.0f / std::sqrt(static_cast(head_dim)); + const int64_t groups = num_q_heads / num_kv_heads; + std::vector out(static_cast(num_q_heads) * head_dim, 0.0f); + + // The KEY-ROW READ COUNT, taken AT the read and nowhere else. An earlier + // revision assigned `sel.size()` here, which restates the index buffer instead + // of measuring the loop: a body that dot-products every one of the `kv_len` + // cached rows and masks the rest with -inf — QsaMaskedAttention's cost wearing + // this function's name — kept reporting the sparse number, and the suite + // stayed green (W4 fresh review, mutation M22c). Counting at the `Dot` makes + // the observable a function of the walk, so an implementation that does the + // dense work cannot report the sparse figure. + int64_t reads = 0; + const auto key_dot = [&](const float* qp, int64_t t, int64_t kvh) { + ++reads; + return Dot(qp, &k[(t * num_kv_heads + kvh) * head_dim], head_dim); + }; + + for (int64_t h = 0; h < num_q_heads; ++h) { + const int64_t kvh = h / groups; + const float* qp = &q[h * head_dim]; + // Pass 1: the max, over the GATHERED rows only. + float m = kNegInf; + for (int64_t t : sel) { + const float l = key_dot(qp, t, kvh) * scale; + m = std::max(m, l); + } + // Pass 2: the softmax weights and the value reduction, ascending. + float denom = 0.0f; + float* dst = &out[h * head_dim]; + for (int64_t t : sel) { + const float l = key_dot(qp, t, kvh) * scale; + const float w = std::exp(l - m); + denom += w; + const float* vp = &v[(t * num_kv_heads + kvh) * head_dim]; + for (int64_t d = 0; d < head_dim; ++d) dst[d] += w * vp[d]; + } + for (int64_t d = 0; d < head_dim; ++d) dst[d] /= denom; + } + if (keys_visited != nullptr) *keys_visited = reads; + return out; +} + +std::vector QsaMaskedAttention(const std::vector& q, + const std::vector& k, + const std::vector& v, + const std::vector& indices, + int64_t kv_len, int64_t num_q_heads, + int64_t num_kv_heads, int64_t head_dim, + int64_t* keys_visited) { + CheckConsumerShapes(q, k, v, kv_len, num_q_heads, num_kv_heads, head_dim); + const std::vector sel = Gathered(indices, kv_len); + std::vector keep(static_cast(kv_len), false); + for (int64_t t : sel) keep[static_cast(t)] = true; + + const float scale = 1.0f / std::sqrt(static_cast(head_dim)); + const int64_t groups = num_q_heads / num_kv_heads; + std::vector out(static_cast(num_q_heads) * head_dim, 0.0f); + + // Counted at the read, in the SAME unit as the gather's, so the two numbers + // are comparable. This path reads every cached row whatever the mask says, + // which is exactly what llama.cpp #27739 measured a sparse mask over a dense + // cache doing under CUDA flash attention. It is a measured `kv_len * + // num_q_heads * 2`, not an asserted `kv_len`. + int64_t reads = 0; + const auto key_dot = [&](const float* qp, int64_t t, int64_t kvh) { + ++reads; + return Dot(qp, &k[(t * num_kv_heads + kvh) * head_dim], head_dim); + }; + + for (int64_t h = 0; h < num_q_heads; ++h) { + const int64_t kvh = h / groups; + const float* qp = &q[h * head_dim]; + float m = kNegInf; + for (int64_t t = 0; t < kv_len; ++t) { + const float raw = key_dot(qp, t, kvh) * scale; + const float l = keep[static_cast(t)] ? raw : kNegInf; + m = std::max(m, l); + } + float denom = 0.0f; + float* dst = &out[h * head_dim]; + for (int64_t t = 0; t < kv_len; ++t) { + const float raw = key_dot(qp, t, kvh) * scale; + const float l = keep[static_cast(t)] ? raw : kNegInf; + // exp(-inf - m) is exactly +0, and adding an exact zero to a float + // accumulator changes nothing. That is why this agrees with the gather + // value for value while doing kv_len times the work. + const float w = std::exp(l - m); + denom += w; + const float* vp = &v[(t * num_kv_heads + kvh) * head_dim]; + for (int64_t d = 0; d < head_dim; ++d) dst[d] += w * vp[d]; + } + for (int64_t d = 0; d < head_dim; ++d) dst[d] /= denom; + } + if (keys_visited != nullptr) *keys_visited = reads; + return out; +} + +} // namespace vllm::qwen4_exp diff --git a/src/vllm/model_executor/models/qwen4_exp_qsa.h b/src/vllm/model_executor/models/qwen4_exp_qsa.h new file mode 100644 index 000000000..a741b998c --- /dev/null +++ b/src/vllm/model_executor/models/qwen4_exp_qsa.h @@ -0,0 +1,352 @@ +// Qwen4-Exp (Qwen3.8-Flash-Next) W4 — Qwen Sparse Attention: the indexer, its +// side cache, and the CONSUMER. Portable host (CPU) reference implementations. +// +// Row MODEL-MM-QWEN4-EXP, issue #1991, spec .agents/specs/qwen4-exp-flash-next.md. +// +// ─── THE POINT OF THIS FILE: A GATHER, NOT A MASK ─────────────────────────── +// llama.cpp #27739 records, with the mechanism named, that a sparse MASK over a +// dense cache costs the same as dense attention under CUDA flash attention, +// because `flash_attn_mask_to_KV_max` only scans back to the first tile that is +// not all -inf. llama.cpp #27742 is mask-only and therefore buys correctness +// without decode speed. So the consumer here is `QsaGatherAttention`, which +// expands selected block `b` into tokens [CR*b, CR*b + CR), appends the ragged +// tail, and reduces over ONLY those rows. `QsaMaskedAttention` exists solely as +// the red-first reference the gather is gated against; it is O(kv_len) by +// construction and must never be the production path. +// +// Both consumers report `keys_visited`, and both COUNT it at the key-row read +// rather than assign it from the index buffer. That distinction is the whole +// instrument: a counter set to `sel.size()` is an assertion about `indices`, and +// a body doing the dense work under the gather's name still satisfies it (W4 +// fresh review, mutation M22c, green on the first revision of this file). Counted +// at the `Dot`, the gather reports `selected * num_q_heads * 2` and the mask +// reports `kv_len * num_q_heads * 2`, so the ratio between them is measured. +// Correctness alone cannot distinguish the two implementations, which is exactly +// why a token gate would have let a mask through. +// +// ─── ORACLES (AGENTS.md 'vLLM is the reference' / 'When vLLM has no impl') ─── +// vLLM registers NO `qwen4_exp` at origin/main = 6a5e8f5979 (read 2026-08-26), +// so per the developer direction recorded in the spec's `## Oracles`, +// transformers supplies the ALGORITHM and vLLM supplies the OPS. +// +// ALGORITHM huggingface/transformers v5.16.0 (the lane pin) +// src/transformers/models/qwen4_exp/modeling_qwen4_exp.py +// `Qwen4ExpTextQSAIndexer` :611-717, `Qwen4ExpTextAttention` :720+ +// (authored delta: modular_qwen4_exp.py :367-507) +// OPS vllm-project/vllm origin/main 6a5e8f5979 +// +// ─── PORT MAP, file:line on BOTH sides ────────────────────────────────────── +// OURS <- ALGORITHM (transformers) + OPS (vLLM) +// QsaValidateConfig <- configuration_qwen4_exp.py :221-231 +// (kv_heads == 1; budget % ratio == 0; +// rotary_dim = int(head_dim * partial_rotary +// _factor) must fit indexer_head_dim) +// QsaSideCacheSpec <- Cache.update_indexer +// + v1/kv_cache_interface.py MLAAttentionSpec with +// tokens_per_state, read through +// v1/attention/backends/mla/indexer.py :624-628 +// ("MLA compression is a whole number of tokens +// per state"). Key-only: one vector per state, +// not 2x for K+V. +// QsaCompressedSlot <- v1/attention/backends/mla/compressor_utils.py +// :49-61 `_compressed_slot_mapping_kernel` +// (`(pos + 1) % COMPRESS_RATIO == 0` boundary, +// `pos // COMPRESS_RATIO` slot, PAD_ID -1) +// QsaRmsNorm <- modeling_qwen4_exp.py :167-179 +// `out * rsqrt(mean(x^2) + eps) * (1.0 + weight)` +// — NOT vLLM's `out * weight` polarity +// QsaApplyRotaryLeadingHalf <- modeling_qwen4_exp.py :566-570,:573-608 +// `rotate_half` over the LEADING rotary_dim dims +// QsaCompressNormRope <- modeling_qwen4_exp.py :677-688 (mean pool over a +// NON-overlapping window of compress_ratio, the +// `.to(raw_keys.dtype)` round-trip, k_layernorm, +// RoPE at the block's FIRST token) +// + scaffolding from the TRITON head_dim=128 kernel +// models/deepseek_v4/common/ops/fused_compress_ +// quant_cache.py :677-830: boundary predicate +// :729-731, gather window :735-736, paged store +// :783-795, block-start RoPE :816. +// QsaBlockScores <- modeling_qwen4_exp.py :690-693 +// relu(q.k) summed over index heads, / sqrt(D) +// + v1/attention/ops/triton_fp8_mqa_logits.py +// :120-156 (the ReLU-then-sum MQA logit shape) +// QsaTopkBlocks <- modeling_qwen4_exp.py :695 +// + csrc/libtorch_stable/sampler.cu :391-411 +// (the rowLen <= topK shortcut: every candidate, +// ASCENDING, -1 padded) and :515 (ties resolve to +// the LOWER candidate index). Both are inherited +// for consistency with the op and neither is +// exercised by the fixtures — see QsaTopkBlocks. +// QsaSelectedTokenIndices <- modeling_qwen4_exp.py :697-703 (block -> token +// remap, the ALWAYS-attended ragged tail, and the +// budget + compress_ratio - 1 buffer width) +// QsaGatherAttention <- NOTHING UPSTREAM. Every DSv4 sparse consumer +// attends the COMPRESSED MLA KV (one state per +// four tokens); MiniMax-M3's consumers attend raw +// tokens but only at KV-page granularity. QSA +// attends RAW tokens selected at ratio-4 +// granularity, which no vLLM consumer does. +// QsaMaskedAttention <- modeling_qwen4_exp.py :705-717 + :491-496 +// (the scatter mask and its `&` with the causal +// mask). REFERENCE ONLY. +// +// ─── TWO THINGS DELIBERATELY NOT INHERITED FROM DeepSeek-V4 ───────────────── +// 1. `weights_proj`. DSv4 folds a per-(token,head) learned logit weight +// (our DsaIndexerWeightFold, sparse_attn_indexer.py :203-207, plus a +// head_scale of n_heads**-0.5 from models/deepseek_v4/attention.py :930, +// `self.n_head**-0.5` in the indexer call). +// QSA has no such tensor and no head_scale: its weight is the constant +// 1/sqrt(indexer_head_dim), applied AFTER the sum over heads. +// 2. The RoPE geometry. The DSv4 indexer kernel is GPT-J style over a TRAILING +// contiguous span (`rope_pair_local = pair_idx - NOPE_PAIRS`, adjacent +// even/odd pairs, fused_compress_quant_cache.py :806-818). QSA is NeoX +// `rotate_half` over the LEADING rotary_dim dims with the NoPE dims trailing. +// The halves are swapped end for end AND the pairing convention differs. +// A third: DeepSeek-V4's compressor does NOT mean-pool, in either of its two +// implementations. `score = tl.softmax(score, dim=0); sum(kv * score)` is a +// LEARNED softmax pool over an OVERLAPPING window of +// `(1 + OVERLAP) * COMPRESS_RATIO`, driven by a score channel this checkpoint +// does not have. Those line numbers are the TRITON +// `_fused_kv_compress_norm_rope_insert_indexer_attn` +// (fused_compress_quant_cache.py :677, softmax at :769); the CuteDSL +// `SparseAttnCompressNormRopeStoreC4Kernel` is a different file +// (models/deepseek_v4/nvidia/ops/sparse_attn_compress_cutedsl.py :75) and does +// the same online softmax at :1121-1130. Only the scaffolding is ported. +// +// ─── WHY HOST REFERENCE, AND WHAT IS UNREACHED ────────────────────────────── +// AGENTS.md 'Nothing lands dead': this TU lands UNREACHED. `Qwen4ExpTextModel` +// does not exist yet — its PLE layer (#1987), hyper-connection stream (#1988) +// and GGUF reader (#1989) are three sibling waves, and the registry entry and +// runner wiring belong to W5. Row MODEL-MM-QWEN4-EXP owns that wiring and issue +// #1978 tracks it; the spec lists this slice under `## Owed`. The device arm is +// owed too: the eventual CUDA kernel ports this same math and these functions +// are its portable oracle, exactly as deepseek_v4_dsa.h is for cuda_deepseek_v4. +#pragma once + +#include +#include + +namespace vllm::qwen4_exp { + +// ── Config ─────────────────────────────────────────────────────────────────── + +// The published Qwen3.8-Flash-Next values are the defaults. `rotary_dim` is NOT +// an independent key: upstream derives it as +// `int(head_dim * partial_rotary_factor)` = int(256 * 0.25) = 64 and then +// requires it to fit `indexer_head_dim` (configuration_qwen4_exp.py :225-231). +struct QsaConfig { + int64_t index_n_heads = 4; + int64_t index_kv_heads = 1; // upstream requires EXACTLY 1 + int64_t index_head_dim = 128; + int64_t token_budget = 2048; // TOKENS, not blocks + int64_t compress_ratio = 4; + int64_t rotary_dim = 64; // int(head_dim * partial_rotary_factor) + float rms_norm_eps = 1e-6f; + + // block_topk = token_budget // compress_ratio (modeling_qwen4_exp.py :622). + int64_t block_topk() const { return token_budget / compress_ratio; } + // The index buffer is `token_budget + compress_ratio - 1` wide because the + // incomplete trailing block is ALWAYS attended on top of the budget + // (modeling_qwen4_exp.py :661-666,:700-701). + int64_t index_width() const { return token_budget + compress_ratio - 1; } +}; + +// Mirrors `Qwen4ExpTextConfig.validate_architecture`'s QSA clause +// (configuration_qwen4_exp.py :210-231). Throws on violation. +void QsaValidateConfig(const QsaConfig& cfg); + +// ── The side cache ─────────────────────────────────────────────────────────── + +// `MLAAttentionSpec(num_kv_heads=1, head_size=index_head_dim, +// tokens_per_state=compress_ratio)`. +// +// `MLAAttentionSpec` is NOT an MLA claim. MiniMax-M3 is plain GQA and uses it +// for its own indexer cache, with the upstream comment "Key-only: +// MLAAttentionSpec budgets one vector/token (not 2x for K+V)". It is a BUDGET +// shape. `tokens_per_state` is the first-class field documented as "Ints > 1 +// compress multiple tokens into one state (DSv4 sparse MLA)", read back at +// v1/attention/backends/mla/indexer.py :624-628. +struct QsaSideCacheSpec { + int64_t num_kv_heads = 1; + int64_t head_size = 128; + int64_t tokens_per_state = 4; + int64_t elem_bytes = 2; // bf16 + + // Only a COMPLETE block produces a state: the compressor early-exits unless + // `(position + 1) % compress_ratio == 0` (compressor_utils.py :52, + // fused_compress_quant_cache.py :729-731). So this floors, and the ragged tail + // costs no state at all — it is attended from the raw KV cache instead. + int64_t StatesForTokens(int64_t num_tokens) const; + // elem_bytes * num_kv_heads * head_size / tokens_per_state. 64 B at the + // published shape, a quarter of what a per-token index cache would cost. + int64_t BytesPerTokenPerLayer() const; + int64_t BytesForTokens(int64_t num_tokens) const; +}; + +QsaSideCacheSpec QsaMakeSideCacheSpec(const QsaConfig& cfg, int64_t elem_bytes); + +// `get_compressed_slot_mapping`'s per-token body (compressor_utils.py :49-61): +// the slot a token's compressed state is written to, or -1 (PAD_ID) when the +// token is not a block boundary and therefore writes nothing. +int64_t QsaCompressedSlot(int64_t position, int64_t compress_ratio); + +// ── Primitives ─────────────────────────────────────────────────────────────── + +// `Qwen4ExpTextRMSNorm.forward` (modeling_qwen4_exp.py :167-179). Note the +// weight polarity: upstream is `out * (1.0 + weight)` with a ZERO-initialised +// weight, where vLLM's RMSNorm is `out * weight` with a ones-initialised one. +// `round_to_bf16` mirrors the closing `.type_as(x)` on a bf16 model path. +std::vector QsaRmsNorm(const std::vector& x, int64_t rows, + int64_t dim, const std::vector& weight, + float eps, bool round_to_bf16); + +// `apply_rotary_pos_emb` (modeling_qwen4_exp.py :573-604) over the LEADING +// `rotary_dim` dims, NeoX `rotate_half` pairing (:566-571), NoPE dims trailing +// and untouched. `cos`/`sin` are [rows, rotary_dim] and already carry the +// interleaved-mRoPE section layout; this function does not build them. +// `round_to_bf16` mirrors bf16 elementwise arithmetic, which rounds per op. +std::vector QsaApplyRotaryLeadingHalf(const std::vector& x, + int64_t rows, int64_t head_dim, + const std::vector& cos, + const std::vector& sin, + int64_t rotary_dim, + bool round_to_bf16); + +// The pooled-key build, i.e. the side cache's contents. +// +// `raw_keys` is [num_keys, index_head_dim] and must be RAW: un-normed and +// un-roped, exactly as `Cache.update_indexer` stores it (modeling_qwen4_exp.py +// :657-658). Per complete block, mean-pool `compress_ratio` consecutive keys in +// float, round back to the cache dtype, apply `k_layernorm`, then apply RoPE at +// the position of the block's FIRST token (:677-688). Returns +// [num_keys / compress_ratio, index_head_dim]. +// +// `cos`/`sin` are the FULL-position tables, [num_positions, rotary_dim]; the +// function selects row `compress_ratio * b` for block b, which is the same +// value the Triton kernel computes as +// `compressed_pos = (position // COMPRESS_RATIO) * COMPRESS_RATIO` +// (fused_compress_quant_cache.py :816). +// +// RECONCILIATION. Upstream forms blocks over the VISIBLE token indices of a +// padded batch (`local_visible_indices`, :668-679) and takes +// `group_starts = block_token_indices[:, 0]`. A serving engine's ragged batch +// has no interior masking — visible is the contiguous range [0, kv_len) — so +// the two coincide and block b is exactly tokens [CR*b, CR*b + CR). This +// function assumes that contiguity and asserts `num_keys % compress_ratio == 0` +// on the caller's slice; it does not accept an arbitrary visibility set. +std::vector QsaCompressNormRope(const std::vector& raw_keys, + int64_t num_keys, + const std::vector& k_norm_weight, + const std::vector& cos, + const std::vector& sin, + const QsaConfig& cfg, + bool round_to_bf16); + +// One score per COMPLETE block for one query token (modeling_qwen4_exp.py +// :690-693): +// score[b] = sum_h ReLU(dot(q[h,:], block_key[b,:])) / sqrt(index_head_dim) +// `q` is [index_n_heads, index_head_dim], already q_layernorm'd and roped. +// There is no per-head weight and no head_scale — see the header note on what +// is not inherited from DeepSeek-V4. The division is applied AFTER the sum, as +// upstream writes it. +std::vector QsaBlockScores(const std::vector& q, + const std::vector& block_keys, + int64_t num_blocks, const QsaConfig& cfg); + +// `scores.topk(min(block_topk, num_complete_blocks)).indices` +// (modeling_qwen4_exp.py :695), with two op-side semantics taken from vLLM's +// `top_k_per_row` rather than invented here: +// * when num_blocks <= k every candidate is selected and the result is +// ASCENDING block order (sampler.cu :391-402, "Indices are not sorted by +// their corresponding logit"); +// * ties resolve to the LOWER block index (sampler.cu :515, +// `logit == otherLogit && i < j`). +// Both are inherited so a QSA top-k resolves the way the op it mirrors resolves +// one. NEITHER is load-bearing here and neither is exercised: the fixtures +// produce no exact tie, and `QsaSelectedTokenIndices` re-sorts, so reversing +// either leaves the suite green. Do not read them as gated behaviour. +// Returns the selected block ids in DESCENDING score order, which is the order +// `torch.topk` produces; `QsaSelectedTokenIndices` re-sorts them. +std::vector QsaTopkBlocks(const std::vector& scores, + int64_t num_blocks, int64_t k); + +// The GATHER index buffer for one query token: selected block ids expanded to +// tokens [CR*b, CR*b + CR), plus the ALWAYS-attended ragged tail +// [CR*num_complete_blocks, kv_len), sorted ASCENDING and padded to +// `cfg.index_width()` with -1. +// +// WHY ASCENDING, when upstream's buffer is in score-rank order. Upstream +// scatters the buffer into a boolean mask (:705-712), so its order is +// unobservable. A GATHER's order IS observable: it fixes the reduction order of +// the softmax. Ascending is the order that makes a sub-budget gather reduce over +// exactly the dense sequence, which is what makes the bit-identity oracle below +// hold, and it is the order vLLM's own all-select shortcut emits +// (sampler.cu :391-402). +// +// THE FREE ORACLE. When kv_len <= token_budget + compress_ratio - 1 every +// candidate is selected, so this returns exactly [0, kv_len) followed by -1 +// padding, and `QsaGatherAttention` over it is bit-identical to dense attention. +// llama.cpp #27742 measures a max logit delta of 0.0 over all 2051 such rows. +// That gates the whole selection and masking path with no checkpoint. The dense +// side of that `==` is `QsaMaskedAttention` over the full causal prefix — a +// separate walk over every cached row. Comparing the gather to a second call to +// ITSELF, as the first revision of the test did, only restates that the function +// is deterministic: scaling its output by 2 left that case green. +std::vector QsaSelectedTokenIndices( + const std::vector& selected_blocks, int64_t num_complete_blocks, + int64_t kv_len, const QsaConfig& cfg); + +// ── Consumers ──────────────────────────────────────────────────────────────── + +// THE GATHER CONSUMER — the point of the wave. +// +// Dense GQA over ONLY the gathered rows. `q` is [num_q_heads, head_dim] for one +// query token; `k` and `v` are [kv_len, num_kv_heads, head_dim]; `indices` is +// one row of `QsaSelectedTokenIndices`, terminated by -1. Returns +// [num_q_heads, head_dim]. +// +// `keys_visited`, when non-null, receives the number of key-row reads this call +// PERFORMED — incremented at the `Dot`, never assigned from `indices`. An honest +// gather reads each selected row once per query head in each of the two softmax +// passes, so the value is `selected * num_q_heads * 2`; the mask reference +// reports `kv_len * num_q_heads * 2` in the same unit. It is the observable that +// separates the two consumers: they agree on every output value and disagree +// here by a factor of kv_len / selected_count. A gate that checks only the +// values measures correctness and NOT the speed lever, which is precisely how a +// mask-only implementation passes a token gate while forfeiting the lever. +// +// WHAT IT DOES NOT SEE, stated so nobody has to rediscover it. It counts READS, +// so a body that iterates 0..kv_len and `continue`s past the unselected rows +// without touching them still reports the sparse figure — correctly, because the +// loop counter is not the cost llama.cpp #27739 measures; the key-row traffic is. +// And it can only count reads that go through this function's own read site. +std::vector QsaGatherAttention(const std::vector& q, + const std::vector& k, + const std::vector& v, + const std::vector& indices, + int64_t kv_len, int64_t num_q_heads, + int64_t num_kv_heads, int64_t head_dim, + int64_t* keys_visited); + +// THE MASK REFERENCE — red-first only, never the production path. +// +// Dense GQA over the WHOLE cache with -inf at every unselected position, i.e. +// what upstream's scatter mask produces (modeling_qwen4_exp.py :705-717). Kept +// so the gather has something to be gated against and so the cost difference is +// measurable, and named `Reference` in its own doc rather than in a comment +// somewhere else. `keys_visited` is COUNTED at the read like the gather's, and +// comes out at `kv_len * num_q_heads * 2` because this walk reads every cached +// row whatever the mask says. +// +// Over the full causal prefix — an index buffer of exactly [0, kv_len) — this is +// plain dense attention with no position removed, which is what makes it the +// independent side of the sub-budget bit-identity oracle. +std::vector QsaMaskedAttention(const std::vector& q, + const std::vector& k, + const std::vector& v, + const std::vector& indices, + int64_t kv_len, int64_t num_q_heads, + int64_t num_kv_heads, int64_t head_dim, + int64_t* keys_visited); + +} // namespace vllm::qwen4_exp diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index fed847391..fc02e045d 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -3099,6 +3099,21 @@ target_include_directories(test_qwen3_32b_nvfp4a16_paged_engine PRIVATE vllm_cpp_add_test(test_registry_downcast_refusal vllm/models/test_registry_downcast_refusal.cpp) +# Qwen4-Exp W4 -- Qwen Sparse Attention (#1991, .agents/specs/qwen4-exp-flash-next.md). +# Gates the indexer, the side-cache sizing and the GATHER consumer against goldens +# captured by RUNNING transformers 5.16.0, this row's accepted ALGORITHM lane pin +# (vLLM registers no `qwen4_exp` at 6a5e8f5979, so the primary oracle has nothing +# to run). Two oracles need no weights: sub-budget bit-identity with dense +# attention, and index-set equality with the reference above the budget. One case +# is NOT a correctness gate at all -- `keys_visited` is what separates the gather +# from a mask, and a mask-only QSA would pass every value comparison here. +# CPU-only, no checkpoint, no GPU, no speed claim. +vllm_cpp_add_test(test_qwen4_exp_qsa vllm/models/test_qwen4_exp_qsa.cpp) +# qwen4_exp_qsa.h is a MODEL-PRIVATE header under src/, not include/vllm/: W4 ships +# nothing on the public ABI and include/vllm/ is a USER_USAGE_PREFIXES surface +# (#515). Same arrangement test_dots3_note_attn uses. +target_include_directories(test_qwen4_exp_qsa PRIVATE ${CMAKE_SOURCE_DIR}/src) + # The five suites that reach src/vllm/support/platform_compat.h, granted per # target rather than globally: the file already carries 123 explicit # ${CMAKE_SOURCE_DIR}/src grants, and a blanket one in vllm_cpp_add_test would diff --git a/tests/vllm/models/fixtures/gen_qwen4_exp_qsa_goldens.py b/tests/vllm/models/fixtures/gen_qwen4_exp_qsa_goldens.py new file mode 100644 index 000000000..2a1f9fcb0 --- /dev/null +++ b/tests/vllm/models/fixtures/gen_qwen4_exp_qsa_goldens.py @@ -0,0 +1,314 @@ +#!/usr/bin/env python3 +"""Regenerate `qwen4_exp_qsa_goldens.inc` by RUNNING the algorithm oracle. + +Row MODEL-MM-QWEN4-EXP W4, issue #1991, `.agents/specs/qwen4-exp-flash-next.md`. + +WHAT THE ORACLE IS. `transformers` **5.16.0**, the lane pin recorded in +`.agents/oracles/transformers.md` and accepted by the developer on 2026-08-26 as +this row's ALGORITHM oracle (vLLM registers no `qwen4_exp` at `6a5e8f5979`, so +there is nothing to mirror on the primary). The selected-token sets below are the +output of the UNMODIFIED `Qwen4ExpTextQSAIndexer.forward` +(`transformers/models/qwen4_exp/modeling_qwen4_exp.py`), called through its own +`__call__`. Nothing in the selection path is transcribed here: a transcription +cannot gate the function it transcribes. + +WHAT IS TRANSCRIBED, AND WHY THAT IS NOT THE GATE. Three things: + * the `torch.split` of `index_qk_proj`'s output into `(q, token_k)` + (modeling_qwen4_exp.py, `Qwen4ExpTextQSAIndexer.forward`), so the C++ side can + start from `q_raw` / `k_raw` instead of re-running a GEMM. The split is a + shape, not arithmetic, and the C++ side never reproduces the projection. + * the dense eager attention used for the CONSUMER golden -- plain + `softmax(q k^T * scale + mask) v` in float32. What is under test in the + consumer is WHICH keys are attended, and that comes from the oracle's own + mask. + * `repeat_kv` for GQA, five lines of indexing. + +WHY THE MODULE RUNS IN BFLOAT16. `pooled_keys.float().mean(1).to(raw_keys.dtype)` +in the oracle rounds the pooled key back to the cache dtype BEFORE `k_layernorm`. +At float32 that cast is a no-op and the gate would never see it; the production +side cache is bf16 (that is what makes it 64 B/token/layer), so the goldens are +captured at bf16 and the C++ reference performs the same round-trip. + +WHY TWO CASES. `sub_budget` has kv_len == indexer_budget + compress_ratio - 1, the +largest context at which EVERY candidate is still selected: llama.cpp #27742 +measures a max logit delta of 0.0 there, so QSA must be bit-identical to dense +attention. `over_budget` runs past `indexer_budget` so that top-k actually +discards blocks -- below that threshold a wrong implementation attending pooled +keys instead of real tokens is indistinguishable and still emits plausible +tokens (spec `## Gates` G2). + +Usage: + python3 tests/vllm/models/fixtures/gen_qwen4_exp_qsa_goldens.py [--out PATH] + +Requires `transformers==5.16.0` and torch. The generator REFUSES to emit under any +other transformers version, because a golden captured off an unpinned oracle is +not reproducible. +""" + +import argparse +import pathlib +import sys + +LANE_PIN = "5.16.0" + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument( + "--out", + default=str(pathlib.Path(__file__).with_name("qwen4_exp_qsa_goldens.inc")), + ) + args = ap.parse_args() + + import torch + import transformers + + if transformers.__version__ != LANE_PIN: + sys.stderr.write( + f"REFUSING: transformers {transformers.__version__} is not the lane pin " + f"{LANE_PIN}. A golden captured off an unpinned oracle is not " + f"reproducible (AGENTS.md 'Pin vLLM' / 'Pin every oracle').\n" + ) + return 2 + + from transformers.models.qwen4_exp.configuration_qwen4_exp import Qwen4ExpTextConfig + from transformers.models.qwen4_exp.modeling_qwen4_exp import ( + Qwen4ExpTextQSAIndexer, + Qwen4ExpTextRotaryEmbedding, + ) + + torch.manual_seed(1991) + dt = torch.bfloat16 + + # A deliberately tiny QSA config. Every invariant `validate_architecture` + # enforces still holds: indexer_kv_heads == 1, budget % compress_ratio == 0, + # rotary_dim (head_dim * partial_rotary_factor = 8) <= indexer_head_dim. + HID, NH, NKV, HD = 64, 4, 2, 32 + IDX_H, IDX_KV, IDX_D, BUDGET, CR = 4, 1, 16, 8, 4 + ROT = HD // 4 # partial_rotary_factor 0.25 + cfg = Qwen4ExpTextConfig( + vocab_size=128, + hidden_size=HID, + num_hidden_layers=4, + num_attention_heads=NH, + num_key_value_heads=NKV, + head_dim=HD, + rms_norm_eps=1e-6, + indexer_n_heads=IDX_H, + indexer_kv_heads=IDX_KV, + indexer_head_dim=IDX_D, + indexer_budget=BUDGET, + indexer_compress_ratio=CR, + rope_parameters={ + "rope_type": "default", + "rope_theta": 10000.0, + "partial_rotary_factor": 0.25, + "mrope_section": [2, 1, 1], + }, + ) + width = BUDGET + CR - 1 + + rotary = Qwen4ExpTextRotaryEmbedding(cfg).to(dt) + cases = [] + + for name, seq in (("sub_budget", width), ("over_budget", 23)): + indexer = Qwen4ExpTextQSAIndexer(cfg, layer_idx=0).to(dt) + # `weight` is zero-initialised on Qwen4ExpTextRMSNorm and the forward is + # `out * (1.0 + weight)`, so a zeroed norm is an identity scale. Randomise + # it, or the gate cannot see a port that drops the (1.0 + w) term. + with torch.no_grad(): + indexer.q_layernorm.weight.normal_(0.0, 0.5) + indexer.k_layernorm.weight.normal_(0.0, 0.5) + + hidden = torch.randn(1, seq, HID, dtype=dt) + pos = torch.arange(seq).view(1, seq) + cos, sin = rotary(hidden, pos) # [1, seq, ROT] each, bf16 + + # Causal bool mask, 4D, exactly the shape the oracle documents it takes + # ("always 4D with either bool (sdpa) or float (eager)"). + causal = torch.ones(seq, seq, dtype=torch.bool).tril().view(1, 1, seq, seq) + + # --- the ORACLE call, unmodified ------------------------------------- + # `apply_rotary_pos_emb` is INTERCEPTED, not reimplemented: the wrapper + # records the arguments the oracle chose and delegates to the original. + # That is how `q_post` and `block_keys` below come out of the oracle's own + # execution rather than out of a second implementation of it. In + # particular the RoPE position of a pooled key -- the block's FIRST token, + # not its last -- is captured rather than asserted. + import transformers.models.qwen4_exp.modeling_qwen4_exp as mod + + original_rope = mod.apply_rotary_pos_emb + rope_calls = [] + + def _recording_rope(*a, **kw): + out = original_rope(*a, **kw) + rope_calls.append((kw.get("cos"), out)) + return out + + mod.apply_rotary_pos_emb = _recording_rope + try: + with torch.no_grad(): + sel_mask = indexer(hidden, (cos, sin), causal, None) # [1,1,seq,seq] + finally: + mod.apply_rotary_pos_emb = original_rope + + # Call 0 is the query rope (cos is [1, seq, ROT]); every later call is one + # query's pooled-key rope (cos is [num_blocks, ROT]). The LAST one belongs + # to the last query, which sees the most complete blocks. + q_post = rope_calls[0][1][0] # [seq, IDX_H, IDX_D] + # The recorder sees the key rope BEFORE the oracle's `.squeeze(1)`. + block_keys = rope_calls[-1][1].squeeze(1) # [num_blocks, IDX_D] + assert q_post.shape == (seq, IDX_H, IDX_D), q_post.shape + assert block_keys.shape == (seq // CR, IDX_D), block_keys.shape + + # The C++ entry point starts from the projection's two halves. Transcribed + # SHAPE only (see module docstring). + with torch.no_grad(): + qk = indexer.index_qk_proj(hidden) + q_raw, k_raw = torch.split(qk, [IDX_H * IDX_D, IDX_KV * IDX_D], dim=-1) + q_raw = q_raw.reshape(1, seq, IDX_H, IDX_D) + k_raw = k_raw.reshape(1, seq, IDX_D) + + # Selected token indices per query, ASCENDING, -1 padded to `width`. The + # oracle emits a scatter mask, so ascending order is the only order the + # mask can express; the C++ gather is defined to produce the same order so + # that a sub-budget gather reduces over keys in dense positional order. + sel = torch.full((seq, width), -1, dtype=torch.int32) + for qi in range(seq): + idx = torch.nonzero(sel_mask[0, 0, qi], as_tuple=False).flatten() + assert idx.numel() <= width, (name, qi, idx.numel()) + sel[qi, : idx.numel()] = idx.to(torch.int32) + + # --- the CONSUMER golden --------------------------------------------- + attn_q = torch.randn(1, seq, NH, HD, dtype=dt) + attn_k = torch.randn(1, seq, NKV, HD, dtype=dt) + attn_v = torch.randn(1, seq, NKV, HD, dtype=dt) + combined = causal & sel_mask # oracle line: `attention_mask & selected_token_mask` + groups = NH // NKV + qf = attn_q[0].float().transpose(0, 1) # [NH, seq, HD] + kf = attn_k[0].float().transpose(0, 1).repeat_interleave(groups, 0) + vf = attn_v[0].float().transpose(0, 1).repeat_interleave(groups, 0) + logits = torch.matmul(qf, kf.transpose(-1, -2)) * (HD**-0.5) + logits = logits.masked_fill(~combined[0, 0].view(1, seq, seq), float("-inf")) + attn_out = torch.matmul(torch.softmax(logits, dim=-1), vf) # [NH, seq, HD] + attn_out = attn_out.transpose(0, 1).contiguous() # [seq, NH, HD] + + cases.append( + dict( + name=name, + seq=seq, + q_raw=q_raw[0], + k_raw=k_raw[0], + cos=cos[0], + sin=sin[0], + q_norm_w=indexer.q_layernorm.weight, + k_norm_w=indexer.k_layernorm.weight, + q_post=q_post, + block_keys=block_keys, + sel=sel, + attn_q=attn_q[0], + attn_k=attn_k[0], + attn_v=attn_v[0], + attn_out=attn_out, + ) + ) + + out = pathlib.Path(args.out) + out.write_text(_emit(cases, cfg, transformers.__version__, torch.__version__, + HID, NH, NKV, HD, IDX_H, IDX_KV, IDX_D, BUDGET, CR, ROT)) + sys.stderr.write(f"wrote {out}\n") + return 0 + + +def _wrap(items, indent=" ") -> str: + """One value per column-bounded line. `repr` on a float always emits a `.` or + an exponent, so `1.0f` never degenerates into the invalid literal `1f`.""" + out, line = [], indent + for it in items: + if len(line) + len(it) + 2 > 96: + out.append(line.rstrip()) + line = indent + line += it + ", " + if line.strip(): + out.append(line.rstrip().rstrip(",")) + else: + out[-1] = out[-1].rstrip(",") + return "\n" + "\n".join(out) + "\n" + + +def _floats(t) -> str: + import torch + + flat = t.detach().to(torch.float32).reshape(-1).tolist() + return _wrap([f"{float(v)!r}f" for v in flat]) + + +def _ints(t) -> str: + flat = t.detach().reshape(-1).tolist() + return _wrap([str(int(v)) for v in flat]) + + +def _emit(cases, cfg, tf_ver, torch_ver, HID, NH, NKV, HD, + IDX_H, IDX_KV, IDX_D, BUDGET, CR, ROT) -> str: + lines = [ + "// GENERATED by tests/vllm/models/fixtures/gen_qwen4_exp_qsa_goldens.py", + "// -- DO NOT EDIT BY HAND.", + "//", + "// Qwen4-Exp Qwen Sparse Attention (QSA) goldens, row MODEL-MM-QWEN4-EXP W4,", + "// issue #1991. Produced by RUNNING the algorithm oracle:", + f"// transformers {tf_ver} (the lane pin; see .agents/oracles/transformers.md)", + f"// torch {torch_ver}", + "// The selected-token sets are the output of the UNMODIFIED", + "// `Qwen4ExpTextQSAIndexer.forward`, not a transcription of it.", + "//", + "// Every float below is exactly bf16-representable: the oracle module runs in", + "// bfloat16 so that the `.to(raw_keys.dtype)` round-trip after the mean pool is", + "// inside the gate rather than optimised away by a float32 capture.", + "#pragma once", + "", + "#include ", + "", + "namespace qwen4_exp_qsa_goldens {", + "", + f"inline constexpr const char* kOracle = \"transformers {tf_ver}\";", + f"inline constexpr int64_t kHiddenSize = {HID};", + f"inline constexpr int64_t kNumAttentionHeads = {NH};", + f"inline constexpr int64_t kNumKeyValueHeads = {NKV};", + f"inline constexpr int64_t kHeadDim = {HD};", + f"inline constexpr int64_t kIndexNHeads = {IDX_H};", + f"inline constexpr int64_t kIndexKvHeads = {IDX_KV};", + f"inline constexpr int64_t kIndexHeadDim = {IDX_D};", + f"inline constexpr int64_t kTokenBudget = {BUDGET};", + f"inline constexpr int64_t kCompressRatio = {CR};", + f"inline constexpr int64_t kRotaryDim = {ROT};", + f"inline constexpr float kRmsNormEps = {cfg.rms_norm_eps:.9g}f;", + f"inline constexpr int64_t kIndexWidth = {BUDGET + CR - 1};", + "", + ] + for c in cases: + p = c["name"].title().replace("_", "") + lines += [ + f"// ---- case '{c['name']}': kv_len = {c['seq']} " + f"({'every candidate selected' if c['seq'] <= BUDGET + CR - 1 else 'top-k discards blocks'})", + f"inline constexpr int64_t k{p}Seq = {c['seq']};", + f"inline constexpr float k{p}QRaw[] = {{{_floats(c['q_raw'])}}};", + f"inline constexpr float k{p}KRaw[] = {{{_floats(c['k_raw'])}}};", + f"inline constexpr float k{p}Cos[] = {{{_floats(c['cos'])}}};", + f"inline constexpr float k{p}Sin[] = {{{_floats(c['sin'])}}};", + f"inline constexpr float k{p}QNormW[] = {{{_floats(c['q_norm_w'])}}};", + f"inline constexpr float k{p}KNormW[] = {{{_floats(c['k_norm_w'])}}};", + f"inline constexpr float k{p}QPost[] = {{{_floats(c['q_post'])}}};", + f"inline constexpr float k{p}BlockKeys[] = {{{_floats(c['block_keys'])}}};", + f"inline constexpr int32_t k{p}Selected[] = {{{_ints(c['sel'])}}};", + f"inline constexpr float k{p}AttnQ[] = {{{_floats(c['attn_q'])}}};", + f"inline constexpr float k{p}AttnK[] = {{{_floats(c['attn_k'])}}};", + f"inline constexpr float k{p}AttnV[] = {{{_floats(c['attn_v'])}}};", + f"inline constexpr float k{p}AttnOut[] = {{{_floats(c['attn_out'])}}};", + "", + ] + lines += ["} // namespace qwen4_exp_qsa_goldens", ""] + return "\n".join(lines) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/vllm/models/fixtures/qwen4_exp_qsa_goldens.inc b/tests/vllm/models/fixtures/qwen4_exp_qsa_goldens.inc new file mode 100644 index 000000000..dc9964ed1 --- /dev/null +++ b/tests/vllm/models/fixtures/qwen4_exp_qsa_goldens.inc @@ -0,0 +1,3366 @@ +// GENERATED by tests/vllm/models/fixtures/gen_qwen4_exp_qsa_goldens.py +// -- DO NOT EDIT BY HAND. +// +// Qwen4-Exp Qwen Sparse Attention (QSA) goldens, row MODEL-MM-QWEN4-EXP W4, +// issue #1991. Produced by RUNNING the algorithm oracle: +// transformers 5.16.0 (the lane pin; see .agents/oracles/transformers.md) +// torch 2.11.0+cu130 +// The selected-token sets are the output of the UNMODIFIED +// `Qwen4ExpTextQSAIndexer.forward`, not a transcription of it. +// +// Every float below is exactly bf16-representable: the oracle module runs in +// bfloat16 so that the `.to(raw_keys.dtype)` round-trip after the mean pool is +// inside the gate rather than optimised away by a float32 capture. +#pragma once + +#include + +namespace qwen4_exp_qsa_goldens { + +inline constexpr const char* kOracle = "transformers 5.16.0"; +inline constexpr int64_t kHiddenSize = 64; +inline constexpr int64_t kNumAttentionHeads = 4; +inline constexpr int64_t kNumKeyValueHeads = 2; +inline constexpr int64_t kHeadDim = 32; +inline constexpr int64_t kIndexNHeads = 4; +inline constexpr int64_t kIndexKvHeads = 1; +inline constexpr int64_t kIndexHeadDim = 16; +inline constexpr int64_t kTokenBudget = 8; +inline constexpr int64_t kCompressRatio = 4; +inline constexpr int64_t kRotaryDim = 8; +inline constexpr float kRmsNormEps = 1e-06f; +inline constexpr int64_t kIndexWidth = 11; + +// ---- case 'sub_budget': kv_len = 11 (every candidate selected) +inline constexpr int64_t kSubBudgetSeq = 11; +inline constexpr float kSubBudgetQRaw[] = { + -0.35546875f, 0.2001953125f, 0.125f, 0.408203125f, -0.53125f, 0.146484375f, 0.29296875f, + 0.10400390625f, 0.134765625f, 0.6640625f, -0.2421875f, -0.3515625f, 0.859375f, + -0.65234375f, -0.66796875f, -0.2236328125f, 0.045166015625f, 0.054443359375f, + -0.0615234375f, 0.03173828125f, -0.259765625f, 0.53515625f, 0.53515625f, 0.3515625f, + -0.404296875f, -0.045166015625f, -0.2265625f, -0.197265625f, 0.53125f, 0.064453125f, + -0.578125f, 0.0390625f, 0.6171875f, -0.255859375f, 0.45703125f, 0.083984375f, 0.33984375f, + 0.1748046875f, 0.498046875f, 0.447265625f, -0.291015625f, 0.494140625f, -0.1357421875f, + -0.28515625f, 0.78515625f, 0.734375f, 0.0947265625f, -0.73046875f, -0.81640625f, + -0.2001953125f, -0.1103515625f, 0.671875f, -1.0f, -0.68359375f, -0.080078125f, + -0.02294921875f, -0.423828125f, 0.0269775390625f, -0.087890625f, 0.4140625f, 0.431640625f, + 0.486328125f, 0.031494140625f, 0.376953125f, -0.5f, -0.353515625f, 0.423828125f, + 0.4765625f, 0.0213623046875f, -0.1591796875f, -0.0137939453125f, -0.359375f, 0.79296875f, + 0.66015625f, 0.353515625f, -0.044921875f, 0.8359375f, 0.037353515625f, 0.82421875f, + 0.640625f, 0.31640625f, -0.30078125f, 1.0078125f, 0.25390625f, -0.224609375f, + -0.427734375f, 0.1162109375f, 0.46484375f, -0.53125f, 0.39453125f, 0.087890625f, + 0.58203125f, 0.04736328125f, -0.166015625f, 0.76953125f, -0.703125f, -0.337890625f, + 0.045166015625f, 0.55078125f, 0.79296875f, -1.1171875f, 0.3671875f, -0.255859375f, + 0.47265625f, 0.71484375f, -0.21875f, -0.1572265625f, 0.73046875f, 0.3046875f, -0.63671875f, + 0.78125f, -0.396484375f, -0.228515625f, 0.158203125f, 0.3203125f, -0.10107421875f, + -0.546875f, 0.5f, -0.11376953125f, 0.71484375f, 0.4765625f, 0.609375f, 0.4453125f, + -0.97265625f, 0.0230712890625f, 0.2099609375f, -0.80078125f, 0.00135040283203125f, + -0.400390625f, 0.09228515625f, -0.58984375f, 0.0228271484375f, -0.69921875f, 0.7265625f, + -0.4375f, -0.51171875f, 0.69140625f, -0.0252685546875f, -0.11767578125f, -0.765625f, + 0.8359375f, -1.0546875f, 0.55078125f, 0.0177001953125f, 0.435546875f, -0.55859375f, + -0.39453125f, -0.55078125f, -0.06640625f, 0.185546875f, 0.07177734375f, 0.5f, 0.115234375f, + 0.640625f, 0.306640625f, -0.20703125f, -0.016845703125f, 0.478515625f, -1.359375f, + 0.1640625f, -0.455078125f, 1.1953125f, 0.6171875f, -0.09619140625f, 1.0390625f, + 0.55859375f, -0.1884765625f, 0.271484375f, 0.84375f, -0.63671875f, 0.59765625f, + 0.052490234375f, 0.60546875f, -0.68359375f, -0.83984375f, -0.84765625f, -0.71484375f, + 0.75390625f, -0.26953125f, -0.04296875f, -0.2109375f, 0.439453125f, 0.54296875f, + -0.8515625f, -1.4609375f, -0.15625f, -0.0703125f, -0.8359375f, -0.4296875f, -1.6875f, + 0.06884765625f, 0.62890625f, -0.78515625f, -0.63671875f, 0.373046875f, 0.2060546875f, + 0.263671875f, -0.37109375f, -0.1826171875f, 0.49609375f, 0.5f, -0.19140625f, -0.88671875f, + 0.376953125f, -0.38671875f, -0.1376953125f, 0.166015625f, 0.9375f, 0.48046875f, -0.078125f, + 0.34765625f, -0.1064453125f, 0.10546875f, -0.291015625f, -0.037109375f, -0.169921875f, + -0.2001953125f, -0.146484375f, 1.25f, 0.197265625f, 0.06591796875f, 0.421875f, + -0.95703125f, -0.474609375f, 0.0439453125f, -0.1328125f, -0.443359375f, 0.8125f, + 0.7734375f, 0.08984375f, -0.1435546875f, 0.6484375f, 0.1328125f, -0.6796875f, 0.498046875f, + 1.0546875f, -0.33984375f, -0.1376953125f, 0.1591796875f, -0.609375f, -0.369140625f, + 0.39453125f, -0.158203125f, 0.189453125f, -0.333984375f, 0.54296875f, -0.96875f, + 0.057861328125f, 0.625f, 0.18359375f, -0.1484375f, 0.265625f, -0.546875f, 0.392578125f, + 0.95703125f, -0.279296875f, 0.216796875f, -0.30859375f, 0.34375f, 0.82421875f, + -0.04833984375f, 0.2490234375f, 1.03125f, -0.61328125f, -0.3828125f, 0.291015625f, + -0.00096893310546875f, 0.26953125f, 0.024169921875f, -0.040771484375f, 0.69921875f, + -0.298828125f, -1.2109375f, 1.1875f, -0.28515625f, -0.87109375f, -0.65625f, 0.291015625f, + 0.0986328125f, -0.6953125f, 0.376953125f, -0.43359375f, 0.337890625f, -1.0078125f, -0.25f, + -0.48828125f, -0.12890625f, -0.71484375f, 0.28515625f, 0.2001953125f, -0.314453125f, + -0.71875f, -1.1015625f, -0.08349609375f, -0.287109375f, -0.0059814453125f, 1.3046875f, + -0.1748046875f, 1.0859375f, 0.302734375f, -1.4296875f, -0.51171875f, 0.5f, -0.51171875f, + 0.78125f, -0.462890625f, -0.953125f, -0.34765625f, 0.6328125f, 1.2578125f, 0.65625f, + -0.77734375f, 1.7734375f, -0.77734375f, 0.74609375f, -0.0732421875f, -0.1591796875f, + -0.283203125f, 0.96875f, 0.25390625f, -0.0966796875f, -0.04833984375f, 0.080078125f, + -0.30078125f, 1.203125f, 0.380859375f, 1.140625f, 0.39453125f, -0.08642578125f, + -0.72265625f, -0.337890625f, 1.1796875f, -0.7109375f, 0.76171875f, -0.5390625f, + -0.70703125f, -0.7578125f, 1.109375f, -0.1337890625f, -0.064453125f, -1.9609375f, + -0.546875f, 1.8125f, -0.150390625f, 0.1923828125f, 1.015625f, -0.44140625f, + 0.0101318359375f, 0.59375f, -1.2421875f, 0.07373046875f, 0.0751953125f, 0.33984375f, + -0.1123046875f, -0.28515625f, -0.4296875f, 0.033203125f, 0.068359375f, 0.5546875f, + -0.310546875f, -0.84375f, -0.57421875f, -0.76953125f, 0.001434326171875f, -0.010009765625f, + -0.75f, 1.015625f, -0.1650390625f, 0.8671875f, -0.486328125f, -0.51953125f, 1.1171875f, + -0.12451171875f, 1.4609375f, -0.06640625f, 0.85546875f, -0.423828125f, -0.298828125f, + 0.2578125f, 0.2490234375f, 0.197265625f, 0.34765625f, -0.9140625f, -0.318359375f, + -0.1826171875f, -0.0693359375f, -1.53125f, 0.134765625f, 0.0888671875f, 0.032958984375f, + -0.40234375f, -0.0225830078125f, -0.265625f, -0.15625f, 0.0556640625f, -0.83203125f, + 0.2431640625f, -0.447265625f, -0.7578125f, -0.390625f, -0.1748046875f, 1.078125f, + -0.0244140625f, -0.0123291015625f, -1.046875f, -0.062255859375f, -0.94140625f, + -0.51953125f, -0.671875f, -0.0045166015625f, -0.201171875f, -0.76171875f, -0.625f, + -0.392578125f, 0.2734375f, -0.5546875f, -0.77734375f, 0.1728515625f, -0.04296875f, + 0.080078125f, 1.265625f, -1.1328125f, 0.49609375f, 0.427734375f, 0.035888671875f, + -0.380859375f, 0.1865234375f, -0.5859375f, 1.2578125f, 0.08056640625f, -0.12060546875f, + 0.19921875f, 0.11474609375f, -0.65625f, -0.4296875f, 0.1240234375f, 0.2333984375f, + -0.000728607177734375f, 0.625f, 0.859375f, -0.92578125f, 1.125f, -0.0263671875f, + 0.52734375f, 0.03564453125f, -0.15234375f, -0.08154296875f, -0.39453125f, -0.251953125f, + -0.330078125f, 0.2177734375f, 1.0546875f, -0.765625f, -0.26171875f, -0.59765625f, + -1.5078125f, 0.50390625f, -0.30078125f, -0.40625f, 0.251953125f, 0.3828125f, -0.49609375f, + 0.63671875f, 0.51953125f, -0.08154296875f, 0.56640625f, 0.625f, 0.4609375f, -0.490234375f, + 0.52734375f, 1.4921875f, 0.54296875f, 0.73828125f, 0.474609375f, 0.58203125f, 0.1953125f, + 0.048583984375f, -0.34375f, -0.408203125f, -0.671875f, 0.04150390625f, 0.54296875f, + 0.4921875f, 0.047119140625f, 0.169921875f, 0.1982421875f, -0.072265625f, 0.2197265625f, + 0.482421875f, -0.9453125f, 0.400390625f, -0.44921875f, -0.296875f, -0.6171875f, + -0.56640625f, -1.296875f, 0.31640625f, -0.0302734375f, -0.71875f, 0.0673828125f, + 0.248046875f, 0.404296875f, 0.55078125f, -0.41015625f, 0.2890625f, -0.3203125f, 0.71875f, + -1.5390625f, -0.421875f, -0.93359375f, 0.00390625f, 0.6953125f, -0.3671875f, 0.30078125f, + -1.078125f, 0.318359375f, 0.5078125f, -0.498046875f, 0.1611328125f, -0.007049560546875f, + -0.21875f, -0.333984375f, -0.0400390625f, -0.478515625f, 1.1640625f, -0.26171875f, + 0.25390625f, 0.2314453125f, 0.52734375f, 0.43359375f, 0.0213623046875f, 0.12158203125f, + -0.08251953125f, -0.80078125f, -0.234375f, 0.25f, -0.029052734375f, -0.0830078125f, + 0.5078125f, -0.53515625f, -0.71484375f, 0.251953125f, 0.86328125f, 0.640625f, -0.27734375f, + -0.384765625f, -0.0023040771484375f, 0.00101470947265625f, 1.2265625f, 0.1455078125f, + 0.05859375f, 0.65234375f, -0.65625f, -0.1064453125f, 0.34375f, -0.01055908203125f, + 1.2734375f, -0.453125f, -0.625f, 0.4765625f, 0.06787109375f, 0.26953125f, -0.498046875f, + 0.1259765625f, -0.146484375f, -0.035888671875f, -0.030517578125f, -0.1064453125f, + -0.080078125f, 1.46875f, 0.369140625f, 0.022705078125f, -0.625f, -0.65625f, 0.21875f, + 0.38671875f, 0.9140625f, 0.06640625f, 0.83984375f, 0.5390625f, -0.0830078125f, -1.0078125f, + 0.345703125f, -1.203125f, -0.5390625f, 1.0859375f, -0.0247802734375f, 0.09619140625f, + 0.8984375f, -0.07763671875f, 0.138671875f, 0.0027923583984375f, 0.04296875f, 0.66796875f, + 0.333984375f, 0.013671875f, -0.55859375f, -0.25f, -0.70703125f, -0.166015625f, + 0.181640625f, 0.421875f, 0.2578125f, 0.2001953125f, 0.65234375f, 0.93359375f, 0.25f, + 0.10400390625f, 0.62890625f, -0.53515625f, 0.30859375f, 0.59765625f, -0.166015625f, + -0.1943359375f, -0.90234375f, 0.1552734375f, 1.3515625f, 0.41015625f, -0.61328125f, + 0.51171875f, -0.043701171875f, -0.60546875f, -0.314453125f, -0.0947265625f, 0.6640625f, + -0.94140625f, 0.69921875f, -0.455078125f, -0.1494140625f, 0.7109375f, 1.359375f, + -0.0303955078125f, -0.55078125f, 0.1728515625f, 0.59765625f, -0.1650390625f, 0.4140625f, + -0.08251953125f, 0.287109375f, 0.6796875f, 0.365234375f, -0.251953125f, 0.921875f, + -0.341796875f, -0.1474609375f, -0.77734375f, 0.29296875f, 0.53125f, 0.2099609375f, + 0.400390625f, -0.056640625f, 1.1796875f, -0.0020294189453125f, 1.1875f, -0.26953125f, + -1.0f, 0.2001953125f, -1.25f, 0.35546875f, 0.28515625f, -0.87109375f, -0.07666015625f, + -0.140625f, -0.60546875f, 0.053466796875f, 0.3828125f, 0.06591796875f, 0.734375f, + -0.65625f, -0.4453125f, -0.236328125f, 0.2099609375f, 0.8828125f, 0.0390625f, -1.3828125f, + 0.142578125f, 0.515625f, 0.703125f, 0.267578125f, 0.0517578125f, -0.470703125f, -0.828125f, + -0.6953125f, 0.314453125f, -0.671875f, 0.68359375f, -0.099609375f, 0.1357421875f, 0.6875f, + 0.26171875f, -0.271484375f, -0.33984375f, -0.09130859375f, 0.255859375f, 0.5546875f, + 0.05078125f, 0.2578125f, 0.044921875f, 0.64453125f, -0.412109375f, 0.87890625f, 0.5390625f, + 0.126953125f, -0.173828125f +}; +inline constexpr float kSubBudgetKRaw[] = { + 0.0250244140625f, -0.2578125f, -0.56640625f, -0.271484375f, 0.259765625f, 0.162109375f, + 0.484375f, 0.1845703125f, 0.12451171875f, 1.0625f, -0.016845703125f, 0.01171875f, + -0.58203125f, 0.203125f, 0.0306396484375f, 0.006134033203125f, -0.166015625f, 0.3125f, + 0.921875f, 0.34765625f, -1.1640625f, 0.6171875f, -0.1865234375f, -0.484375f, 0.146484375f, + -0.296875f, 0.130859375f, -0.037841796875f, 0.06787109375f, -1.1171875f, 0.26171875f, + -0.29296875f, -0.40234375f, 0.27734375f, -0.2333984375f, 0.01904296875f, 0.66015625f, + 0.796875f, 0.6484375f, -0.8046875f, 0.038330078125f, 0.412109375f, -0.337890625f, + 0.369140625f, -0.671875f, -0.162109375f, -0.51171875f, 1.015625f, 0.9140625f, 0.109375f, + -0.19140625f, -0.00506591796875f, -0.115234375f, 0.69140625f, 0.2890625f, 1.484375f, + -0.5546875f, 0.50390625f, -0.328125f, 0.2890625f, -0.376953125f, 0.1142578125f, + 0.69921875f, -0.66015625f, 0.052734375f, 0.439453125f, 0.361328125f, 0.006011962890625f, + -0.06396484375f, 0.06005859375f, 0.5859375f, 1.109375f, -0.5546875f, 0.3359375f, + -0.042724609375f, -0.462890625f, 0.419921875f, 0.050537109375f, 0.66015625f, + 0.004852294921875f, -0.431640625f, 0.052978515625f, 0.451171875f, 0.5390625f, 0.5859375f, + -0.314453125f, 0.71875f, 0.0595703125f, 1.1015625f, 0.0439453125f, -0.4453125f, + -0.212890625f, 0.82421875f, 0.158203125f, -1.046875f, -0.11962890625f, -0.6640625f, + -0.07958984375f, -0.248046875f, -0.95703125f, 0.76171875f, -0.15234375f, -0.71484375f, + 0.78515625f, 0.265625f, 0.1728515625f, -0.34765625f, -0.58203125f, 0.125f, 0.59375f, + -0.40625f, -0.275390625f, -0.158203125f, -0.455078125f, 0.1865234375f, 0.310546875f, + -0.484375f, 0.09814453125f, 0.578125f, 0.1181640625f, 0.423828125f, 0.056396484375f, + -0.028564453125f, -0.28125f, 0.52734375f, 0.1376953125f, 0.296875f, -0.09033203125f, + 0.61328125f, -0.099609375f, 0.81640625f, -0.5390625f, -0.23828125f, -0.2041015625f, 1.25f, + 0.85546875f, 0.98828125f, 0.30078125f, -0.640625f, 0.296875f, -0.30859375f, + 0.048095703125f, -0.263671875f, 0.279296875f, -0.40625f, -0.6171875f, 0.40234375f, + -0.34765625f, 0.4609375f, 0.09033203125f, -0.88671875f, -0.384765625f, -0.5546875f, + -0.427734375f, -0.0732421875f, 0.1708984375f, -0.1376953125f, -0.08251953125f, -0.890625f, + 0.5234375f, 0.3828125f, 0.8671875f, -0.7421875f, 0.8046875f, -0.11865234375f, + 0.0162353515625f, -0.275390625f, -0.80078125f, -0.373046875f, 0.2265625f, -0.1435546875f, + -0.10986328125f, 0.185546875f, 0.2451171875f, 0.08935546875f, -0.0184326171875f +}; +inline constexpr float kSubBudgetCos[] = { + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 0.5390625f, 0.99609375f, 1.0f, 1.0f, + 0.5390625f, 0.99609375f, 1.0f, 1.0f, -0.416015625f, 0.98046875f, 1.0f, 1.0f, -0.416015625f, + 0.98046875f, 1.0f, 1.0f, -0.98828125f, 0.95703125f, 1.0f, 1.0f, -0.98828125f, 0.95703125f, + 1.0f, 1.0f, -0.65234375f, 0.921875f, 1.0f, 1.0f, -0.65234375f, 0.921875f, 1.0f, 1.0f, + 0.283203125f, 0.87890625f, 1.0f, 1.0f, 0.283203125f, 0.87890625f, 1.0f, 1.0f, 0.9609375f, + 0.82421875f, 1.0f, 1.0f, 0.9609375f, 0.82421875f, 1.0f, 1.0f, 0.75390625f, 0.765625f, + 0.99609375f, 1.0f, 0.75390625f, 0.765625f, 0.99609375f, 1.0f, -0.1455078125f, 0.6953125f, + 0.99609375f, 1.0f, -0.1455078125f, 0.6953125f, 0.99609375f, 1.0f, -0.91015625f, + 0.62109375f, 0.99609375f, 1.0f, -0.91015625f, 0.62109375f, 0.99609375f, 1.0f, -0.83984375f, + 0.5390625f, 0.99609375f, 1.0f, -0.83984375f, 0.5390625f, 0.99609375f, 1.0f +}; +inline constexpr float kSubBudgetSin[] = { + 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.83984375f, 0.10009765625f, + 0.010009765625f, 0.00099945068359375f, 0.83984375f, 0.10009765625f, 0.010009765625f, + 0.00099945068359375f, 0.91015625f, 0.19921875f, 0.02001953125f, 0.0019989013671875f, + 0.91015625f, 0.19921875f, 0.02001953125f, 0.0019989013671875f, 0.1416015625f, 0.294921875f, + 0.030029296875f, 0.00299072265625f, 0.1416015625f, 0.294921875f, 0.030029296875f, + 0.00299072265625f, -0.7578125f, 0.390625f, 0.0400390625f, 0.003997802734375f, -0.7578125f, + 0.390625f, 0.0400390625f, 0.003997802734375f, -0.95703125f, 0.48046875f, 0.050048828125f, + 0.0050048828125f, -0.95703125f, 0.48046875f, 0.050048828125f, 0.0050048828125f, + -0.279296875f, 0.56640625f, 0.06005859375f, 0.0059814453125f, -0.279296875f, 0.56640625f, + 0.06005859375f, 0.0059814453125f, 0.65625f, 0.64453125f, 0.06982421875f, + 0.006988525390625f, 0.65625f, 0.64453125f, 0.06982421875f, 0.006988525390625f, 0.98828125f, + 0.71875f, 0.080078125f, 0.00799560546875f, 0.98828125f, 0.71875f, 0.080078125f, + 0.00799560546875f, 0.412109375f, 0.78515625f, 0.08984375f, 0.00897216796875f, 0.412109375f, + 0.78515625f, 0.08984375f, 0.00897216796875f, -0.54296875f, 0.84375f, 0.10009765625f, + 0.010009765625f, -0.54296875f, 0.84375f, 0.10009765625f, 0.010009765625f +}; +inline constexpr float kSubBudgetQNormW[] = { + -0.921875f, 0.33203125f, 0.263671875f, 0.1552734375f, -0.38671875f, -1.3828125f, + -0.19140625f, 0.341796875f, -0.91015625f, -0.2734375f, 0.1015625f, -0.14453125f, + 0.0185546875f, -0.2392578125f, 0.625f, 1.25f +}; +inline constexpr float kSubBudgetKNormW[] = { + -0.11376953125f, -0.36328125f, 0.46484375f, -0.50390625f, -0.5f, 0.287109375f, + -0.0162353515625f, -0.8125f, 0.34375f, 0.73046875f, -0.294921875f, -0.111328125f, + -0.8046875f, -1.4609375f, 0.70703125f, -0.06298828125f +}; +inline constexpr float kSubBudgetQPost[] = { + -0.0634765625f, 0.609375f, 0.361328125f, 1.078125f, -0.74609375f, -0.12890625f, + 0.54296875f, 0.3203125f, 0.0277099609375f, 1.1015625f, -0.609375f, -0.6875f, 2.0f, + -1.1328125f, -2.484375f, -1.1484375f, 0.010986328125f, 0.2255859375f, -0.2421875f, + 0.1142578125f, -0.49609375f, -0.63671875f, 1.34375f, 1.46875f, -0.11328125f, + -0.10205078125f, -0.77734375f, -0.5234375f, 1.6875f, 0.15234375f, -2.921875f, 0.2734375f, + 0.10498046875f, -0.7421875f, 1.2578125f, 0.2109375f, 0.453125f, -0.1455078125f, 0.875f, + 1.3046875f, -0.056884765625f, 0.78125f, -0.32421875f, -0.53125f, 1.7421875f, 1.21875f, + 0.333984375f, -3.578125f, -0.134765625f, -0.56640625f, -0.294921875f, 1.640625f, + -1.296875f, 0.5546875f, -0.13671875f, -0.0654296875f, -0.08056640625f, 0.04150390625f, + -0.205078125f, 0.75f, 0.9296875f, 0.78515625f, 0.1083984375f, 1.796875f, -0.06494140625f, + -0.9609375f, 1.0859375f, 1.109375f, -0.05224609375f, 0.02734375f, -0.01171875f, + -0.97265625f, 0.1435546875f, 0.96875f, 0.78515625f, -0.07763671875f, 1.71875f, + 0.057373046875f, 2.703125f, 2.90625f, 0.271484375f, -0.87109375f, 2.671875f, 0.6171875f, + -0.11279296875f, 0.2578125f, 0.2236328125f, 1.3125f, -0.10009765625f, 0.6015625f, + 0.203125f, 1.046875f, 0.10107421875f, -0.265625f, 2.625f, -3.3125f, 0.9921875f, + 0.130859375f, 1.234375f, 1.625f, -0.69140625f, -0.2373046875f, -0.35546875f, 1.125f, + 0.11376953125f, -0.28125f, -0.306640625f, 1.109375f, 0.55078125f, -0.859375f, 2.25f, + -1.578125f, 0.56640625f, 0.4765625f, 0.84375f, -0.2451171875f, -0.408203125f, -0.3515625f, + -0.1826171875f, 2.0f, 0.08935546875f, 0.921875f, 1.0234375f, -1.734375f, 0.048828125f, + 0.33203125f, -2.703125f, 0.006317138671875f, 0.7109375f, 0.30859375f, -1.296875f, + 0.048828125f, 0.263671875f, -0.435546875f, -0.6484375f, -1.203125f, 0.109375f, + -0.0322265625f, -0.2275390625f, -1.1484375f, 1.5f, -1.40625f, 1.5703125f, 0.06982421875f, + 0.04638671875f, -1.4375f, -1.0078125f, -1.2890625f, 0.0966796875f, -0.439453125f, + 0.09716796875f, 1.359375f, 0.0208740234375f, 0.94140625f, 0.68359375f, -0.357421875f, + -0.03466796875f, 0.734375f, -4.46875f, 0.74609375f, -0.8359375f, 2.375f, 1.1640625f, + -0.166015625f, -0.44140625f, 0.158203125f, -0.203125f, 0.5390625f, 0.1123046875f, -0.6875f, + 0.9765625f, 0.06689453125f, 0.9140625f, -0.7734375f, -2.03125f, -2.828125f, 0.19140625f, + 1.3828125f, -0.47265625f, -0.064453125f, 0.00439453125f, 0.0478515625f, 0.5859375f, + -1.546875f, -0.177734375f, -0.154296875f, -0.10498046875f, -0.96875f, -0.59375f, + -1.7421875f, 0.1513671875f, 1.921875f, 0.07568359375f, -1.71875f, 0.953125f, 0.47265625f, + -0.337890625f, -0.23046875f, -0.267578125f, 1.3359375f, 0.08984375f, -0.279296875f, + -1.9609375f, 0.6484375f, -0.7890625f, -0.2099609375f, 0.5390625f, 4.21875f, + -0.09912109375f, -0.283203125f, 0.94140625f, -0.26171875f, -0.125f, 0.1630859375f, + -0.0361328125f, -0.48828125f, -0.03857421875f, -0.228515625f, 2.953125f, 0.361328125f, + 0.1435546875f, 0.6875f, -3.328125f, -2.296875f, -0.13671875f, -0.306640625f, -1.0703125f, + 1.8046875f, -0.90234375f, -0.1640625f, -0.255859375f, 1.6875f, 0.0230712890625f, + -0.953125f, 1.0625f, 1.7421875f, -0.66796875f, -0.2021484375f, 0.5f, -2.640625f, + 0.1201171875f, 1.1796875f, -0.369140625f, 0.45703125f, 0.4140625f, -0.08984375f, + -1.6484375f, 0.1630859375f, 0.1171875f, 0.279296875f, -0.341796875f, 0.474609375f, + -1.1640625f, 0.625f, 3.25f, -1.3125f, -0.0732421875f, -0.75f, 0.87890625f, 2.09375f, + 0.014404296875f, -0.54296875f, 1.8671875f, -1.796875f, -0.0751953125f, 0.462890625f, + -0.0023345947265625f, 0.50390625f, 0.053955078125f, -0.06787109375f, 2.484375f, + -1.4765625f, -0.3671875f, 2.265625f, -0.546875f, -1.515625f, 0.5078125f, 0.77734375f, + 0.0986328125f, -1.4140625f, 0.05126953125f, -0.4765625f, 0.5625f, -1.3046875f, + -0.384765625f, -0.5625f, -0.31640625f, -2.421875f, -0.75f, 0.333984375f, -0.5546875f, + -1.1875f, 0.60546875f, 0.1904296875f, -0.35546875f, -0.0162353515625f, 0.1669921875f, + -0.181640625f, 1.703125f, 0.369140625f, -2.078125f, -0.5546875f, 1.15625f, -1.640625f, + 0.31640625f, -0.4765625f, -1.5234375f, -0.494140625f, -0.375f, -0.85546875f, 0.6015625f, + -1.296875f, 0.1982421875f, -0.703125f, 1.0234375f, -0.078125f, -0.2021484375f, + -0.26953125f, 1.9609375f, 0.7109375f, 1.0625f, 0.0205078125f, 0.08251953125f, -0.52734375f, + 0.326171875f, -0.240234375f, 1.3984375f, 0.79296875f, -0.01165771484375f, -0.7890625f, + -0.55859375f, 1.515625f, -1.0859375f, 0.87109375f, -1.3203125f, -2.390625f, -1.3203125f, + 1.3515625f, -0.275390625f, -0.08349609375f, -0.322265625f, 1.015625f, 1.65625f, + -0.228515625f, 0.0196533203125f, 0.8359375f, -0.55078125f, 0.00982666015625f, 0.6875f, + -1.0703125f, 0.1357421875f, 0.19140625f, 0.0537109375f, -0.236328125f, -0.7578125f, + -0.98046875f, -0.0390625f, -0.1884765625f, 0.85546875f, -0.83203125f, -0.150390625f, + -0.828125f, -1.6796875f, 0.00244140625f, -0.020263671875f, -1.1328125f, 3.28125f, + -0.73828125f, -0.08203125f, -0.458984375f, -0.99609375f, 1.9609375f, -0.1318359375f, + -1.2265625f, -0.1318359375f, 1.7578125f, -0.05810546875f, -0.330078125f, 0.43359375f, + 0.32421875f, 0.306640625f, 0.40234375f, -2.265625f, -1.09375f, 0.0029296875f, + -0.1318359375f, -3.6875f, 0.298828125f, 0.1083984375f, -0.1201171875f, -0.84765625f, + -0.056396484375f, -0.045654296875f, -0.2177734375f, 0.11767578125f, -1.3671875f, + 0.474609375f, -0.65234375f, -2.359375f, -1.6875f, -0.33203125f, 1.9921875f, + 0.025146484375f, -0.017333984375f, -1.046875f, 1.421875f, -1.3046875f, -1.1953125f, + -0.103515625f, -0.005615234375f, -0.37890625f, -1.1171875f, -1.09375f, -0.51171875f, + 0.76171875f, -2.140625f, 0.2578125f, -0.08984375f, -0.126953125f, 0.1435546875f, + 1.2421875f, 0.7890625f, 0.64453125f, 0.92578125f, 0.005218505859375f, -0.447265625f, + 0.33203125f, -0.80859375f, 2.078125f, 0.09912109375f, -0.31640625f, 0.72265625f, + 0.0927734375f, -1.375f, -1.0859375f, 0.259765625f, 0.259765625f, -0.94140625f, 0.8984375f, + 2.1875f, -0.158203125f, 1.5546875f, -0.05517578125f, 0.859375f, 0.0693359375f, + -0.220703125f, -0.251953125f, -1.6875f, 0.46875f, -0.640625f, 0.4921875f, 1.96875f, + -0.5859375f, -0.330078125f, -0.73828125f, -3.21875f, 0.072265625f, -0.349609375f, + -0.71484375f, 0.34375f, 0.625f, -0.6015625f, 1.65625f, 1.8671875f, 0.318359375f, 1.171875f, + 1.15625f, 0.87109375f, -0.380859375f, 0.546875f, 2.078125f, 1.2109375f, 0.10986328125f, + 0.5703125f, 1.0625f, 0.275390625f, 0.08203125f, -0.431640625f, -1.09375f, -2.5f, + -0.12353515625f, 1.125f, 1.15625f, 0.09716796875f, 0.150390625f, 0.7578125f, + -0.02783203125f, 0.546875f, 0.08056640625f, -1.28125f, 0.8203125f, -0.71484375f, -0.5625f, + -0.875f, -1.7109375f, -5.4375f, -0.1337890625f, 0.11328125f, -1.5390625f, 0.134765625f, + 0.21484375f, -0.2373046875f, 0.62109375f, -0.90625f, 0.042724609375f, -0.3828125f, + 1.3046875f, -2.15625f, -0.70703125f, -1.1640625f, 0.01043701171875f, 2.578125f, -0.609375f, + 0.283203125f, -2.734375f, 0.73828125f, -0.1474609375f, 0.84375f, 0.041015625f, + -0.01318359375f, -0.03955078125f, -0.48828125f, -0.0888671875f, -0.82421875f, 2.390625f, + -0.400390625f, 0.83203125f, 1.046875f, 0.09375f, 0.3828125f, 0.08984375f, 0.29296875f, + 0.1025390625f, 1.3359375f, -0.3984375f, 0.71875f, -0.00555419921875f, -0.12890625f, + 1.1875f, -0.9765625f, -1.546875f, 0.408203125f, 2.984375f, 3.0625f, -1.296875f, + -0.5546875f, -0.01171875f, -0.01025390625f, -0.23046875f, -0.7109375f, 0.08203125f, + 1.5390625f, -0.103515625f, -0.1357421875f, 0.6640625f, -0.015869140625f, 2.28125f, + -0.60546875f, -1.78125f, 1.8828125f, 0.1826171875f, 0.5f, -1.296875f, 0.3046875f, + 0.0380859375f, 0.5546875f, -0.15625f, -0.294921875f, -0.01495361328125f, 2.21875f, + 0.84765625f, 0.04052734375f, -1.328125f, -1.0390625f, 0.73828125f, 1.8125f, -0.0673828125f, + -0.3828125f, 1.5859375f, 0.984375f, 0.1171875f, 0.4765625f, 0.578125f, -2.46875f, + -0.07470703125f, 1.21875f, -0.0419921875f, 0.126953125f, 1.4140625f, -0.0908203125f, + 0.34765625f, 0.00970458984375f, 0.310546875f, 1.0703125f, 1.0625f, 0.0400390625f, + 0.70703125f, 1.703125f, -1.1953125f, -0.50390625f, 0.036865234375f, 0.69140625f, 0.640625f, + 0.38671875f, 1.5f, 1.6015625f, 0.9140625f, 0.52734375f, -0.00439453125f, -0.8828125f, + 0.80078125f, 1.21875f, 0.19921875f, -0.90625f, -1.21875f, 0.37890625f, 0.2138671875f, + 0.52734375f, -1.1875f, 0.7734375f, -0.07861328125f, -0.8125f, -0.90234375f, -0.375f, + -0.015625f, -0.9375f, 1.2890625f, -0.87109375f, 0.1728515625f, -1.9140625f, 1.9453125f, + -0.0751953125f, -0.08203125f, 0.2080078125f, 1.0859375f, -0.2333984375f, 0.69921875f, + -0.10400390625f, 0.7734375f, 2.53125f, -0.1171875f, -0.6953125f, 1.8203125f, -0.64453125f, + 0.0966796875f, -0.197265625f, 0.5625f, 1.1328125f, 0.0301513671875f, 0.466796875f, + -0.10009765625f, 1.6171875f, -0.0033111572265625f, 1.4453125f, -0.703125f, -3.609375f, + -0.578125f, -1.765625f, 0.87890625f, 0.64453125f, 0.84375f, -2.65625f, -0.1298828125f, + -1.546875f, 0.0091552734375f, 0.53125f, 0.138671875f, 1.203125f, -1.2734375f, -0.6484375f, + -0.734375f, 0.90234375f, 0.1845703125f, 0.41796875f, -2.875f, 0.267578125f, -0.494140625f, + -0.166015625f, 0.064453125f, 0.11669921875f, -0.0693359375f, -0.984375f, -1.25f, + 0.439453125f, -1.1171875f, 0.8515625f, -0.265625f, 0.5f, -0.177734375f, 0.63671875f, + -0.91015625f, -0.92578125f, 0.0419921875f, 0.56640625f, 0.96484375f, 0.15234375f, + 0.0546875f, 0.0771484375f, 1.671875f, -0.83203125f, 2.109375f, 0.96875f, 0.486328125f, + -0.921875f +}; +inline constexpr float kSubBudgetBlockKeys[] = { + 0.345703125f, 0.294921875f, -0.10693359375f, 0.047119140625f, -0.189453125f, 3.0625f, + 1.2734375f, 0.07470703125f, -0.345703125f, 3.0625f, -0.408203125f, 0.58984375f, + -0.3203125f, 0.466796875f, 0.86328125f, 0.06787109375f, 0.9296875f, 0.12158203125f, + 0.98046875f, -0.04833984375f, 0.5078125f, -0.349609375f, 1.1171875f, 0.36328125f, + 1.546875f, 0.984375f, -0.56640625f, -1.2734375f, 0.345703125f, -0.404296875f, -0.7890625f, + -0.419921875f +}; +inline constexpr int32_t kSubBudgetSelected[] = { + 0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0, 1, + 2, -1, -1, -1, -1, -1, -1, -1, -1, 0, 1, 2, 3, -1, -1, -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, + -1, -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, 5, -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, 5, 6, -1, -1, + -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, -1, -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, -1, -1, 0, 1, 2, 3, + 4, 5, 6, 7, 8, 9, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 +}; +inline constexpr float kSubBudgetAttnQ[] = { + 0.000453948974609375f, 0.302734375f, -0.06396484375f, -0.30859375f, 1.2734375f, + -0.31640625f, -0.67578125f, -1.4375f, 0.9375f, 0.248046875f, 0.51171875f, -0.88671875f, + 0.9375f, -1.7265625f, 0.9296875f, 0.59375f, -0.63671875f, 0.75390625f, -0.98046875f, + -0.2431640625f, -2.140625f, 1.0625f, -1.34375f, 0.0f, 1.65625f, 0.50390625f, 0.75390625f, + 0.734375f, 1.4453125f, 1.984375f, 0.7890625f, 0.0f, 0.55078125f, -1.109375f, -0.51171875f, + -1.5078125f, 0.37109375f, 0.271484375f, -1.3828125f, 0.349609375f, -0.66015625f, + -0.6953125f, 0.54296875f, -1.75f, -0.72265625f, 0.427734375f, -1.3203125f, 0.125f, + -0.90625f, -0.333984375f, 0.3984375f, 0.1162109375f, 2.609375f, 0.1416015625f, 0.546875f, + 0.79296875f, -0.7890625f, 0.50390625f, 1.453125f, -0.0458984375f, -1.03125f, 0.9453125f, + 0.82421875f, -0.8984375f, -2.03125f, 0.6328125f, -1.40625f, -1.1171875f, -0.79296875f, + 1.359375f, 1.8125f, -1.6640625f, 0.2578125f, 1.1796875f, 2.125f, 0.458984375f, + -0.53515625f, -0.74609375f, 1.5625f, 0.1318359375f, 2.875f, -1.265625f, -0.5078125f, + 1.140625f, 0.4140625f, 1.3125f, -0.30859375f, 2.640625f, 0.283203125f, -0.6640625f, + 0.6953125f, -0.67578125f, -0.26171875f, -0.3828125f, -0.9765625f, 0.458984375f, -2.171875f, + -0.31640625f, -1.4140625f, 0.212890625f, 0.68359375f, 1.0f, 0.7890625f, -0.369140625f, + -0.06591796875f, 0.95703125f, 0.36328125f, 0.18359375f, -0.494140625f, -0.93359375f, + 0.4453125f, 0.046630859375f, 0.69140625f, -0.3203125f, -0.578125f, 0.734375f, 2.59375f, + 0.6640625f, 0.205078125f, -0.77734375f, 0.4609375f, -0.34765625f, 0.1572265625f, + -0.404296875f, -0.498046875f, -0.48046875f, -0.1923828125f, 0.36328125f, 1.1171875f, + -1.453125f, -0.984375f, -0.83984375f, -0.6640625f, 0.4375f, 1.046875f, 1.0390625f, + 1.1171875f, 0.85546875f, 0.6640625f, 0.443359375f, 0.6015625f, -1.0078125f, -0.859375f, + -0.70703125f, -0.38671875f, -1.2109375f, 1.53125f, -0.2890625f, 0.0169677734375f, -1.5625f, + 0.10546875f, 1.125f, 0.87890625f, -0.4765625f, 0.6328125f, -1.90625f, -0.08642578125f, + 0.3984375f, -0.111328125f, 0.75f, 1.1640625f, -0.50390625f, 0.1689453125f, -0.18359375f, + 1.0078125f, -0.01953125f, -1.4140625f, 0.341796875f, -1.1640625f, 0.040283203125f, + 1.6484375f, -0.6484375f, 0.099609375f, 0.8515625f, 2.640625f, -1.03125f, 0.72265625f, + 0.047119140625f, -1.546875f, -0.09619140625f, -0.1279296875f, -2.234375f, 0.32421875f, + -0.65625f, -1.1328125f, 1.96875f, -0.392578125f, -1.7109375f, 0.12353515625f, -0.60546875f, + 0.98046875f, -0.2333984375f, -2.015625f, -0.171875f, 1.1875f, -2.015625f, -0.5f, + -0.859375f, -0.80078125f, 1.328125f, 1.0625f, -0.94140625f, 0.92578125f, 0.06494140625f, + -1.0859375f, -0.486328125f, 1.6875f, -1.2421875f, 0.263671875f, 1.40625f, -1.09375f, + -0.287109375f, 0.4375f, 1.0f, -1.4609375f, -1.0f, -0.443359375f, -1.0234375f, 0.279296875f, + -0.498046875f, 0.322265625f, -2.09375f, 0.1162109375f, 1.734375f, 1.375f, -0.1396484375f, + 0.3984375f, -1.5390625f, 0.96875f, -1.6875f, 1.390625f, -0.984375f, -0.17578125f, + -1.171875f, 0.74609375f, 1.1796875f, -0.1865234375f, 0.89453125f, 0.3828125f, -0.97265625f, + -0.3046875f, 0.1435546875f, -0.06884765625f, 0.5859375f, -0.216796875f, 1.828125f, + -0.4921875f, -0.34765625f, 1.09375f, 2.015625f, 0.2255859375f, -0.6171875f, 0.8828125f, + -0.53125f, -1.0703125f, -2.296875f, -0.6796875f, -0.314453125f, -1.15625f, -1.0078125f, + -0.8671875f, -1.5390625f, -0.22265625f, 0.30859375f, 0.0859375f, 1.2734375f, 0.201171875f, + -1.2421875f, 0.06884765625f, -0.77734375f, 0.11767578125f, 0.02294921875f, -0.83203125f, + 0.2470703125f, -2.28125f, -0.1396484375f, -0.328125f, 0.5234375f, -1.4765625f, 0.75f, + 0.26953125f, 1.25f, -2.421875f, -0.2099609375f, -0.8515625f, -0.1513671875f, 1.2578125f, + -0.90234375f, -0.369140625f, 1.5703125f, -0.1728515625f, -0.43359375f, -0.83203125f, + -0.021728515625f, -0.010009765625f, 0.04638671875f, -0.1865234375f, -0.6796875f, + -0.376953125f, 0.6796875f, 0.375f, 0.30859375f, -0.177734375f, -1.234375f, 1.7265625f, + -1.2421875f, 0.17578125f, -0.1484375f, -0.828125f, -2.015625f, -1.0546875f, -0.8125f, + 2.59375f, -0.6484375f, 1.171875f, -1.6875f, 0.48828125f, 0.72265625f, -0.66015625f, + -0.625f, -0.37890625f, 0.138671875f, 0.73046875f, -1.09375f, -0.1201171875f, 0.0f, + -0.294921875f, -0.56640625f, -0.310546875f, 0.0654296875f, 0.73046875f, -0.6640625f, + -1.3671875f, 0.0f, -0.51171875f, 1.8515625f, 0.423828125f, -0.0634765625f, -0.65234375f, + -0.60546875f, -0.212890625f, 0.890625f, -0.072265625f, -0.376953125f, 0.6015625f, + 1.3671875f, 0.3828125f, -0.63671875f, -1.7890625f, 1.0390625f, -1.2890625f, 0.79296875f, + 0.56640625f, -1.296875f, -1.1875f, 1.1015625f, -1.6328125f, 0.5390625f, 0.1123046875f, + 0.00069427490234375f, 0.0f, 1.0625f, 0.98046875f, 0.333984375f, 1.0390625f, 0.6875f, + -1.625f, 1.4296875f, 1.0390625f, 0.6953125f, -0.423828125f, 0.58984375f, 1.203125f, + 0.267578125f, -1.09375f, 0.4765625f, -1.4296875f, -0.326171875f, 0.0004100799560546875f, + -0.85546875f, 1.4609375f, 1.7734375f, 2.046875f, 2.140625f, -1.5703125f, 1.2890625f, + -1.859375f, -1.6953125f, 1.703125f, 0.71875f, 0.265625f, 1.1015625f, -0.6171875f, + 0.2890625f, -0.3828125f, 1.140625f, 0.16796875f, -0.2333984375f, 0.400390625f, + -0.0712890625f, 1.2265625f, 2.140625f, 0.80859375f, -0.435546875f, 0.056884765625f, + 1.28125f, -0.447265625f, -0.345703125f, -1.3828125f, -0.138671875f, 1.0390625f, + 0.75390625f, -1.515625f, -2.3125f, -0.77734375f, 0.8984375f, 0.392578125f, -0.7421875f, + 1.6171875f, -1.703125f, 1.0859375f, 2.34375f, -0.047119140625f, -0.8203125f, 1.578125f, + -0.91796875f, -0.416015625f, 1.6796875f, 0.98046875f, -1.6953125f, 0.671875f, -0.29296875f, + 0.000408172607421875f, 0.890625f, -1.1875f, -0.83203125f, -1.1328125f, 1.2421875f, + -1.234375f, 0.3515625f, 0.84375f, 2.484375f, -0.8046875f, 2.15625f, -1.015625f, + -0.90234375f, 0.55859375f, 1.4140625f, 0.2470703125f, -1.703125f, 0.1318359375f, + 0.91796875f, 0.5234375f, -0.4140625f, 0.80078125f, -2.140625f, 1.9609375f, 1.0f, + 0.2333984375f, 0.75f, -0.81640625f, 2.78125f, -0.439453125f, 1.0f, -1.4296875f, + -1.4453125f, -0.392578125f, -0.79296875f, 1.40625f, 0.0224609375f, 0.09423828125f, + 0.0830078125f, 0.75390625f, 2.046875f, -0.01806640625f, 1.0859375f, -1.9296875f, + -0.59765625f, -0.48046875f, -1.1953125f, -2.25f, -0.287109375f, -0.169921875f, 0.69140625f, + 0.6484375f, 0.462890625f, -0.419921875f, 0.44921875f, 0.072265625f, 0.078125f, 0.765625f, + 1.2890625f, 2.921875f, -2.359375f, 1.5f, 1.0234375f, -1.546875f, 0.294921875f, 0.96875f, + 0.91796875f, 0.76953125f, -0.201171875f, -0.54296875f, 1.0703125f, -0.6328125f, + 0.46484375f, 1.375f, -0.466796875f, -0.25f, -0.8046875f, -1.4140625f, 0.60546875f, + 0.52734375f, 0.7265625f, -0.5234375f, -2.15625f, -0.0244140625f, -0.59375f, -1.15625f, + 0.671875f, -0.248046875f, 1.359375f, -0.90625f, -0.65625f, 1.0625f, 1.546875f, 1.6328125f, + 1.3203125f, 0.0f, -0.73046875f, 2.375f, 1.6875f, 0.1728515625f, 0.57421875f, 0.60546875f, + -0.498046875f, 0.0f, 1.6640625f, 1.265625f, -0.85546875f, -0.4296875f, -0.53515625f, + -0.09619140625f, -0.80078125f, -0.1103515625f, 0.6875f, -0.416015625f, 1.5703125f, + -0.400390625f, 0.609375f, -0.94921875f, 0.435546875f, 0.058349609375f, -0.1767578125f, + 0.439453125f, -0.51171875f, 0.2119140625f, 0.55078125f, 0.0306396484375f, -1.4765625f, + 0.76953125f, -1.125f, -1.734375f, 1.90625f, -0.2236328125f, 0.1259765625f, -0.03173828125f, + -0.546875f, -0.92578125f, -0.3046875f, 2.21875f, -1.3828125f, -0.8984375f, 0.1259765625f, + -0.265625f, 1.2421875f, 0.57421875f, 2.046875f, -0.431640625f, -0.98046875f, + -0.2158203125f, -0.0625f, -1.359375f, -0.28515625f, -0.8359375f, 0.0f, -0.040283203125f, + 0.44140625f, 0.65234375f, -0.341796875f, -1.6796875f, -0.1787109375f, -1.4609375f, + 0.000919342041015625f, 0.8046875f, -0.05859375f, 0.0008697509765625f, 1.75f, 0.69140625f, + 0.4296875f, 0.047119140625f, 1.8984375f, -0.75390625f, -0.388671875f, 1.796875f, + -1.109375f, -1.3515625f, -0.625f, 0.061279296875f, 0.494140625f, 0.6328125f, 1.9765625f, + 2.171875f, 0.734375f, 0.11669921875f, 0.205078125f, 0.275390625f, 0.060791015625f, + -0.041015625f, 0.09716796875f, -0.27734375f, 0.2421875f, -1.6875f, -0.37109375f, -2.59375f, + -0.2392578125f, -0.84375f, -0.1201171875f, -0.58984375f, -0.515625f, 2.09375f, + -0.65234375f, -0.44921875f, -0.296875f, -0.91796875f, -0.482421875f, -0.58203125f, + -1.828125f, 0.63671875f, 0.294921875f, -2.0f, 0.5625f, 0.06787109375f, 0.1513671875f, + -1.1875f, -1.078125f, -1.4453125f, 2.234375f, -0.5078125f, 0.9921875f, 1.4375f, -1.5f, + -0.62109375f, 0.609375f, -0.46484375f, -0.28515625f, -1.625f, 0.18359375f, 0.2353515625f, + -0.6953125f, -0.31640625f, 1.046875f, 0.39453125f, -0.5703125f, -0.984375f, 0.82421875f, + -0.220703125f, -0.248046875f, -1.46875f, 1.3359375f, -0.39453125f, -0.2236328125f, + 0.0673828125f, -1.125f, 0.0189208984375f, 0.01116943359375f, -0.81640625f, 1.3359375f, + 0.337890625f, 0.0478515625f, 2.8125f, 1.2265625f, 0.79296875f, -0.296875f, 2.46875f, + 0.9375f, -0.380859375f, -1.2734375f, 0.87890625f, 0.1669921875f, 1.4453125f, -0.83203125f, + 0.470703125f, -0.0634765625f, 0.5859375f, 0.63671875f, -0.59765625f, -0.26171875f, + 0.515625f, 0.8046875f, 2.6875f, -2.546875f, -0.34765625f, -0.87109375f, 1.140625f, + -0.75390625f, -0.045654296875f, 0.314453125f, -3.265625f, -0.72265625f, -2.015625f, + 0.67578125f, 1.140625f, 0.271484375f, 0.263671875f, -0.1728515625f, 0.67578125f, + -0.3671875f, -1.2265625f, -0.318359375f, -0.49609375f, -0.796875f, -1.3984375f, 1.1484375f, + 0.462890625f, -0.087890625f, 0.498046875f, -1.2421875f, -0.345703125f, -1.125f, 1.078125f, + 0.0f, 0.09228515625f, 0.58984375f, -0.59765625f, -1.1171875f, -0.921875f, -0.076171875f, + -0.322265625f, 0.5703125f, 1.203125f, -0.296875f, 0.083984375f, -1.3359375f, -0.296875f, + -0.41796875f, -0.83984375f, 0.042236328125f, -1.125f, 0.0002880096435546875f, 1.1796875f, + -0.546875f, -1.1171875f, 1.5703125f, 1.5f, -2.53125f, -1.375f, 0.359375f, -0.053955078125f, + -0.59375f, -1.375f, 0.5625f, 0.3359375f, 2.15625f, 0.06591796875f, 1.0859375f, -0.9609375f, + -0.055419921875f, 0.2275390625f, -1.046875f, -1.171875f, -0.26171875f, -0.451171875f, + 0.263671875f, 0.4453125f, 1.8359375f, 1.1484375f, -0.9375f, -0.376953125f, -1.4375f, + 0.1005859375f, -0.474609375f, -0.6484375f, -0.53515625f, 0.1591796875f, 0.9765625f, 0.75f, + 0.71484375f, -0.98046875f, -0.7734375f, 0.3125f, 0.81640625f, -0.43359375f, 0.02392578125f, + 0.64453125f, -0.255859375f, -0.734375f, 0.4375f, -1.3671875f, -0.921875f, -1.0625f, + 1.0625f, 0.796875f, 0.10009765625f, 0.267578125f, -0.66015625f, -0.1806640625f, 0.6875f, + -0.9296875f, -0.06884765625f, -0.83984375f, 0.9765625f, -0.388671875f, -0.458984375f, + -0.9921875f, -1.875f, -1.109375f, -0.76953125f, 0.0f, 0.185546875f, 0.8515625f, -3.28125f, + 0.94140625f, -0.1298828125f, -0.494140625f, 1.546875f, 0.0f, 0.48046875f, 0.404296875f, + -0.56640625f, 0.80859375f, -0.283203125f, 0.50390625f, 0.5234375f, 0.8359375f, 1.953125f, + 1.015625f, 0.27734375f, 1.765625f, 0.412109375f, -1.5234375f, -0.287109375f, -0.8359375f, + 0.240234375f, 1.1875f, 0.90625f, 0.94140625f, -0.64453125f, -1.390625f, -0.44921875f, + -0.287109375f, 1.7265625f, 1.140625f, 1.375f, 0.77734375f, 1.03125f, 0.2890625f, + 0.490234375f, 0.8671875f, 0.0f, -2.21875f, 0.41796875f, 1.1640625f, -0.4453125f, + 0.259765625f, 0.1533203125f, 0.5390625f, -1.015625f, -0.76171875f, -1.03125f, + -0.427734375f, 1.9921875f, 1.484375f, -0.5859375f, 0.06640625f, -1.7578125f, -0.298828125f, + -1.90625f, 1.9296875f, -1.3515625f, -0.66015625f, 0.07958984375f, -1.09375f, + -0.1318359375f, 1.171875f, 0.29296875f, -1.0625f, 0.34375f, -0.69921875f, -0.484375f, + -0.27734375f, -1.109375f, 1.671875f, -1.4921875f, 0.345703125f, -0.470703125f, 0.0f, + -0.2734375f, -0.09912109375f, 0.63671875f, 2.8125f, -1.6328125f, 0.55078125f, -0.82421875f, + 0.0f, -0.96875f, -1.125f, -0.99609375f, 0.0f, 0.52734375f, -0.62109375f, 1.65625f, + 1.109375f, -0.341796875f, -0.1533203125f, 0.91796875f, 0.37109375f, -1.828125f, -0.859375f, + 1.046875f, 1.15625f, -0.06982421875f, -0.004638671875f, -0.396484375f, -0.92578125f, + -1.3671875f, -0.90625f, 0.8203125f, -0.19140625f, 2.21875f, -1.0703125f, 1.3671875f, + -0.5390625f, 0.5234375f, -0.91796875f, 0.6953125f, 1.8828125f, -0.1435546875f, -0.1328125f, + 0.2021484375f, 0.73828125f, 1.2734375f, -1.84375f, -1.59375f, -1.03125f, -0.08447265625f, + 1.1796875f, -0.57421875f, -0.96875f, 0.158203125f, -1.015625f, -0.158203125f, 1.1328125f, + -0.267578125f, 0.14453125f, -0.2353515625f, 0.3125f, -0.96484375f, 1.7109375f, + 0.12158203125f, -0.73828125f, -0.15625f, -1.3828125f, -0.7421875f, -0.033935546875f, + 0.546875f, 1.1796875f, -2.484375f, 0.1533203125f, -0.404296875f, -2.78125f, -1.6171875f, + 1.4765625f, 0.49609375f, -0.07666015625f, -1.015625f, 0.2333984375f, -0.296875f, + 0.251953125f, -0.140625f, 0.1396484375f, -0.36328125f, 1.421875f, -0.1533203125f, + 0.412109375f, 1.5f, 0.341796875f, -2.5f, 0.5625f, -0.328125f, 0.5078125f, 1.2265625f, + 0.83203125f, 1.953125f, -1.703125f, 0.86328125f, 0.427734375f, -1.46875f, 1.1953125f, + 0.255859375f, -0.2138671875f, -0.06494140625f, 0.296875f, 0.171875f, -0.984375f, + 0.83203125f, 1.625f, 0.58203125f, -0.490234375f, 1.140625f, -0.2353515625f, -0.53125f, + 0.00115966796875f, -0.3515625f, -0.380859375f, -1.15625f, -1.96875f, -2.21875f, + 0.30078125f, -0.06591796875f, 2.390625f, 0.55078125f, -0.90234375f, 0.2392578125f, + -1.890625f, -0.88671875f, -0.39453125f, -0.44921875f, -1.28125f, 0.2138671875f, + -0.4921875f, 1.0390625f, 1.0703125f, -0.7265625f, 1.2890625f, 0.25390625f, -1.265625f, + 1.078125f, 0.1591796875f, -0.70703125f, -2.296875f, -0.6953125f, 0.94140625f, 0.0f, + 0.046875f, -0.84765625f, 1.1875f, -0.73828125f, -0.73828125f, -1.8046875f, -2.5625f, 0.0f, + 0.98828125f, -1.3671875f, -0.515625f, -0.2373046875f, -0.55078125f, -0.11865234375f, + 0.83984375f, 1.28125f, 1.2734375f, 1.3671875f, -1.328125f, -0.69140625f, 1.3359375f, + -1.3515625f, -0.4609375f, 0.1904296875f, 0.2216796875f, 2.546875f, -2.671875f, -0.6015625f, + -0.3515625f, -0.44921875f, 1.359375f, 2.78125f, 0.59765625f, 0.302734375f, 0.1669921875f, + 0.359375f, -0.072265625f, 0.3828125f, -0.640625f, 0.412109375f, 0.2470703125f, 0.98828125f, + 0.37890625f, -0.12890625f, -2.5f, 1.6328125f, -0.2255859375f, 1.4765625f, -0.86328125f, + -0.123046875f, -0.56640625f, -0.5625f, 1.46875f, -0.8984375f, -0.80078125f, 0.33203125f, + -0.1494140625f, -0.435546875f, 0.65625f, 0.478515625f, 1.1640625f, -0.314453125f, + 0.73046875f, -0.279296875f, -0.158203125f, -1.25f, -1.109375f, 0.51953125f, -0.9609375f, + 0.765625f, -0.46484375f, -0.5625f, 0.71484375f, 2.328125f, -0.91015625f, -1.75f, + 0.6484375f, 1.296875f, 0.1865234375f, 0.18359375f, 1.03125f, 0.77734375f, 0.66796875f, + -0.41796875f, 0.3671875f, 0.0f, 0.423828125f, 0.82421875f, -0.56640625f, -1.625f, + -0.97265625f, -2.296875f, 0.1220703125f, 0.80078125f, -0.1787109375f, 0.056396484375f, + 1.09375f, -0.7109375f, 1.5f, 0.328125f, 0.02734375f, -0.84765625f, 0.125f, -0.81640625f, + -1.59375f, -1.3203125f, -1.5859375f, -0.193359375f, 1.046875f, 0.71484375f, 0.361328125f, + 0.318359375f, 0.04248046875f, 1.609375f, 0.337890625f, 0.3046875f, -1.890625f, + 0.404296875f, 1.0078125f, 0.625f, 0.89453125f, -0.88671875f, -1.7265625f, -0.1669921875f, + -0.41015625f, 0.55078125f, -0.21484375f, -0.89453125f, -0.734375f, 0.6171875f, + -0.73046875f, 0.421875f, 0.43359375f, 1.25f, -1.8046875f, 0.6015625f, 1.9140625f, + -0.6953125f, 0.796875f, 2.125f, 0.62109375f, 0.208984375f, 1.1015625f, -0.265625f, + 0.05859375f, -0.050537109375f, 1.5078125f, -2.734375f, -1.3046875f, -0.62890625f, + -0.55859375f, -0.400390625f, -0.06640625f, -0.14453125f, 0.8984375f, -1.21875f, + -0.39453125f, 0.8125f, -0.7421875f, -0.09912109375f, 1.328125f, -1.4609375f, 0.96875f, + 2.84375f, 0.1279296875f, 2.265625f, 1.15625f, -1.125f, -0.90234375f, -0.65234375f, + -1.09375f, 0.494140625f, -0.181640625f, -0.58984375f, 1.3046875f, 0.7265625f, 0.14453125f, + -0.50390625f, 0.55078125f, -0.578125f, -0.83984375f, -2.359375f, -0.08447265625f, + 1.2109375f, -1.4375f, 0.5859375f, 1.171875f, 0.5703125f, -1.90625f, 1.5390625f, 0.7265625f, + 1.0078125f, 0.5390625f, 0.58984375f, -2.140625f, -0.57421875f, -1.5625f, 0.345703125f, + 0.41015625f, 0.1240234375f, -0.73828125f, -1.46875f, -0.99609375f, -1.15625f, -1.421875f, + 1.4140625f, -0.5078125f, -1.4609375f, 0.78125f, 0.12353515625f, -0.361328125f, 0.65234375f, + -0.98828125f, 1.1015625f, -1.203125f, -1.5859375f, -0.4296875f, -0.0196533203125f, + -1.671875f, -0.61328125f, -0.6796875f, 0.0030059814453125f, 1.546875f, -0.2392578125f, + -0.55859375f, -2.625f, -0.625f, -1.1328125f, -0.609375f, -0.47265625f, 0.91796875f, + -1.5859375f, -1.21875f, 0.208984375f, 0.9765625f, -1.1171875f, 1.2421875f, 0.37890625f, + -0.39453125f, -0.88671875f, 0.048095703125f, 2.578125f, 0.2412109375f, -2.3125f, + -0.359375f, 0.484375f, -1.3984375f, 2.15625f, -0.29296875f, -0.1669921875f, 2.34375f, + -0.32421875f, 0.478515625f, 2.5f, 1.359375f, 0.6484375f, -0.439453125f, -1.1015625f, + -0.3671875f, -0.49609375f, 0.80078125f, 0.435546875f, 0.201171875f, -0.60546875f, + -0.0546875f, -1.453125f, -0.482421875f, -0.65234375f, 0.06201171875f, -1.7578125f, + -0.177734375f, -0.3125f, 0.8828125f, -0.8671875f, 1.265625f, -0.375f, -1.6484375f, + 1.3515625f, -0.97265625f, -0.9921875f, 0.58984375f, 0.0279541015625f, -1.734375f, + 0.25390625f, 0.75390625f, 0.9296875f, -0.63671875f, 0.859375f, -1.6953125f, -1.890625f, + -0.90234375f, 0.3671875f, 1.7109375f, -0.0308837890625f, -0.333984375f, 1.609375f, + 0.66796875f, 0.1796875f, -0.78515625f, -1.3984375f, -1.3046875f, 1.1484375f, + -0.10791015625f, -0.8515625f, -0.29296875f, 1.7578125f, 0.45703125f, -0.734375f, + 0.66015625f, 1.09375f, -0.341796875f, -0.81640625f, -1.171875f, -0.7578125f, 1.3828125f, + 0.06982421875f, -0.21484375f, -0.50390625f, 0.54296875f, 0.70703125f, 1.6171875f, + -0.038818359375f, -0.0289306640625f, 0.6796875f, 0.875f, -0.93359375f, 0.026611328125f, + 0.498046875f, 0.859375f, -1.5546875f, -0.515625f, 0.87109375f, -1.796875f, + 0.01263427734375f, 0.7890625f, -0.392578125f, -0.193359375f, 0.07470703125f, -0.45703125f, + 0.0673828125f, 1.015625f, -0.3359375f, 0.326171875f, 0.65625f, 0.134765625f, + -0.04736328125f, -0.49609375f, 2.8125f, 0.236328125f, 0.07861328125f, -0.89453125f, + 0.84765625f, -0.283203125f, -1.1640625f, 0.64453125f, -1.1484375f, 0.5078125f, + -0.1318359375f, -0.027099609375f, 0.326171875f, -0.61328125f, -1.234375f, 0.7109375f, + -1.2109375f +}; +inline constexpr float kSubBudgetAttnK[] = { + 2.296875f, 1.6328125f, -1.1875f, 0.1943359375f, -0.287109375f, 0.01397705078125f, + 0.4609375f, -1.4375f, -0.9921875f, -1.265625f, -0.5546875f, 0.5390625f, -0.234375f, + -0.37109375f, -0.02978515625f, -0.73046875f, 0.8046875f, -0.030029296875f, 1.3046875f, + 0.94921875f, -0.8828125f, 0.74609375f, -0.953125f, 0.193359375f, -0.1279296875f, + 0.294921875f, 0.29296875f, -0.88671875f, -0.875f, -0.267578125f, 1.34375f, -0.65234375f, + -0.87890625f, -0.7734375f, 0.43359375f, 1.59375f, -0.404296875f, -0.265625f, 0.51953125f, + 1.0078125f, -1.5234375f, 0.81640625f, 0.50390625f, 1.9375f, 0.019287109375f, 1.53125f, + 0.05126953125f, -0.94140625f, -1.3984375f, -0.047119140625f, -0.50390625f, 0.482421875f, + -0.74609375f, -0.65625f, -1.203125f, -1.1640625f, 0.244140625f, 1.015625f, 0.60546875f, + -0.349609375f, 2.4375f, 0.59765625f, -1.1953125f, -0.296875f, 1.875f, 1.1796875f, + 0.80859375f, 0.259765625f, 1.875f, -1.140625f, 0.4296875f, 0.65625f, 0.6171875f, 1.765625f, + 0.7734375f, -0.294921875f, 1.0625f, 0.451171875f, 0.28515625f, -0.16796875f, 0.63671875f, + -0.29296875f, -1.28125f, -0.03955078125f, -1.359375f, 0.1787109375f, 0.578125f, + -0.322265625f, 0.63671875f, -0.83984375f, 0.041259765625f, 0.8515625f, 1.578125f, + 1.0234375f, 0.5234375f, -0.8359375f, -0.04248046875f, 0.56640625f, 0.50390625f, + -0.14453125f, 1.8046875f, 0.177734375f, 1.9296875f, -0.43359375f, -1.703125f, -0.640625f, + 0.11279296875f, -0.79296875f, 0.96484375f, 1.734375f, 0.1904296875f, 0.24609375f, + -0.9453125f, 1.6171875f, -2.265625f, 1.5f, 0.06640625f, -2.078125f, -0.79296875f, + -0.96875f, -1.0f, -1.1015625f, -1.1875f, -0.3359375f, -0.2890625f, -1.703125f, + 0.466796875f, 1.625f, 0.431640625f, 0.96875f, -0.78125f, -0.90625f, -0.765625f, + -0.6015625f, 0.2578125f, 0.310546875f, -0.98828125f, -0.2490234375f, -1.0234375f, + 1.8203125f, 0.314453125f, -0.26953125f, -0.1748046875f, -0.2412109375f, 0.455078125f, + -1.2421875f, -2.359375f, -0.30859375f, -2.875f, 0.71484375f, -0.375f, -0.91015625f, + 1.1796875f, -0.86328125f, -0.63671875f, 0.6484375f, 0.228515625f, -2.71875f, + -0.1748046875f, -1.828125f, -2.3125f, 0.96484375f, -1.53125f, -0.19140625f, -1.640625f, + -1.53125f, 0.90625f, -1.7890625f, 0.29296875f, 1.1171875f, -0.263671875f, 0.625f, + 0.58984375f, 0.267578125f, 1.2265625f, 0.80859375f, 0.703125f, -0.435546875f, 0.98046875f, + -1.2734375f, 1.0859375f, 1.2421875f, -0.478515625f, -0.408203125f, 1.0546875f, + -0.2275390625f, 0.5234375f, 0.12158203125f, 0.578125f, -1.40625f, 1.0859375f, -0.75390625f, + 0.345703125f, 0.83984375f, 0.427734375f, -1.1796875f, 0.7265625f, 1.9375f, -1.296875f, + 0.2265625f, -0.13671875f, 0.421875f, -0.271484375f, 1.0703125f, -0.458984375f, + 0.09521484375f, -0.392578125f, -0.06591796875f, -0.55078125f, -1.765625f, -0.484375f, + -1.3046875f, 1.6484375f, -2.09375f, -1.1875f, 0.2314453125f, -1.953125f, 0.451171875f, + 0.97265625f, 0.466796875f, -1.6484375f, 0.75f, 1.1484375f, 1.0390625f, 1.3984375f, + -2.109375f, 0.0625f, 1.671875f, 0.248046875f, -1.0234375f, -0.23046875f, 0.039306640625f, + 0.2431640625f, 0.201171875f, -1.6640625f, -0.4296875f, 1.6484375f, -1.7734375f, + -0.302734375f, -0.390625f, -0.68359375f, -2.25f, -0.1611328125f, -0.05224609375f, + 1.1015625f, -0.99609375f, 2.09375f, 1.171875f, -0.5078125f, -0.57421875f, -0.51171875f, + 0.0712890625f, 0.053955078125f, 1.3671875f, -1.0703125f, 0.484375f, -0.1826171875f, + -0.84375f, -1.09375f, -0.27734375f, -0.51171875f, -0.033203125f, 0.64453125f, -1.15625f, + 1.0546875f, -0.890625f, -0.05029296875f, 1.8515625f, -0.07177734375f, -0.37890625f, + -1.3515625f, 1.1171875f, 0.5703125f, -0.06396484375f, -0.0888671875f, -0.96484375f, + 0.3203125f, 0.8203125f, -1.9453125f, -0.39453125f, 1.484375f, -0.06103515625f, + -0.154296875f, 0.091796875f, 0.77734375f, 1.3671875f, -1.859375f, -0.859375f, 0.80078125f, + 0.1513671875f, -0.85546875f, -1.515625f, 1.21875f, -0.267578125f, -0.462890625f, + 0.041748046875f, -0.2060546875f, 0.76171875f, 1.4375f, 1.1640625f, 0.2109375f, + 0.419921875f, 0.77734375f, -1.109375f, 1.265625f, 0.267578125f, 0.6796875f, 0.75390625f, + -0.7890625f, 0.90234375f, -0.64453125f, 0.29296875f, 0.0f, -0.181640625f, 0.169921875f, + 0.427734375f, -0.6875f, -0.953125f, -0.796875f, -0.458984375f, 0.478515625f, 0.5625f, + 0.7421875f, -0.162109375f, -1.890625f, 0.1962890625f, -0.79296875f, -0.04345703125f, + 1.5625f, -1.6953125f, -1.34375f, -0.462890625f, 0.09033203125f, -0.44921875f, + 0.1767578125f, 0.9375f, 0.10205078125f, -2.109375f, -1.6171875f, -0.67578125f, 0.6640625f, + 0.07568359375f, 1.0078125f, -0.84765625f, -0.115234375f, -0.75f, 1.09375f, 0.73828125f, + -0.48046875f, -1.09375f, -0.640625f, -0.8984375f, -0.0791015625f, 0.201171875f, + 0.2216796875f, 1.3984375f, 2.046875f, -0.52734375f, -1.3359375f, -0.88671875f, 0.6328125f, + -1.2265625f, 0.50390625f, -0.455078125f, 0.5625f, 1.0625f, 0.6015625f, -1.5390625f, + 0.3515625f, 0.2412109375f, 0.84765625f, -0.6796875f, -0.2333984375f, 1.78125f, + -0.181640625f, -0.1513671875f, -0.55078125f, -2.390625f, 0.400390625f, 0.06494140625f, + 0.0296630859375f, 1.703125f, -0.08154296875f, -1.7265625f, -1.4296875f, 0.35546875f, + -0.51171875f, -0.1796875f, 1.4140625f, -0.5625f, -0.72265625f, 1.2890625f, -0.43359375f, + 0.431640625f, -0.310546875f, 1.765625f, 1.109375f, -0.609375f, 0.220703125f, -0.375f, + 0.154296875f, 2.453125f, 0.23046875f, -0.58984375f, -0.16015625f, 0.84765625f, 0.45703125f, + -0.1376953125f, -0.46484375f, -0.236328125f, 1.5390625f, 0.44140625f, 0.322265625f, + 0.59765625f, -0.07275390625f, 1.1015625f, -0.1337890625f, -0.294921875f, 0.51171875f, + -1.2265625f, -1.09375f, -0.43359375f, 0.326171875f, 1.625f, -0.232421875f, -0.84375f, + -0.6171875f, -0.15234375f, -1.4375f, 1.953125f, 0.255859375f, 0.671875f, -0.421875f, + 0.71875f, -0.2333984375f, 1.0390625f, -0.88671875f, -0.09619140625f, -1.1015625f, + 0.55078125f, -1.953125f, 0.318359375f, -0.375f, -1.0390625f, -0.59765625f, -1.09375f, + 0.1923828125f, -0.86328125f, -0.4453125f, -0.029541015625f, -0.185546875f, -0.8984375f, + 0.82421875f, 0.7265625f, -1.7890625f, -0.8203125f, 0.16015625f, 1.2890625f, -0.404296875f, + 1.6796875f, -0.55859375f, 0.59375f, 1.203125f, -0.7109375f, -0.4140625f, 0.109375f, + -0.049560546875f, -1.1484375f, -0.051025390625f, 0.1923828125f, -1.3046875f, 1.1796875f, + -1.3125f, 0.73046875f, 0.33203125f, -0.58203125f, 0.2294921875f, 0.41015625f, 1.078125f, + -0.6015625f, -0.6015625f, -0.88671875f, -0.423828125f, -1.8671875f, -0.5703125f, + -0.80078125f, 2.265625f, -2.46875f, -1.046875f, 0.96875f, -1.34375f, 0.478515625f, + 0.80859375f, 1.390625f, -0.2890625f, -1.5f, -1.703125f, 1.3515625f, 0.7578125f, 0.25f, + 1.1171875f, -0.357421875f, -1.3203125f, -0.26171875f, 0.90234375f, -1.5234375f, 0.7578125f, + 0.6484375f, -0.52734375f, 0.345703125f, 0.337890625f, 1.0625f, 0.62890625f, 0.07421875f, + -0.41015625f, -0.022705078125f, 0.2333984375f, 1.234375f, 1.1953125f, 0.578125f, + -0.10009765625f, 1.0390625f, 0.43359375f, 0.490234375f, -0.53515625f, -1.59375f, + -0.388671875f, -1.4453125f, -0.921875f, -1.640625f, 0.31640625f, -0.419921875f, -1.09375f, + 0.15234375f, 0.28125f, -1.21875f, 0.298828125f, -0.4453125f, -1.9296875f, -0.05908203125f, + -0.57421875f, -0.0146484375f, -0.76953125f, -0.765625f, -0.99609375f, 0.22265625f, + -1.1015625f, -1.03125f, -0.63671875f, 0.484375f, -1.1328125f, 0.9453125f, -0.50390625f, + -0.1728515625f, -0.84765625f, -0.423828125f, -0.57421875f, 0.85546875f, 2.375f, + 0.53515625f, 0.032958984375f, -0.345703125f, 0.05322265625f, 0.06787109375f, 1.3828125f, + -0.119140625f, -0.2578125f, 0.28515625f, -0.08203125f, 0.032958984375f, 0.35546875f, + 1.4375f, -0.81640625f, 0.03857421875f, 0.1923828125f, -0.96484375f, 0.2578125f, -1.296875f, + 0.035888671875f, 0.08203125f, 0.81640625f, 2.359375f, 0.392578125f, -0.1357421875f, + 1.0390625f, 0.60546875f, 0.50390625f, 0.033935546875f, 0.81640625f, 0.6484375f, + 0.55859375f, -0.74609375f, -1.375f, -1.1796875f, 0.478515625f, -0.0439453125f, -1.1796875f, + -1.453125f, 0.5078125f, 0.126953125f, -1.0546875f, -1.28125f, -0.86328125f, 0.431640625f, + 0.96875f, -0.1357421875f, 1.859375f, 0.7265625f, 0.3203125f, 0.478515625f, -2.015625f, + 1.2265625f, -0.408203125f, 0.1669921875f, 0.498046875f, -0.1591796875f, 0.039306640625f, + 1.0859375f, 0.828125f, -1.7890625f, -1.640625f, -0.56640625f, 1.8203125f, -0.294921875f, + 1.9765625f, -0.51953125f, -0.578125f, -0.6015625f, 0.8203125f, -0.875f, 0.287109375f, + 0.37890625f, -0.251953125f, -2.40625f, -0.50390625f, -1.3046875f, 1.1640625f, 0.79296875f, + -1.7421875f, 0.8125f, -0.2177734375f, 1.7109375f, 0.58984375f, -1.2578125f, -0.52734375f, + -0.5078125f, -0.30859375f, 0.263671875f, -0.08935546875f, 1.265625f, -0.80859375f, + -0.5625f, -0.9765625f, -2.34375f, -0.88671875f, -0.60546875f, -2.890625f, -0.37109375f, + -0.2265625f, 0.1513671875f, -0.5546875f, 1.4375f, -1.3828125f, -0.1103515625f, 0.7421875f, + 1.515625f, 1.5234375f, -1.1484375f, -0.78125f, 0.59375f, -0.56640625f, -0.60546875f, + -0.478515625f, -0.6953125f, 0.10546875f, -0.75390625f, -0.224609375f, 0.921875f, 0.0f, + -1.3203125f, 1.1640625f, -0.421875f, 0.53125f, 0.06005859375f, -1.234375f, 1.1171875f, + 0.0f, -0.984375f, -1.6328125f, 1.8671875f, 0.53515625f, -2.734375f, 1.1953125f, + -0.70703125f, -1.3984375f, -0.056884765625f, -2.015625f, 0.7734375f, -2.328125f, + 0.087890625f, 1.3828125f, 0.82421875f, -0.7890625f, -0.376953125f +}; +inline constexpr float kSubBudgetAttnV[] = { + -0.026611328125f, 1.2109375f, -0.083984375f, -1.296875f, -1.421875f, -0.5703125f, + 2.328125f, 0.283203125f, -1.0703125f, -1.3671875f, -0.95703125f, 0.66015625f, 1.09375f, + 0.80859375f, 0.1142578125f, -0.59375f, 0.24609375f, 0.89453125f, 2.296875f, -1.2265625f, + -0.40625f, 0.0625f, -0.81640625f, -0.333984375f, -0.134765625f, -0.89453125f, + -0.1494140625f, 1.1875f, 1.0546875f, -1.6640625f, 0.6953125f, -1.3359375f, -1.3671875f, + -0.53515625f, -1.5703125f, 3.109375f, -0.34375f, 0.9140625f, 0.05859375f, -0.5078125f, + 0.5390625f, 0.62109375f, 0.48046875f, 0.15234375f, 0.2314453125f, 0.40625f, 0.82421875f, + -0.81640625f, 1.1953125f, -1.5546875f, -1.1953125f, -1.4765625f, 0.2314453125f, + 0.66015625f, 2.09375f, 0.30859375f, 0.6015625f, 0.91796875f, -0.80859375f, -0.6875f, + -0.10009765625f, -0.2373046875f, 0.20703125f, 0.859375f, -0.6796875f, -0.453125f, + -1.1484375f, 0.546875f, 0.2734375f, -0.97265625f, -0.5625f, 0.82421875f, 0.578125f, + 0.337890625f, 0.3125f, -1.3671875f, 2.671875f, -2.125f, -0.43359375f, 0.9609375f, + 0.578125f, 0.80078125f, 1.0859375f, -0.61328125f, 0.5078125f, -0.7578125f, 0.6015625f, + -0.2119140625f, 1.75f, 1.203125f, 1.8125f, 0.3125f, 1.234375f, 2.28125f, 0.322265625f, + -0.462890625f, -0.08935546875f, -0.046142578125f, 0.328125f, -0.1826171875f, 0.265625f, + -1.109375f, -0.259765625f, -1.171875f, -0.58984375f, 2.015625f, -1.109375f, -1.0f, -1.625f, + -0.5625f, 1.1640625f, -1.234375f, -1.0078125f, 1.234375f, -0.59765625f, -0.037353515625f, + -0.90625f, -0.1611328125f, 1.4609375f, -1.765625f, 0.17578125f, 0.6953125f, + 0.019287109375f, 1.6328125f, 0.74609375f, -1.0625f, -1.3671875f, -0.92578125f, -0.0703125f, + 0.173828125f, -0.34765625f, 0.55859375f, 0.60546875f, 0.55859375f, 0.1484375f, -0.6796875f, + -0.80078125f, 0.181640625f, -1.3984375f, 1.2734375f, 0.447265625f, -0.4609375f, + 0.66796875f, 0.671875f, -0.9921875f, -0.06396484375f, 1.40625f, -0.83984375f, -1.3359375f, + 0.3203125f, 0.26953125f, -0.287109375f, -1.046875f, 0.166015625f, 0.8359375f, -1.0390625f, + 1.5546875f, 0.53515625f, -0.671875f, 2.296875f, 0.1708984375f, 0.55078125f, 1.8984375f, + 0.9765625f, 0.30859375f, 0.435546875f, -0.7578125f, 1.328125f, 0.98046875f, 0.40625f, + -1.7734375f, -2.234375f, 1.1328125f, 0.6875f, 1.84375f, -1.2421875f, -0.17578125f, + 2.28125f, 0.60546875f, 1.359375f, 0.162109375f, 0.68359375f, 1.234375f, 0.1015625f, + -1.1640625f, -1.6484375f, -1.1796875f, -0.74609375f, 0.072265625f, -0.640625f, + 0.443359375f, -0.76953125f, 1.1171875f, -0.484375f, 0.314453125f, -0.400390625f, + -0.359375f, 0.435546875f, 0.373046875f, -0.66796875f, -1.046875f, -0.2451171875f, + -0.85546875f, -0.494140625f, 2.40625f, -2.65625f, 0.90625f, -0.87890625f, -0.34375f, + 0.171875f, 2.53125f, 1.859375f, -1.3359375f, -1.328125f, 1.671875f, -0.94921875f, -1.375f, + -0.1611328125f, -1.390625f, -0.1201171875f, 0.99609375f, -1.3984375f, 0.59765625f, + -0.388671875f, -0.087890625f, -1.9453125f, -1.7734375f, -0.333984375f, 0.1826171875f, + 0.56640625f, 0.5078125f, -0.765625f, 0.00836181640625f, 2.265625f, 0.310546875f, + 0.8671875f, 1.453125f, -1.9140625f, -0.7421875f, 2.3125f, -1.1796875f, 1.625f, 1.203125f, + 0.224609375f, -2.609375f, -1.609375f, -0.234375f, -0.68359375f, -0.03564453125f, + 0.32421875f, -0.875f, -1.3671875f, -0.66015625f, 0.72265625f, -0.091796875f, -0.38671875f, + 0.09326171875f, 0.05029296875f, -0.2177734375f, -0.5390625f, 0.703125f, 0.05224609375f, + -0.8046875f, -0.302734375f, 0.7421875f, -1.3359375f, -1.0078125f, 0.53125f, 0.8515625f, + -0.75390625f, 0.115234375f, 0.5078125f, 0.46484375f, 0.32421875f, -2.09375f, -2.03125f, + -1.046875f, -1.140625f, 0.498046875f, 0.1435546875f, -1.5703125f, 0.5390625f, -1.875f, + -0.44921875f, -2.28125f, -0.1962890625f, -0.68359375f, 0.05517578125f, 0.0123291015625f, + 0.2197265625f, 0.255859375f, 1.3671875f, -0.2265625f, -2.875f, 0.046630859375f, + -0.43359375f, -1.9375f, -0.7421875f, -0.5859375f, -1.125f, -0.71875f, 1.1875f, 0.65625f, + -0.75390625f, 0.1650390625f, -0.12158203125f, 0.30078125f, -1.0234375f, 2.0625f, + -1.1328125f, 0.546875f, 0.41015625f, -1.0f, -0.6640625f, -0.75f, 1.6015625f, 0.73828125f, + -0.232421875f, 0.86328125f, 2.359375f, 1.6328125f, 0.578125f, 1.921875f, -0.478515625f, + 0.95703125f, 0.44921875f, 0.84375f, 0.19921875f, 1.890625f, -0.61328125f, 1.28125f, + 0.50390625f, 0.34375f, -0.9453125f, -1.4140625f, -1.2109375f, 0.80859375f, -0.671875f, + -0.8203125f, -0.482421875f, 1.3203125f, 1.640625f, 1.3671875f, -0.5859375f, 0.265625f, + -1.1640625f, 0.224609375f, 1.578125f, -1.0859375f, -0.90234375f, -0.75f, 1.328125f, + -1.09375f, -0.62890625f, 1.3203125f, 1.4453125f, 0.5f, 0.51953125f, 0.7578125f, 1.9765625f, + -1.546875f, -0.28125f, -0.1689453125f, -0.98046875f, -1.9140625f, 1.4453125f, 0.208984375f, + 1.890625f, 0.94921875f, 0.2373046875f, -0.1650390625f, 0.7109375f, -0.1083984375f, + -1.53125f, -0.72265625f, 1.0859375f, -0.64453125f, 2.3125f, -1.390625f, 0.64453125f, + 1.0625f, -1.1796875f, 0.1611328125f, 1.625f, -0.08447265625f, 0.462890625f, 0.0341796875f, + -1.375f, -1.0859375f, 0.10986328125f, -0.384765625f, 0.052001953125f, -0.55859375f, + -1.0625f, -0.173828125f, 0.197265625f, 0.447265625f, 1.0703125f, 1.734375f, 0.26171875f, + 0.80859375f, 1.546875f, -2.5f, -0.4296875f, -0.53515625f, 1.3125f, -0.4765625f, + -0.462890625f, -0.3203125f, 1.96875f, 1.46875f, -0.5f, -0.9296875f, -0.0849609375f, + -1.5078125f, -0.74609375f, 0.1494140625f, 1.3359375f, -0.4296875f, -0.7109375f, -0.421875f, + -0.56640625f, -0.6875f, -0.31640625f, 0.546875f, -1.25f, 0.10986328125f, 1.0703125f, + 0.73046875f, -0.66015625f, -0.52734375f, -0.41796875f, 0.94921875f, 0.396484375f, + 1.140625f, -0.5625f, -0.055908203125f, -0.47265625f, 1.609375f, 0.6328125f, -1.0703125f, + -0.90625f, -0.29296875f, 0.8515625f, -0.306640625f, -1.03125f, -0.30859375f, 0.494140625f, + -1.4765625f, -0.3359375f, -1.40625f, -2.546875f, 0.2109375f, -1.5234375f, -0.09619140625f, + 0.2294921875f, 1.3359375f, -0.2734375f, 1.6328125f, -1.7734375f, 0.31640625f, + -0.412109375f, 0.76953125f, 0.051513671875f, -1.1640625f, 0.0966796875f, 0.1962890625f, + -1.3203125f, 0.1083984375f, -0.49609375f, 0.36328125f, -0.48046875f, 0.001129150390625f, + -0.173828125f, 0.328125f, -0.6953125f, 1.0546875f, 0.59375f, -0.21484375f, -0.30078125f, + -0.283203125f, -0.34375f, -0.6484375f, -0.255859375f, 0.60546875f, -0.79296875f, + 0.1611328125f, 0.0654296875f, 0.4296875f, 0.2314453125f, 0.890625f, -0.361328125f, + -0.26171875f, 0.78515625f, -0.70703125f, -1.734375f, -0.7890625f, -1.484375f, -0.4296875f, + -0.53515625f, -0.71484375f, 2.234375f, -1.65625f, 1.0234375f, -0.400390625f, -0.044921875f, + 2.171875f, 0.5859375f, -0.94140625f, 0.4453125f, 0.5078125f, 0.05029296875f, 1.65625f, + -0.8203125f, 0.40234375f, 0.287109375f, -0.322265625f, -1.9453125f, -0.28515625f, + -0.08056640625f, 0.0f, -0.3828125f, -0.5859375f, 1.6484375f, -0.64453125f, -0.3359375f, + 1.1640625f, -0.255859375f, 0.44140625f, -2.171875f, -0.859375f, 1.3203125f, 0.2578125f, + 0.06103515625f, -0.58984375f, 0.2041015625f, 0.279296875f, -1.5078125f, 0.77734375f, + -0.384765625f, 0.3125f, -0.1669921875f, 0.396484375f, 1.9921875f, -0.337890625f, -1.15625f, + -0.2314453125f, 2.21875f, -0.486328125f, -1.203125f, -0.2080078125f, 1.015625f, + 0.94921875f, -0.08984375f, -0.9296875f, 0.671875f, 0.470703125f, 0.26953125f, -0.96484375f, + 0.1005859375f, 0.80859375f, -0.474609375f, 0.53125f, 0.349609375f, 0.44921875f, + -1.6640625f, 0.306640625f, -0.703125f, -0.291015625f, 1.078125f, -0.68359375f, -1.328125f, + -0.34765625f, 0.9453125f, 0.796875f, -0.99609375f, -2.640625f, -0.28515625f, -2.25f, + -0.484375f, -1.9765625f, -0.57421875f, 1.703125f, -2.171875f, -1.390625f, -0.90625f, + 0.107421875f, -0.2255859375f, 0.0634765625f, 0.3046875f, -0.5546875f, -0.55078125f, -0.5f, + 1.8359375f, 0.5859375f, -0.50390625f, 0.095703125f, -1.96875f, 0.59375f, 1.265625f, + -0.232421875f, 0.318359375f, -2.234375f, 0.875f, -0.58203125f, 1.625f, 1.046875f, + -0.546875f, 2.453125f, 0.07421875f, 1.7109375f, -0.53515625f, 0.7734375f, -1.4375f, + -2.015625f, 0.330078125f, -0.96875f, -0.5625f, -0.9375f, 0.07666015625f, -0.224609375f, + -0.2021484375f, 0.44921875f, -2.0f, 0.9375f, -0.4296875f, 0.58203125f, -0.8203125f, + -0.9765625f, 1.515625f, -0.9453125f, -0.494140625f, -0.875f, 0.72265625f, -1.453125f, + -2.125f, 0.2021484375f, -1.421875f, 0.5f, 0.333984375f, -1.2734375f, -1.453125f, 2.546875f, + 0.1904296875f, 1.171875f, -0.1611328125f, -0.63671875f, -0.5234375f, 1.7421875f, -0.25f, + 1.359375f, 0.6953125f, -0.69140625f, -0.51171875f, -1.8203125f, -0.69140625f, -0.7265625f, + -1.15625f, 0.54296875f, 0.455078125f, -0.6328125f, 0.78125f, -0.88671875f, -0.384765625f, + -1.578125f, 0.68359375f, -1.3515625f, 0.0791015625f, 0.98828125f, 0.1953125f, 0.8046875f, + 0.40625f, 2.3125f, -1.609375f, 0.921875f, -0.09814453125f, -0.197265625f, -0.91796875f, + -0.5703125f, 0.5859375f, 0.63671875f, -0.90625f, 1.7265625f, -1.1171875f, 0.447265625f, + 0.484375f, 0.78515625f, 1.9140625f, 0.134765625f, 0.8828125f, -0.328125f, 0.287109375f, + 0.291015625f, 0.19921875f, -1.859375f, 1.1796875f, 1.890625f, 0.0654296875f, -0.072265625f, + 0.234375f, -2.203125f, 0.283203125f, -0.828125f, -0.341796875f +}; +inline constexpr float kSubBudgetAttnOut[] = { + -0.026611328125f, 1.2109375f, -0.083984375f, -1.296875f, -1.421875f, -0.5703125f, + 2.328125f, 0.283203125f, -1.0703125f, -1.3671875f, -0.95703125f, 0.66015625f, 1.09375f, + 0.80859375f, 0.1142578125f, -0.59375f, 0.24609375f, 0.89453125f, 2.296875f, -1.2265625f, + -0.40625f, 0.0625f, -0.81640625f, -0.333984375f, -0.134765625f, -0.89453125f, + -0.1494140625f, 1.1875f, 1.0546875f, -1.6640625f, 0.6953125f, -1.3359375f, + -0.026611328125f, 1.2109375f, -0.083984375f, -1.296875f, -1.421875f, -0.5703125f, + 2.328125f, 0.283203125f, -1.0703125f, -1.3671875f, -0.95703125f, 0.66015625f, 1.09375f, + 0.80859375f, 0.1142578125f, -0.59375f, 0.24609375f, 0.89453125f, 2.296875f, -1.2265625f, + -0.40625f, 0.0625f, -0.81640625f, -0.333984375f, -0.134765625f, -0.89453125f, + -0.1494140625f, 1.1875f, 1.0546875f, -1.6640625f, 0.6953125f, -1.3359375f, -1.3671875f, + -0.53515625f, -1.5703125f, 3.109375f, -0.34375f, 0.9140625f, 0.05859375f, -0.5078125f, + 0.5390625f, 0.62109375f, 0.48046875f, 0.15234375f, 0.2314453125f, 0.40625f, 0.82421875f, + -0.81640625f, 1.1953125f, -1.5546875f, -1.1953125f, -1.4765625f, 0.2314453125f, + 0.66015625f, 2.09375f, 0.30859375f, 0.6015625f, 0.91796875f, -0.80859375f, -0.6875f, + -0.10009765625f, -0.2373046875f, 0.20703125f, 0.859375f, -1.3671875f, -0.53515625f, + -1.5703125f, 3.109375f, -0.34375f, 0.9140625f, 0.05859375f, -0.5078125f, 0.5390625f, + 0.62109375f, 0.48046875f, 0.15234375f, 0.2314453125f, 0.40625f, 0.82421875f, -0.81640625f, + 1.1953125f, -1.5546875f, -1.1953125f, -1.4765625f, 0.2314453125f, 0.66015625f, 2.09375f, + 0.30859375f, 0.6015625f, 0.91796875f, -0.80859375f, -0.6875f, -0.10009765625f, + -0.2373046875f, 0.20703125f, 0.859375f, -0.28824520111083984f, 0.5442845821380615f, + -0.5104231834411621f, -0.5582360625267029f, -0.7427027821540833f, -0.7314985394477844f, + 1.1700893640518188f, 0.4999435842037201f, -0.40991923213005066f, -0.6841030120849609f, + -0.4484345614910126f, -0.15203356742858887f, 1.7259747982025146f, -0.3666558861732483f, + -0.1052212342619896f, 0.029085349291563034f, 0.3791113495826721f, 0.8569733500480652f, + 1.8117519617080688f, -0.9808712005615234f, -0.0400603711605072f, -0.266131728887558f, + -0.2483428418636322f, -0.28508085012435913f, 0.6203048825263977f, -0.054173000156879425f, + 0.6365634799003601f, 0.8369595408439636f, 1.1266734600067139f, -0.08350041508674622f, + 0.5458633303642273f, -0.9861794710159302f, -0.4638381004333496f, 0.0968673825263977f, + -0.796623170375824f, -0.06250622868537903f, -0.2868833839893341f, -0.8396769165992737f, + 0.39288586378097534f, 0.6454067230224609f, 0.03329688683152199f, -0.22565791010856628f, + -0.1070951372385025f, -0.6971263885498047f, 2.1502861976623535f, -1.1554125547409058f, + -0.2525223195552826f, 0.4470948874950409f, 0.4683847725391388f, 0.8317667841911316f, + 1.4861667156219482f, -0.8159779906272888f, 0.20570403337478638f, -0.4866895377635956f, + 0.13290703296661377f, -0.252259761095047f, 1.1270626783370972f, 0.509824812412262f, + 1.1640641689300537f, 0.6016979217529297f, 1.1749862432479858f, 0.9772775769233704f, + 0.44556206464767456f, -0.7514430284500122f, -0.3640505373477936f, -0.15126565098762512f, + -0.07998133450746536f, 0.5250610709190369f, 0.13462790846824646f, -0.6743974685668945f, + -0.1913280487060547f, -1.02912175655365f, -0.34716323018074036f, 1.7158432006835938f, + -0.7676069736480713f, -0.752281129360199f, -1.2259206771850586f, -0.3542482256889343f, + 1.0910063982009888f, -1.1445244550704956f, -0.5342076420783997f, 0.6348114013671875f, + -0.726134181022644f, -0.3467397093772888f, -0.6616801023483276f, 0.015419373288750648f, + 1.596972942352295f, -1.3197311162948608f, 0.2673112750053406f, 0.7431768178939819f, + -0.15868210792541504f, 1.1340159177780151f, 0.5641883611679077f, -0.8851081132888794f, + -1.028778314590454f, -0.542026937007904f, -1.1830483675003052f, -0.4646880030632019f, + -1.29674232006073f, 2.6349897384643555f, -0.2559373676776886f, 0.6224794387817383f, + 0.012717274017632008f, -0.6035057306289673f, 0.3763839602470398f, 0.8220495581626892f, + 0.25136780738830566f, -0.013712212443351746f, -0.03607364743947983f, 0.26665040850639343f, + 0.8731911182403564f, -0.8766366839408875f, 0.8778359889984131f, -1.152775764465332f, + -1.1091885566711426f, -1.269168496131897f, 0.06750024110078812f, 0.541806161403656f, + 2.0025599002838135f, 0.00969301164150238f, 0.5402061939239502f, 0.8858833312988281f, + -0.6892938017845154f, -0.35313647985458374f, 0.02184089459478855f, -0.356217622756958f, + -0.019818071275949478f, 0.6021289825439453f, -0.17284919321537018f, 0.17840655148029327f, + -0.4595184326171875f, 0.3506753742694855f, 0.32203584909439087f, 0.1635088473558426f, + 0.2649783492088318f, -0.30783960223197937f, -0.5876909494400024f, 0.03734195604920387f, + -1.0479618310928345f, 0.7400622367858887f, 0.9110136032104492f, -0.6133343577384949f, + 0.4124186038970947f, 0.5824004411697388f, -0.578084409236908f, 0.19476161897182465f, + 1.4486135244369507f, -0.8428226709365845f, -0.9078854322433472f, 0.10172073543071747f, + 0.20756401121616364f, -0.2790588438510895f, -0.4528118669986725f, 0.23111537098884583f, + 0.8987100720405579f, -0.5538639426231384f, 1.4427694082260132f, 0.5988655090332031f, + -0.34500452876091003f, 1.4075053930282593f, -0.14756473898887634f, 0.37467798590660095f, + -0.39089953899383545f, 0.0326094776391983f, -0.01617608591914177f, 0.01447354070842266f, + 0.6597731709480286f, -0.18649539351463318f, -0.6741884350776672f, -0.23300518095493317f, + -1.0221494436264038f, 0.71188884973526f, 0.9570266008377075f, -0.3469093143939972f, + 0.34954899549484253f, 0.35677623748779297f, -0.41141989827156067f, 0.334006130695343f, + 1.6107980012893677f, -0.915798008441925f, -0.8021577000617981f, 0.08894476294517517f, + 0.011520971544086933f, -0.2892974615097046f, -0.3779183328151703f, 0.01885122060775757f, + 0.7011169195175171f, -0.2112209051847458f, 1.366315484046936f, 0.17050714790821075f, + -0.13940295577049255f, 0.8646422624588013f, -0.30836933851242065f, 0.08668415993452072f, + 0.5183581709861755f, 1.1786308288574219f, 0.12583506107330322f, 0.08993637561798096f, + -0.3939102292060852f, 0.08988497406244278f, 0.38784703612327576f, 0.9518250823020935f, + -0.985105574131012f, -1.2379652261734009f, 0.059222619980573654f, 0.234000101685524f, + 1.3717433214187622f, -1.1290582418441772f, -0.07230464369058609f, 0.9649280905723572f, + -0.22902122139930725f, 0.1967168003320694f, -0.14492930471897125f, 0.42046892642974854f, + 1.5268484354019165f, -0.41272222995758057f, -0.2971172332763672f, -0.2677343487739563f, + -0.7183435559272766f, -0.0070132422260940075f, 0.23245973885059357f, -0.6640809178352356f, + -0.16901549696922302f, -0.393371045589447f, -0.5722255110740662f, -0.10739006847143173f, + -0.09522511810064316f, 1.5158764123916626f, 0.015196459367871284f, 0.14139650762081146f, + -0.24707718193531036f, -0.2663099765777588f, 0.28395000100135803f, 1.0183137655258179f, + -0.5931264162063599f, -0.8128821849822998f, -0.14448393881320953f, 0.1631011664867401f, + 1.1872903108596802f, -1.0572261810302734f, 0.14286011457443237f, 0.29938533902168274f, + -0.5548667907714844f, -0.30705562233924866f, -0.1533816009759903f, 0.4006100594997406f, + 1.6758016347885132f, -0.41304028034210205f, 0.02538485638797283f, 0.2084995061159134f, + -0.6332796812057495f, 0.04766543582081795f, 0.21613407135009766f, -0.6041248440742493f, + -0.2429109662771225f, -0.12207042425870895f, 0.2844766080379486f, 0.19283348321914673f, + -0.11157319694757462f, -0.1896081119775772f, -0.19976051151752472f, 0.18203461170196533f, + 0.715401828289032f, -0.37914761900901794f, -0.8840084671974182f, -0.31934964656829834f, + -1.0496277809143066f, 0.4617823660373688f, 1.3089120388031006f, -0.9089680910110474f, + 0.5614433884620667f, -0.10532765090465546f, -0.42374536395072937f, 0.27817502617836f, + 1.9587037563323975f, -0.08238305896520615f, -1.0326207876205444f, -0.3017435669898987f, + 0.45811817049980164f, -0.5016294717788696f, -0.8095319271087646f, -0.1483440399169922f, + -0.05486529693007469f, -0.1565733104944229f, 1.2460068464279175f, -0.5251826643943787f, + 0.09682674705982208f, 0.46367886662483215f, 0.4247056245803833f, -0.055124007165431976f, + -0.07952900975942612f, -0.07968451827764511f, -0.07827107608318329f, 0.26447123289108276f, + 0.4375561475753784f, -0.456141859292984f, -0.8389180302619934f, -0.17024773359298706f, + -0.9774956107139587f, 0.21205322444438934f, 1.567678689956665f, -1.4346059560775757f, + 0.6323004961013794f, -0.14576514065265656f, -0.44613733887672424f, 0.21260517835617065f, + 1.9881293773651123f, 0.35160204768180847f, -1.0960534811019897f, -0.5318057537078857f, + 0.8067365288734436f, -0.5852891206741333f, -0.885981559753418f, -0.02147264964878559f, + -0.2043510228395462f, -0.2781490087509155f, 1.2237143516540527f, -0.4498051106929779f, + 0.12563610076904297f, 0.4791536331176758f, -0.22286976873874664f, -0.3251565396785736f, + 0.17746502161026f, 0.9928246736526489f, 0.14304973185062408f, 0.4395642876625061f, + -0.2444477528333664f, 0.24632667005062103f, 0.5280030369758606f, 1.0328238010406494f, + -0.7575457692146301f, -0.8974823951721191f, 0.8077029585838318f, -0.1047036424279213f, + 0.9525941610336304f, -0.28814437985420227f, -0.21171264350414276f, 1.264561414718628f, + 0.2920287251472473f, 0.39907264709472656f, -0.5863776803016663f, 0.05122097209095955f, + 1.06709885597229f, -0.19859720766544342f, -0.42537257075309753f, -0.46169957518577576f, + -0.9341620206832886f, -0.6945581436157227f, -0.08701997995376587f, -0.2605833411216736f, + 0.11951788514852524f, -0.36070558428764343f, -0.35143059492111206f, -0.6983656287193298f, + -0.5034327507019043f, 0.927893340587616f, 0.08880984783172607f, 0.43301206827163696f, + -0.03335339203476906f, -0.18533486127853394f, 0.3384522497653961f, 1.313585638999939f, + -0.3617214262485504f, -0.3571200668811798f, 0.7032938599586487f, -0.46411189436912537f, + 0.5701541304588318f, 0.1505691409111023f, -0.2750309109687805f, 0.9298515915870667f, + 0.2173275500535965f, 0.055976465344429016f, -0.9391694664955139f, -0.2441277951002121f, + 0.9538595676422119f, -0.3379974365234375f, -0.14752314984798431f, -0.006387163419276476f, + -0.8383407592773438f, -0.686703622341156f, -0.16059935092926025f, -0.09196863323450089f, + -0.014935356564819813f, -0.225823312997818f, 0.05972803756594658f, 0.3894052803516388f, + -0.25532621145248413f, -0.5071677565574646f, -0.5436530709266663f, -0.29216957092285156f, + 0.9851586222648621f, 0.06438352167606354f, -0.6264452934265137f, -0.6349583864212036f, + -0.7488199472427368f, 0.12182945013046265f, 1.5525555610656738f, -0.6361541748046875f, + 0.22370806336402893f, -0.13632448017597198f, 0.0582905039191246f, 0.596360981464386f, + 1.7861419916152954f, -0.5243282318115234f, -0.5472158193588257f, -0.38077154755592346f, + 0.11821922659873962f, -0.3961513042449951f, -0.18092235922813416f, -0.15185829997062683f, + 0.03975784778594971f, 0.42150261998176575f, 0.981306254863739f, -0.507681667804718f, + 0.37014439702033997f, -0.475343257188797f, 0.3214341104030609f, 0.10809741914272308f, + -0.1304014027118683f, -0.38792362809181213f, -0.39273661375045776f, -0.06279347091913223f, + 0.7057474851608276f, -0.1582683026790619f, -0.7009918689727783f, -0.46068939566612244f, + -0.7838643193244934f, -0.02611425891518593f, 1.75420343875885f, -1.2290356159210205f, + 0.4145972728729248f, -0.2566286325454712f, -0.08959803730249405f, 0.44703206419944763f, + 1.9096194505691528f, 0.09633997827768326f, -0.7801675796508789f, -0.6224793195724487f, + 0.5725762844085693f, -0.5312857031822205f, -0.5119985342025757f, -0.08876793086528778f, + -0.27594494819641113f, 0.1806916743516922f, 0.9910980463027954f, -0.6078197956085205f, + 0.36377832293510437f, -0.29465606808662415f, -0.39655178785324097f, -0.18006093800067902f, + -0.13880524039268494f, 1.5672663450241089f, -0.07063768804073334f, -0.4622584283351898f, + -0.10481903702020645f, -0.10978244990110397f, -0.20564429461956024f, 0.4014115035533905f, + -0.4621962904930115f, -0.7706999182701111f, 0.20452746748924255f, 0.415378212928772f, + 0.8641610145568848f, -0.5852794647216797f, 0.20964880287647247f, 0.24622701108455658f, + -0.04020072519779205f, -0.4241288900375366f, 0.4086697995662689f, -0.14934730529785156f, + 1.1517211198806763f, 0.08924311399459839f, -0.38184866309165955f, -0.2348359376192093f, + -0.847701370716095f, 0.01786571741104126f, 0.17083363234996796f, -0.2671675682067871f, + 0.35496488213539124f, 0.7086684107780457f, -0.19159482419490814f, -0.41379350423812866f, + -0.19186338782310486f, 0.11109080910682678f, 0.18948525190353394f, -0.6650078892707825f, + -0.09451484680175781f, -1.0041921138763428f, -0.40301087498664856f, 1.868169903755188f, + -0.7182402014732361f, -0.5853957533836365f, -0.8756381273269653f, -0.6720739006996155f, + 0.7916818857192993f, -0.55763840675354f, -0.8203418850898743f, 1.0505304336547852f, + -0.29637691378593445f, -0.11956309527158737f, -1.039546251296997f, -0.3630942106246948f, + 1.1902916431427002f, -1.334233045578003f, 0.13568902015686035f, 0.5962766408920288f, + -0.23735420405864716f, 0.881001889705658f, 0.4182606637477875f, -0.6535521745681763f, + -0.9426628351211548f, -0.6074494123458862f, 0.6300150752067566f, -0.05743039771914482f, + 0.24027083814144135f, -0.22995337843894958f, 0.13214536011219025f, 0.10119984298944473f, + 0.2907949984073639f, -0.13933996856212616f, 0.009737621992826462f, -0.34544235467910767f, + -0.29201075434684753f, -0.1313297003507614f, 1.648990511894226f, -1.6420944929122925f, + 0.06543562561273575f, -0.3347803056240082f, 0.10222917795181274f, 0.15429621934890747f, + 1.0704069137573242f, 0.16978377103805542f, -0.39749786257743835f, -0.3906928300857544f, + 1.036831021308899f, -0.5492029190063477f, -0.40591856837272644f, -0.06807099282741547f, + -0.27179068326950073f, 0.2620823383331299f, 0.3773680031299591f, -0.37906378507614136f, + 0.05769737809896469f, 0.1579088568687439f, 0.2624657452106476f, 0.05316108465194702f, + 0.04729829728603363f, -0.025982851162552834f, 0.46854740381240845f, -0.13784049451351166f, + 0.03296022117137909f, 0.18000036478042603f, 0.627250611782074f, -0.3084731996059418f, + 0.023523230105638504f, -0.08601755648851395f, 1.3842198848724365f, -1.2760721445083618f, + -0.3734070360660553f, 0.11122675240039825f, 0.3565559983253479f, 0.18801942467689514f, + 0.22259986400604248f, -0.7383838891983032f, 0.10194186866283417f, -0.03206953778862953f, + 0.7427340745925903f, -0.30639052391052246f, 0.21306738257408142f, 0.17108671367168427f, + 0.418451726436615f, 0.36544522643089294f, 0.06366733461618423f, 0.45085662603378296f, + -0.2290327250957489f, 0.4292183220386505f, -0.3886135220527649f, -1.2901840209960938f, + -1.0696032047271729f, 0.5929092764854431f, 0.11617711186408997f, 0.4188016951084137f, + 0.3506823778152466f, -0.3026833236217499f, -0.11329810321331024f, 1.46333646774292f, + 0.1019873097538948f, 0.24682089686393738f, 0.7222087979316711f, -0.8481384515762329f, + -0.13116303086280823f, 1.3315917253494263f, -0.43047621846199036f, 0.8919723033905029f, + 0.5498601198196411f, -0.009047841653227806f, -1.454352855682373f, -1.040893793106079f, + 0.25520578026771545f, -0.24882251024246216f, -0.10181034356355667f, 0.47575250267982483f, + -0.9157996773719788f, -0.8146288394927979f, -0.252695232629776f, 0.20818212628364563f, + 0.042895667254924774f, 0.13711997866630554f, -0.8028650879859924f, -0.443246990442276f, + 0.722126841545105f, 1.4668160676956177f, 0.2774066627025604f, 0.22859607636928558f, + 0.368970662355423f, 1.1107999086380005f, -0.8741307854652405f, 0.09607403725385666f, + -0.32124799489974976f, -0.9022133350372314f, -0.9817913770675659f, 0.9551556706428528f, + 0.5206575393676758f, 0.8223052620887756f, 0.6088659167289734f, 0.3116883933544159f, + -0.1283876597881317f, 0.2954239249229431f, 0.00023011182202026248f, -0.878615140914917f, + 0.11317215859889984f, 0.6128676533699036f, -0.5046163201332092f, 1.2424665689468384f, + -1.1548466682434082f, 0.3125862777233124f, 0.6625553965568542f, -0.8135644197463989f, + 0.1888323724269867f, 1.0968080759048462f, 0.3316984176635742f, 0.35758137702941895f, + 0.22067642211914062f, -0.6016979217529297f, -0.055777665227651596f, 0.01578574627637863f, + 0.31675228476524353f, 0.008533247746527195f, 0.15279808640480042f, -0.7800213098526001f, + -0.2850501835346222f, 0.30899086594581604f, 0.9478123784065247f, -0.4143315553665161f, + 0.1629449874162674f, -0.10397317260503769f, 0.3868854343891144f, 0.44715872406959534f, + -0.2628350853919983f, -0.7875645756721497f, -0.26869121193885803f, 0.2371138036251068f, + 0.336860328912735f, -0.3413252830505371f, -0.3590680658817291f, 0.20585857331752777f, + 0.0825614333152771f, 0.2618445158004761f, -0.33991676568984985f, -0.3477679491043091f, + -0.4427129328250885f, 0.054823558777570724f, 0.8136963844299316f, -0.18967488408088684f, + 0.2653498649597168f, -0.41536250710487366f, -0.25671085715293884f, 0.3034062385559082f, + 0.3561212718486786f, -0.42821550369262695f, -0.6753195524215698f, -0.3481963574886322f, + -0.6366428732872009f, -0.19988542795181274f, 1.8694627285003662f, -1.8372306823730469f, + 0.682015061378479f, -0.5715850591659546f, -0.13153202831745148f, 0.2731260657310486f, + 1.6322993040084839f, 0.9810581207275391f, -0.9438574314117432f, -0.6675696969032288f, + 1.1809300184249878f, -0.7542684078216553f, -0.9443447589874268f, 0.010719231329858303f, + -0.6620496511459351f, -0.024527201429009438f, 0.6922031044960022f, -0.9305359125137329f, + 0.19670498371124268f, -0.12817740440368652f, -0.34585750102996826f, -0.4922694265842438f, + -0.18558764457702637f, 0.47160372138023376f, 0.14551155269145966f, -0.22852295637130737f, + 0.08537975698709488f, -0.264299213886261f, -0.41369977593421936f, 1.070685625076294f, + -0.3627164661884308f, -0.33800968527793884f, -0.4015396237373352f, -0.30772197246551514f, + 0.38571757078170776f, 0.2635144889354706f, -0.24872475862503052f, 0.7918441891670227f, + 0.08526305854320526f, -0.025643089786171913f, -0.7857487201690674f, -0.6961181163787842f, + 0.7050402760505676f, -0.434672087430954f, -0.19365879893302917f, 0.5932007431983948f, + -0.6307589411735535f, 0.3591266870498657f, 0.268642395734787f, -0.5095994472503662f, + -0.3659704625606537f, 0.10674092918634415f, -0.13437704741954803f, 0.23892195522785187f, + -0.08028996735811234f, 0.24887485802173615f, -0.0647776648402214f, -0.3500787615776062f, + -0.23625732958316803f, -0.05687416344881058f, -0.01584310084581375f, 0.03058207780122757f, + -0.2927148938179016f, -0.04397781565785408f, 0.2185097187757492f, -0.21016274392604828f, + 0.18187232315540314f, -0.06744520366191864f, 0.2623229920864105f, 0.7009584903717041f, + 0.5514501929283142f, -0.13480505347251892f, -0.25718674063682556f, -0.5384837985038757f, + 1.0052144527435303f, 0.14434920251369476f, -0.659771740436554f, -0.21845269203186035f, + -0.6329430937767029f, 0.3778187930583954f, 0.041794028133153915f, -0.6387669444084167f, + -0.10814376175403595f, 0.37058496475219727f, 0.15835008025169373f, 0.21364726126194f, + -0.06030448153614998f, -0.5161888003349304f, -0.003798362333327532f, -0.12661373615264893f, + 0.3018805980682373f, -0.0037550150882452726f, 0.07385117560625076f, -0.6321976780891418f, + -0.3678427040576935f, 0.10628897696733475f, 1.18121337890625f, -0.731518566608429f, + 0.15796469151973724f, -0.010481252335011959f, 0.17466412484645844f, 0.39757612347602844f, + 0.33132073283195496f, -0.7440836429595947f, -0.38827821612358093f, -0.16291482746601105f, + 0.4119349718093872f, -0.3347005844116211f, -0.3446824252605438f, 0.12987981736660004f, + -0.03167911246418953f, 0.1568833291530609f, 0.06350003182888031f, -0.1835283637046814f, + -0.18943415582180023f, 0.016633624210953712f, -0.03349590674042702f, 0.30922117829322815f, + -0.20765434205532074f, -0.696196973323822f, -0.21798798441886902f, -0.2303563803434372f, + 0.5722498893737793f, -0.0036030265036970377f, -0.17638541758060455f, -0.5806092619895935f, + -0.37291693687438965f, 0.11809410899877548f, 1.0442742109298706f, -0.4201852083206177f, + 0.3410540521144867f, -0.030751291662454605f, -0.053187888115644455f, 0.45125627517700195f, + 0.7997682690620422f, -0.7421514987945557f, -0.4418151080608368f, -0.02180287055671215f, + 0.1893148273229599f, -0.3813473880290985f, -0.2940068244934082f, 0.02496071718633175f, + 0.23827852308750153f, 0.10039705783128738f, 0.5818777084350586f, -0.24310126900672913f, + -0.12140879780054092f, -0.0027800118550658226f, -0.1571798175573349f, -0.3233298361301422f, + -0.12919265031814575f, 0.10788017511367798f, 0.38223662972450256f, -0.33464065194129944f, + -0.04213321954011917f, 0.0645168125629425f, 0.2923935651779175f, 0.7391561269760132f, + -0.06337347626686096f, -0.6236850619316101f, 0.1191263347864151f, 0.17129068076610565f, + 0.11770343035459518f, -0.8284963965415955f, -0.6199625134468079f, 0.1966424584388733f, + 0.05092868208885193f, 0.03453398495912552f, -0.7561595439910889f, 0.8269903063774109f, + -0.2619340121746063f, 0.20573319494724274f, -0.44538530707359314f, -0.20087628066539764f, + 0.5260416865348816f, 0.13625021278858185f, -0.38993778824806213f, -0.011228980496525764f, + 0.15035808086395264f, -0.21682600677013397f, -0.49922212958335876f, -0.13538409769535065f, + 0.1465466171503067f, 0.9127625226974487f, 0.144523024559021f, -0.23277787864208221f, + -0.018099525943398476f, 0.027001099660992622f, -0.3065710663795471f, 0.6245961785316467f, + -0.3958510458469391f, -0.6604485511779785f, -0.675020694732666f, 0.29087600111961365f, + 0.651290237903595f, -0.4058667719364166f, 0.10844723880290985f, 0.2936103641986847f, + -0.28078797459602356f, -0.14339657127857208f, -0.23431728780269623f, -0.1313508152961731f, + 0.8413128852844238f, -0.10851076245307922f, -0.2366591989994049f, 0.5830575823783875f, + -0.47061771154403687f, 0.5637975931167603f, 0.35522082448005676f, -0.7050225734710693f, + -0.23729680478572845f, 0.2900647521018982f, 0.9604904651641846f, -0.2907956838607788f, + 0.32633501291275024f, -0.14525312185287476f, 0.06932016462087631f, -0.8604225516319275f, + -0.011448361910879612f, -0.10819929093122482f, 0.25357505679130554f, -0.4049817621707916f, + -0.25576484203338623f, 0.8798804879188538f, 0.09863530844449997f, -0.5759591460227966f, + 0.5620373487472534f, -0.21031564474105835f, 0.17562392354011536f, -1.0046433210372925f, + -0.35454821586608887f, 0.27040567994117737f, 0.017382066696882248f, 0.1895163655281067f, + 0.10521729290485382f, -0.1339138299226761f, -0.10222163796424866f, -0.7653024792671204f, + 0.4366779327392578f, -0.14168374240398407f, 0.17697970569133759f, -0.1646272987127304f, + -0.08286996185779572f, 1.3455053567886353f, 0.21323701739311218f, 0.0849476084113121f, + -0.09228820353746414f, -0.7989469766616821f, -0.17368976771831512f, -0.35705262422561646f, + 0.3308413624763489f, -0.02611749991774559f, 0.04321155697107315f, -0.680622935295105f, + -0.24876068532466888f, -0.05582026392221451f, 1.1282925605773926f, -0.7756540179252625f, + 0.37297505140304565f, -0.19212178885936737f, 0.09386657923460007f, 0.25796741247177124f, + 0.4147082269191742f, -0.42593449354171753f, -0.39644917845726013f, -0.3408060073852539f, + 0.41103801131248474f, -0.37882399559020996f, -0.41228142380714417f, -0.06616388261318207f, + -0.23770208656787872f, 0.10995201021432877f, 0.1363956481218338f, -0.42393234372138977f, + -0.023161204531788826f, -0.179918110370636f, -0.48830869793891907f, -0.31802311539649963f, + 0.4088783860206604f, 1.3494062423706055f, 0.07072962075471878f, 0.09483068436384201f, + -0.028377719223499298f, 0.8441050052642822f, 0.05686011165380478f, 0.1642424613237381f, + -0.5439181923866272f, -0.582214891910553f, 0.004336376674473286f, 0.4983421862125397f, + 0.39778703451156616f, 0.17432761192321777f, 0.4749343693256378f, 0.47176361083984375f, + 0.192714661359787f, 0.31736326217651367f, -0.04503212124109268f, -0.5514306426048279f, + 0.5634661912918091f, 0.2540591359138489f, -0.5333276391029358f, 0.4427283704280853f, + -0.9510871767997742f, -0.28910085558891296f, 0.17130863666534424f, -0.39394351840019226f, + 0.2897428870201111f, 0.25914210081100464f, -0.2426905333995819f, -0.4439747929573059f, + -0.14499858021736145f, 0.5646568536758423f, 0.12915906310081482f, -0.8537189364433289f, + 0.12404440343379974f, -0.1757524162530899f, -0.6394346356391907f, 0.6126939654350281f, + -0.2769235074520111f, -0.4650982916355133f, -0.38351330161094666f, 0.10415992885828018f, + 0.21302232146263123f, 0.0007759648142382503f, -0.2592250406742096f, 0.4006202518939972f, + 0.16885149478912354f, -0.20217105746269226f, -0.23884548246860504f, -0.5940000414848328f, + 0.2522220313549042f, -0.09664376825094223f, -0.4135498106479645f, 0.2930152714252472f, + -0.37506580352783203f, 0.5255398154258728f, 0.19631560146808624f, -0.24395130574703217f, + 0.054592348635196686f, 0.5365335941314697f, -0.22470133006572723f, -0.11731003224849701f, + -0.5463300347328186f, -0.3689008355140686f, -0.49534887075424194f, -0.5796115398406982f, + 0.3660888373851776f, -0.4177853763103485f, -0.38193923234939575f, -0.5240340232849121f, + -0.37386929988861084f, 0.2617330849170685f, 0.6570069193840027f, -0.4137464463710785f, + 0.4467026889324188f, -0.025142403319478035f, -0.030047938227653503f, 0.2661203145980835f, + 0.2980414032936096f, -0.2421569675207138f, -0.30116015672683716f, -0.43113940954208374f, + 0.17048710584640503f, 0.021459801122546196f, -0.18545298278331757f, 0.037163883447647095f, + -0.0175457913428545f, -0.021135762333869934f, 0.2930169999599457f, 0.30985933542251587f, + 0.10735605657100677f, 0.2941305339336395f, 0.03931208327412605f, 0.22167518734931946f, + -0.16932925581932068f, -1.0513966083526611f, -0.4734361171722412f, -0.3985033929347992f, + 0.8123400807380676f, -0.09657879173755646f, -0.2031606286764145f, -0.7095786333084106f, + -0.16473115980625153f, -0.11801759898662567f, 1.0645439624786377f, -0.5407727360725403f, + 0.3743923008441925f, -0.41151803731918335f, -0.09533414989709854f, 0.3825282156467438f, + 1.0335546731948853f, -0.3597835600376129f, -0.3676178455352783f, -0.29181405901908875f, + 0.3287993371486664f, -0.4197346866130829f, -0.3868423104286194f, -0.2982743978500366f, + -0.2684146761894226f, 0.28785642981529236f, 0.5380104780197144f, -0.6512636542320251f, + 0.16866597533226013f, -0.375243216753006f, -0.21709655225276947f, 0.390668123960495f, + -0.27078723907470703f, 0.3180198073387146f, -0.13034109771251678f, -0.5204602479934692f, + -0.31452953815460205f, -0.21411362290382385f, 0.15552152693271637f, -0.13549792766571045f, + -0.13508838415145874f, 0.17812851071357727f, -0.026082275435328484f, -0.11823255568742752f, + 0.04171301797032356f, -0.5812446475028992f, 0.4339809715747833f, 0.10353337228298187f, + 0.26052093505859375f, -0.43400099873542786f, -0.12181729078292847f, -0.17484959959983826f, + 1.1085436344146729f, 0.13401587307453156f, -0.5140317678451538f, -0.029369449242949486f, + -0.2365301102399826f, 0.5976351499557495f, -0.027960430830717087f, -0.6662565469741821f, + -0.2008592188358307f, 0.27927184104919434f, -0.08626505732536316f, -0.001982332905754447f, + -0.04373234137892723f, 0.3059651553630829f, 0.12041379511356354f, -0.7188816666603088f, + -0.25147759914398193f, -0.022559819743037224f, 0.06575259566307068f, 0.2004852443933487f, + -0.21911443769931793f, -0.19745084643363953f, -0.2019301801919937f, 0.12051981687545776f, + 0.03914763033390045f, -0.8078148365020752f, 0.018183767795562744f, -0.015802649781107903f, + 0.10424472391605377f, -0.24622416496276855f, -0.29907262325286865f, 0.22094574570655823f, + 0.3316322863101959f, 0.03232875093817711f, -0.4671972095966339f, 0.12682834267616272f, + 0.08945731073617935f, 0.36959630250930786f, -0.10023785382509232f, -0.3701481521129608f, + -0.0021899864077568054f, 0.0367092601954937f, -0.3867378830909729f, 0.3087184429168701f, + -0.4057835340499878f, -1.1042346954345703f, -0.5475915670394897f, -0.59185791015625f, + 0.8709693551063538f, -0.09480389952659607f, -0.22474072873592377f, -0.7824532985687256f, + -0.20976100862026215f, 0.08608207106590271f, 0.6805365681648254f, 0.02561233565211296f, + 0.36739400029182434f, -0.19945979118347168f, -0.19782176613807678f, 0.44300904870033264f, + 0.8516229391098022f, -0.856132984161377f, -0.3752572536468506f, -0.18114778399467468f, + -0.11381193995475769f, -0.24056725203990936f, -0.33342233300209045f, -0.240420401096344f, + -0.09411133080720901f, 0.1900024712085724f, 0.608700692653656f, -0.38893675804138184f, + 0.13384422659873962f, -0.302290141582489f, -0.23083047568798065f, -0.13326691091060638f, + -0.4191693365573883f, 0.08006859570741653f, 0.21656320989131927f, -0.29173165559768677f, + -0.12477077543735504f, -0.14963246881961823f, 0.008793368935585022f, -0.253532350063324f, + -0.4929405152797699f, 0.25039103627204895f, 0.9934872984886169f, -0.8249835968017578f, + 0.17857597768306732f, 0.43621936440467834f, -0.06283655017614365f, 0.2087729126214981f, + 0.3546321988105774f, -0.7173406481742859f, -0.46813106536865234f, -0.28754937648773193f, + 0.25970712304115295f, -0.11578009277582169f, -0.1820487380027771f, 0.41263678669929504f, + 0.3874438405036926f, -0.26384469866752625f, 0.48001912236213684f, 0.7293397784233093f, + -0.16321246325969696f, 0.6530229449272156f, 0.739771842956543f, -0.16780133545398712f, + 0.8555228114128113f, 0.4903743267059326f, 0.27073389291763306f, -0.7888914942741394f, + -0.8077353239059448f, 0.5288298726081848f, 0.0758044496178627f, -0.01391003467142582f, + -0.7176733613014221f, -0.5585671663284302f, 0.18694016337394714f, 0.25263094902038574f, + 0.4460717439651489f, -0.986214816570282f, 0.2657785713672638f, 0.29765647649765015f, + 0.34359365701675415f, 0.03262334689497948f, -0.23831208050251007f, 0.475517600774765f, + -0.05603095516562462f, -0.10882974416017532f, -0.5767168998718262f, 0.024054644629359245f, + -0.6571133136749268f, -0.7861530780792236f, -0.1339605152606964f, -0.434553861618042f, + 0.3547247052192688f, -0.16652081906795502f, -0.006515860091894865f, 0.05210874229669571f, + 0.3217316269874573f, 0.6298952698707581f, -0.01819034107029438f, -0.5469860434532166f, + -0.39381542801856995f, 0.15931974351406097f, 0.2560712695121765f, 0.2648566961288452f, + -0.6743631958961487f, -0.48141932487487793f, 0.10383067280054092f, 0.053870003670454025f, + 0.47376856207847595f, -0.5900781154632568f, 0.13109523057937622f, 0.6911965012550354f, + 0.275994211435318f, 0.10883095115423203f, -0.08027701079845428f, -0.22389967739582062f, + 0.9247481226921082f, -0.2137146145105362f, -0.47703731060028076f, -0.13353972136974335f, + -0.5279659628868103f, 0.12848295271396637f, 0.008636358194053173f, -0.45507732033729553f, + -0.06128239259123802f, -0.21829012036323547f +}; + +// ---- case 'over_budget': kv_len = 23 (top-k discards blocks) +inline constexpr int64_t kOverBudgetSeq = 23; +inline constexpr float kOverBudgetQRaw[] = { + -0.205078125f, 0.2099609375f, -0.359375f, 0.59375f, -0.95703125f, 0.546875f, -0.388671875f, + -0.359375f, 0.2734375f, -0.796875f, -0.08349609375f, 0.07275390625f, 0.41015625f, + -0.034912109375f, -0.369140625f, 1.25f, -0.267578125f, -0.51953125f, -0.107421875f, + 0.006439208984375f, 0.6171875f, -0.66015625f, -0.1142578125f, -0.6328125f, 0.60546875f, + -0.7421875f, -0.8125f, 0.42578125f, 0.341796875f, 0.69140625f, -0.2734375f, -0.298828125f, + -0.30078125f, -0.265625f, -0.66796875f, -0.55859375f, 0.2373046875f, 1.15625f, + -0.50390625f, -0.1279296875f, -0.251953125f, 0.23828125f, 0.3046875f, 1.0546875f, + -0.44140625f, 0.0010223388671875f, 0.58984375f, -0.80078125f, 0.5859375f, -0.490234375f, + -0.1484375f, 0.5703125f, 0.337890625f, -0.8203125f, -0.1318359375f, -0.62109375f, + -0.60546875f, -0.70703125f, -0.6015625f, 0.11572265625f, 0.83984375f, 1.03125f, + -0.322265625f, -0.25390625f, -0.1943359375f, -0.169921875f, 0.55078125f, + 0.00022792816162109375f, -0.00921630859375f, -0.296875f, 0.369140625f, 0.388671875f, + 0.052490234375f, 0.734375f, 0.1845703125f, -0.04248046875f, 0.0101318359375f, + 0.0693359375f, 0.515625f, -0.11328125f, 1.0078125f, -0.474609375f, -0.62109375f, + -0.6328125f, 0.75390625f, 0.3125f, -0.19921875f, 0.039306640625f, 0.2490234375f, + -0.365234375f, 0.314453125f, 1.46875f, 0.50390625f, -0.546875f, -1.0234375f, + -0.08837890625f, 0.59765625f, -0.57421875f, -0.07421875f, 1.046875f, -0.28515625f, + -0.3125f, 0.058837890625f, -0.546875f, -0.041259765625f, 0.5234375f, 0.435546875f, + -0.064453125f, 0.271484375f, 0.02685546875f, 0.0022125244140625f, 0.095703125f, + 0.283203125f, 0.6015625f, -0.2734375f, 0.59375f, -0.5625f, 0.828125f, -0.65234375f, + -0.30078125f, -0.44140625f, 0.2080078125f, -0.052001953125f, 0.294921875f, -0.6484375f, + 0.47265625f, 0.3515625f, 0.1513671875f, -0.265625f, 1.0234375f, 0.24609375f, -0.48828125f, + -0.6015625f, 1.1171875f, -0.038330078125f, 0.2333984375f, -0.78515625f, -0.4921875f, + 0.2138671875f, -0.458984375f, -0.3984375f, 0.11767578125f, 0.19921875f, 1.4921875f, + -0.58984375f, -0.890625f, 0.033935546875f, -0.87890625f, 0.451171875f, 0.91796875f, + 0.9453125f, 0.30859375f, 0.482421875f, 0.59765625f, -0.07763671875f, -0.419921875f, + -0.1337890625f, 0.94921875f, -0.0294189453125f, -0.07470703125f, 0.6953125f, 0.2578125f, + 1.375f, -0.06787109375f, 0.33984375f, -0.236328125f, 0.3359375f, -0.69140625f, 0.46875f, + 0.353515625f, 0.365234375f, -0.2734375f, -1.3359375f, -0.78125f, -0.302734375f, + -0.88671875f, 0.59375f, 0.9140625f, 0.5703125f, 0.08056640625f, -0.09814453125f, + -0.201171875f, 0.67578125f, -0.3359375f, -0.173828125f, -0.875f, 0.474609375f, + -0.0067138671875f, 0.458984375f, -0.56640625f, 0.15234375f, -0.400390625f, 0.4453125f, + 0.279296875f, -1.0234375f, 0.380859375f, -0.66796875f, -0.8671875f, -0.271484375f, + 0.2890625f, -0.341796875f, 0.83203125f, -0.99609375f, 0.3828125f, -0.6484375f, 0.3828125f, + 0.05126953125f, 0.1025390625f, -0.2890625f, 1.09375f, -0.326171875f, 0.62109375f, + -0.30078125f, 0.2470703125f, -0.08544921875f, 0.1083984375f, 0.353515625f, 0.94140625f, + 0.796875f, 0.33984375f, -1.5f, 0.8359375f, -0.43359375f, -0.83984375f, 0.08935546875f, + 0.263671875f, -1.140625f, 0.11767578125f, 0.5f, 0.453125f, 0.345703125f, -0.37109375f, + 0.244140625f, -0.1474609375f, 0.41796875f, 0.7421875f, 0.294921875f, 0.341796875f, + -0.60546875f, -0.37109375f, 0.5234375f, 0.08837890625f, -0.625f, 0.359375f, + 0.0076904296875f, 0.56640625f, 0.71484375f, 0.88671875f, 0.7578125f, -0.41015625f, + 0.65625f, -0.134765625f, 0.357421875f, -0.08740234375f, 0.640625f, -0.71484375f, + 0.216796875f, -0.1962890625f, 0.46484375f, -1.234375f, 0.07177734375f, 0.91015625f, + -0.40625f, -0.283203125f, -0.33203125f, -0.19921875f, 0.166015625f, -0.345703125f, + -0.314453125f, -0.3359375f, -0.283203125f, -0.2109375f, 0.2197265625f, -0.6875f, + 0.4140625f, -0.060546875f, 0.10400390625f, 0.8828125f, 0.0098876953125f, -0.57421875f, + -0.87890625f, -0.435546875f, 0.41796875f, -0.859375f, -0.388671875f, -0.05419921875f, + 0.2734375f, 0.55078125f, -0.470703125f, -0.447265625f, 0.08251953125f, 0.53515625f, + 0.34375f, -0.255859375f, 0.248046875f, -0.61328125f, 1.3828125f, -0.1220703125f, + 1.3671875f, -0.447265625f, 1.28125f, -0.431640625f, 0.3515625f, 0.1875f, -0.4375f, + -0.4609375f, -0.1796875f, -0.0220947265625f, 0.04052734375f, -0.1357421875f, 0.83203125f, + -0.408203125f, -0.169921875f, -0.458984375f, 0.93359375f, -0.91796875f, 0.2578125f, + -0.51171875f, 0.04833984375f, 1.234375f, 1.421875f, 0.421875f, -0.74609375f, + -0.1728515625f, 0.75f, -0.9375f, -0.58203125f, 0.75390625f, -0.294921875f, -0.515625f, + -0.205078125f, 0.306640625f, -0.07666015625f, 0.8828125f, -0.44921875f, 0.578125f, + -0.5625f, 0.236328125f, -0.390625f, 1.1640625f, 0.498046875f, 0.30078125f, 0.3046875f, + -0.26171875f, 0.56640625f, 0.490234375f, -0.130859375f, -0.5390625f, -0.03466796875f, + 0.91015625f, -1.03125f, -0.75390625f, -0.333984375f, -0.07421875f, -0.7578125f, + 0.023681640625f, -0.400390625f, -0.1787109375f, 0.5390625f, -0.54296875f, -0.22265625f, + -0.1064453125f, -0.34375f, 0.5390625f, -0.10400390625f, 0.1767578125f, -0.306640625f, + -0.455078125f, 0.5f, -0.294921875f, -0.259765625f, 0.02294921875f, 0.271484375f, + -0.1396484375f, 0.84765625f, 1.3984375f, 0.71875f, 0.2470703125f, 0.267578125f, + -0.1474609375f, 0.11181640625f, -0.74609375f, 0.5546875f, -0.98828125f, -0.388671875f, + -0.02783203125f, 0.6171875f, 0.4453125f, 0.05615234375f, 0.2373046875f, 0.62109375f, + -0.45703125f, 0.1162109375f, -1.125f, -0.265625f, 0.1640625f, -0.9453125f, -0.33984375f, + -0.232421875f, -0.62109375f, -0.5390625f, 0.392578125f, 0.498046875f, -0.044189453125f, + -0.255859375f, 0.07568359375f, -0.33203125f, 0.35546875f, -1.3359375f, 0.2041015625f, + -0.1962890625f, -0.96875f, -0.458984375f, 0.111328125f, 0.98828125f, 0.8046875f, + -0.37109375f, -0.376953125f, -0.345703125f, -0.9375f, -0.5546875f, -0.2333984375f, + -0.78125f, 1.3671875f, -0.7421875f, -0.515625f, -0.0595703125f, 1.140625f, -0.4609375f, + 0.76171875f, -0.15625f, -0.2080078125f, -0.349609375f, -0.6796875f, 0.59375f, + -0.1806640625f, -0.328125f, -0.640625f, 0.1396484375f, 0.86328125f, -0.05712890625f, + 1.4453125f, 0.055908203125f, 0.74609375f, 0.203125f, 0.2431640625f, -0.216796875f, + -0.1982421875f, 1.109375f, -0.33203125f, -0.40625f, 0.458984375f, 0.7109375f, + -0.435546875f, -0.234375f, 0.9921875f, -1.2734375f, -0.2490234375f, 0.451171875f, + -0.384765625f, 1.171875f, 0.302734375f, -0.361328125f, 0.439453125f, 0.46875f, + 0.1884765625f, -0.21875f, -0.4921875f, -0.2412109375f, -0.80859375f, -1.515625f, + -0.56640625f, 0.6484375f, -0.87890625f, -0.98046875f, -0.234375f, 1.5546875f, -0.34765625f, + 0.287109375f, -0.83984375f, -0.9921875f, -0.83984375f, -0.031982421875f, 0.060302734375f, + -0.28125f, 0.361328125f, -0.2353515625f, 0.5625f, -1.1796875f, 0.0615234375f, + -0.287109375f, 0.88671875f, -0.375f, 0.43359375f, 0.8671875f, 0.77734375f, 0.474609375f, + -1.0f, 0.6796875f, -0.32421875f, 0.1767578125f, -0.61328125f, -0.318359375f, -1.34375f, + 0.361328125f, -0.671875f, 0.263671875f, 0.0024566650390625f, 0.0927734375f, 0.326171875f, + -0.66796875f, 0.208984375f, 2.234375f, -0.63671875f, -0.287109375f, 1.3828125f, 0.671875f, + -0.4375f, 0.6796875f, -0.95703125f, 0.6640625f, -0.8671875f, -0.44921875f, 0.8828125f, + 1.046875f, 0.2001953125f, -0.220703125f, 0.30078125f, -0.287109375f, -0.31640625f, + -0.09228515625f, 0.609375f, 0.051025390625f, 0.10595703125f, 0.021484375f, -0.439453125f, + -0.220703125f, 0.059814453125f, -1.0390625f, -0.341796875f, 0.62109375f, -1.3046875f, + 0.12109375f, -1.0f, -0.87890625f, -0.333984375f, -0.83984375f, -1.0703125f, -1.765625f, + 0.26953125f, -0.79296875f, -0.287109375f, 0.2578125f, -1.078125f, 0.1748046875f, 1.15625f, + -1.25f, 0.140625f, -0.58203125f, -0.76953125f, -0.1806640625f, -0.5546875f, 0.8671875f, + 0.7109375f, 0.267578125f, -0.031982421875f, 0.443359375f, 0.443359375f, 0.050048828125f, + 0.447265625f, 0.09375f, 0.00445556640625f, -0.0810546875f, -0.08642578125f, 0.326171875f, + -0.439453125f, 0.99609375f, 0.138671875f, -0.138671875f, 0.83984375f, -0.8671875f, + 0.53515625f, 0.2373046875f, -0.5546875f, -0.427734375f, -0.51953125f, -0.48828125f, + 0.1435546875f, 0.171875f, 0.287109375f, -0.828125f, 0.73828125f, -0.53515625f, -1.0f, + -0.68359375f, 0.08154296875f, 0.703125f, 0.73046875f, -0.359375f, 0.0186767578125f, + 1.0546875f, 1.515625f, -0.255859375f, 1.2109375f, 0.416015625f, 0.1396484375f, 0.62109375f, + 0.431640625f, -0.51171875f, -0.25f, 0.9921875f, 0.65625f, 1.234375f, -0.173828125f, + 0.48046875f, -0.01446533203125f, -0.4453125f, 0.15625f, 0.28125f, 0.5546875f, + -0.294921875f, -0.40625f, -0.296875f, -0.56640625f, 0.5859375f, 0.1494140625f, + -0.80078125f, -0.349609375f, 0.42578125f, 0.306640625f, -0.02685546875f, -1.0078125f, + 0.380859375f, 0.30078125f, 0.75f, -0.404296875f, 0.419921875f, -0.00188446044921875f, + -0.0927734375f, -0.404296875f, 0.62890625f, -0.875f, 0.51953125f, 0.08447265625f, + 0.00799560546875f, 0.1337890625f, 0.302734375f, 0.16796875f, -0.427734375f, 0.88671875f, + -0.1650390625f, 0.74609375f, -0.8359375f, 1.46875f, -0.1162109375f, -0.69921875f, + 0.052734375f, 0.25390625f, -0.5859375f, 0.423828125f, 0.421875f, 1.3046875f, -0.56640625f, + -0.3828125f, -0.1201171875f, -0.0712890625f, 0.1220703125f, -0.0150146484375f, -0.421875f, + 0.14453125f, -0.1875f, 0.23828125f, -0.78515625f, -0.82421875f, -0.1337890625f, 1.1953125f, + -0.2431640625f, 0.4921875f, -0.54296875f, 0.053466796875f, 0.15234375f, -0.4140625f, + 0.0224609375f, 1.2265625f, 1.640625f, 1.0078125f, 0.4140625f, 0.357421875f, -0.70703125f, + -0.28515625f, 0.609375f, -0.55859375f, -0.6328125f, 1.7109375f, 1.2578125f, -0.333984375f, + 0.984375f, 0.9140625f, 0.82421875f, 0.1650390625f, 0.69921875f, 0.07763671875f, -0.21875f, + -0.3984375f, -0.5859375f, -0.45703125f, -0.765625f, 0.020751953125f, -0.58203125f, + -0.0024261474609375f, -0.79296875f, -0.326171875f, 0.36328125f, 0.216796875f, 0.43359375f, + -0.71484375f, -0.3359375f, -0.703125f, -0.1748046875f, -0.94921875f, -0.380859375f, + 0.59375f, -0.09521484375f, -0.021728515625f, -0.396484375f, -1.046875f, 1.0234375f, + 0.2109375f, 0.474609375f, 0.2109375f, 0.07861328125f, -0.640625f, -0.53125f, 0.27734375f, + 0.1044921875f, 0.05224609375f, 0.76171875f, -0.27734375f, -0.953125f, -0.8046875f, + 0.427734375f, 0.50390625f, -0.380859375f, 0.83203125f, 0.04443359375f, 0.056396484375f, + 0.53515625f, -0.1982421875f, -0.275390625f, -0.2177734375f, -0.0361328125f, 0.4453125f, + 0.1982421875f, 0.240234375f, -0.185546875f, -0.032958984375f, -0.396484375f, -0.392578125f, + -0.26171875f, 0.2431640625f, 0.44921875f, -0.00775146484375f, -0.54296875f, 0.5390625f, + -0.171875f, 0.158203125f, 0.0211181640625f, -0.47265625f, 1.8359375f, -0.05322265625f, + -0.69140625f, -0.109375f, -0.0947265625f, -0.11083984375f, -0.84375f, -0.80859375f, + 0.234375f, -0.0751953125f, 0.7890625f, -0.10107421875f, 0.078125f, 0.462890625f, 0.75f, + 0.89453125f, 0.85546875f, -1.9296875f, 1.375f, -0.96484375f, 0.33203125f, 0.6328125f, + 0.66015625f, -0.67578125f, 1.25f, 0.376953125f, -0.90625f, 1.0703125f, 0.470703125f, + 0.31640625f, 0.048828125f, -0.6015625f, -0.3828125f, -0.34375f, -0.73046875f, 0.302734375f, + 0.19921875f, 0.052001953125f, 0.0693359375f, -0.8984375f, 0.10693359375f, -0.953125f, + -0.7421875f, 0.5546875f, -0.09912109375f, 0.470703125f, -0.06494140625f, -1.0703125f, + -0.05908203125f, 0.6953125f, 0.8671875f, 0.5703125f, 1.4296875f, -0.1484375f, 0.9921875f, + -0.0155029296875f, -0.8671875f, 0.1962890625f, -0.470703125f, 0.06005859375f, 0.404296875f, + 0.146484375f, -0.7734375f, -0.1494140625f, 0.7578125f, -0.0576171875f, -0.28515625f, + 0.1689453125f, -0.2060546875f, 0.2333984375f, -0.90625f, 0.19921875f, 0.1279296875f, + 0.3984375f, 0.68359375f, 0.37109375f, -0.357421875f, -0.00994873046875f, -0.045166015625f, + 0.44921875f, 1.1875f, 0.0400390625f, -1.0703125f, -0.388671875f, -0.490234375f, + 0.1044921875f, -0.2275390625f, -0.291015625f, 1.0390625f, -0.12890625f, -0.6640625f, + 1.515625f, 0.455078125f, 0.482421875f, -0.96484375f, -0.3203125f, -0.546875f, + 0.09912109375f, 0.275390625f, 1.28125f, 0.56640625f, 0.04931640625f, 0.09716796875f, + -0.0517578125f, -0.265625f, -0.46875f, -0.0233154296875f, -0.39453125f, -0.296875f, + 0.018798828125f, 0.08740234375f, -0.82421875f, 0.37109375f, 0.06982421875f, 0.439453125f, + -1.125f, -0.16015625f, 0.3125f, -0.193359375f, -0.88671875f, -0.984375f, -0.75390625f, + 0.6015625f, -0.1953125f, -0.040771484375f, -0.7265625f, 1.0703125f, -0.7578125f, + -0.294921875f, -0.0234375f, -0.419921875f, 0.66015625f, -0.1552734375f, 1.015625f, + -0.91796875f, -1.1796875f, 1.03125f, 0.072265625f, 0.30859375f, 0.515625f, 0.5078125f, + -0.458984375f, 0.5859375f, -0.25f, -0.111328125f, -0.26953125f, -0.48046875f, -0.9375f, + -0.279296875f, 0.0634765625f, 0.275390625f, 0.7265625f, 1.0546875f, 0.84765625f, + -0.322265625f, 2.0625f, -0.287109375f, 0.1396484375f, 0.1826171875f, 0.8203125f, + 0.2099609375f, 0.0693359375f, -0.032958984375f, 0.890625f, 0.1875f, 0.271484375f, + -0.498046875f, -0.0194091796875f, 0.26171875f, 0.34375f, 0.22265625f, -0.5859375f, + -0.2421875f, -0.59375f, -0.099609375f, -0.40234375f, -0.1689453125f, -0.5078125f, + 0.1318359375f, 0.20703125f, 0.65234375f, 0.359375f, -0.154296875f, 0.1630859375f, + 0.458984375f, 0.1962890625f, -1.6953125f, -0.45703125f, -0.0186767578125f, -0.8359375f, + 0.298828125f, -0.828125f, 0.77734375f, 0.1357421875f, -0.73828125f, 0.90234375f, + 0.56640625f, -1.2109375f, -0.498046875f, -0.361328125f, -0.1484375f, -0.28515625f, + -0.439453125f, -0.1884765625f, 0.25f, -0.3046875f, -0.8125f, -0.37890625f, -0.404296875f, + 0.259765625f, -0.5546875f, 0.1943359375f, 0.0830078125f, -0.29296875f, 0.08740234375f, + 0.57421875f, 0.06884765625f, 0.5625f, -0.6953125f, 0.123046875f, 0.7109375f, + -0.0203857421875f, -0.55078125f, -0.322265625f, 0.6171875f, -0.0966796875f, 0.4765625f, + -0.48828125f, 0.07666015625f, -0.59375f, -0.60546875f, 1.09375f, -1.0625f, -1.03125f, + -0.36328125f, -0.46484375f, -0.0400390625f, -0.09228515625f, -0.287109375f, -0.126953125f, + 0.392578125f, -0.1357421875f, 0.1591796875f, -0.8984375f, -0.1435546875f, 0.515625f, + -0.4453125f, 0.1484375f, 0.7578125f, -0.01373291015625f, 0.3125f, -0.5546875f, + -0.318359375f, 0.201171875f, -0.453125f, -0.91015625f, -0.298828125f, -0.31640625f, + 0.443359375f, -0.3515625f, 0.1630859375f, -0.369140625f, -0.51953125f, -0.12060546875f, + 0.1259765625f, 0.1201171875f, 0.515625f, -0.53515625f, 1.390625f, 0.5390625f, + 0.07080078125f, 0.68359375f, -0.01904296875f, -0.6171875f, 0.453125f, 0.44921875f, + 0.1953125f, 0.13671875f, 0.240234375f, 0.5546875f, 0.052978515625f, -0.73046875f, + -0.80078125f, -0.349609375f, 0.138671875f, -0.68359375f, -0.53515625f, -0.7421875f, + 0.41015625f, 1.2421875f, -0.458984375f, -0.357421875f, -0.86328125f, 0.0810546875f, + 0.58984375f, 0.2578125f, -1.0703125f, 0.77734375f, -0.486328125f, -0.3671875f, -0.8125f, + -0.984375f, -1.25f, -0.166015625f, -0.205078125f, 0.478515625f, -0.671875f, -1.1796875f, + 0.1669921875f, -0.361328125f, 0.1767578125f, 0.44140625f, -0.1181640625f, 0.080078125f, + 0.004791259765625f, -1.4296875f, 0.462890625f, 0.59765625f, 0.447265625f, + -0.005462646484375f, -0.37109375f, -0.091796875f, 0.240234375f, -0.84375f, 0.435546875f, + 0.375f, -0.66796875f, -0.59375f, 0.9375f, 0.60546875f, 0.20703125f, -0.7890625f, + 0.0634765625f, -0.6875f, 0.314453125f, -0.53515625f, 0.49609375f, 0.318359375f, + 0.0400390625f, 0.004425048828125f, -0.01025390625f, -1.109375f, 0.3828125f, 0.8984375f, + -0.44140625f, -0.87890625f, -0.51953125f, 0.169921875f, -0.34375f, 0.33984375f, 0.375f, + -0.1728515625f, -0.078125f, -0.52734375f, 0.0269775390625f, -0.244140625f, 1.3125f, + 0.06298828125f, -0.103515625f, 0.5078125f, 0.515625f, -0.64453125f, 0.2236328125f, + 0.671875f, 0.259765625f, -0.0888671875f, 0.392578125f, 0.078125f, -0.61328125f, + -0.10302734375f, -0.06494140625f, 0.1279296875f, -0.75f, -0.80859375f, -0.396484375f, + 0.036376953125f, -0.09619140625f, 0.09423828125f, 0.02978515625f, 0.9375f, 0.361328125f, + 0.71484375f, -0.1357421875f, -0.47265625f, 0.12890625f, 0.10400390625f, -0.73828125f, + -0.3359375f, 0.138671875f, 0.44921875f, 0.0732421875f, 0.057861328125f, -0.177734375f, + 0.27734375f, 0.4453125f, 0.28125f, -0.00860595703125f, 0.1943359375f, -0.466796875f, + 0.3671875f, -0.16015625f, -0.30859375f, -0.7109375f, 0.265625f, -0.91796875f, + -0.166015625f, 0.244140625f, -0.96875f, -0.06884765625f, 0.216796875f, 0.373046875f, + 0.66796875f, 0.89453125f, -0.396484375f, 0.010986328125f, 0.007171630859375f, + -0.09814453125f, -1.03125f, 0.482421875f, 0.0859375f, -0.609375f, 1.2109375f, + 0.0189208984375f, -0.26953125f, 0.60546875f, 0.0869140625f, -0.30859375f, -0.43359375f, + 0.3671875f, -0.021728515625f, 0.4921875f, 0.193359375f, -0.0947265625f, 0.390625f, + 0.0380859375f, -0.259765625f, 1.109375f, -0.447265625f, -1.0546875f, -0.123046875f, + 0.10791015625f, 0.076171875f, 0.458984375f, 1.0234375f, -0.2265625f, 1.0625f, -0.7578125f, + -0.1318359375f, 0.1064453125f, -0.02392578125f, -1.2421875f, -0.58203125f, -0.384765625f, + 0.0186767578125f, -0.482421875f, 0.0225830078125f, -0.1337890625f, -0.2353515625f, + -0.1923828125f, 0.330078125f, 0.337890625f, -0.671875f, 0.0014495849609375f, 0.2255859375f, + 0.31640625f, 0.6484375f, -0.042724609375f, 0.94921875f, 0.0007171630859375f, -0.451171875f, + -0.0308837890625f, 0.51953125f, -0.053466796875f, 0.0732421875f, 0.091796875f, + -0.361328125f, -0.09326171875f, 0.00848388671875f, 0.060791015625f, 0.1962890625f, + 0.2138671875f, -0.099609375f, 0.427734375f, 0.71484375f, 0.78515625f, -0.09375f, 0.84375f, + 0.80078125f, -0.10888671875f, -0.330078125f, 0.25f, -0.05029296875f, 0.271484375f, + -0.82421875f, 0.052490234375f, -0.1435546875f, 0.306640625f, 0.56640625f, -0.62109375f, + -0.60546875f, 0.14453125f, -0.259765625f, 0.51953125f, -1.109375f, 0.0001678466796875f, + -0.98046875f, 0.87109375f, -0.26171875f, -0.189453125f, -0.39453125f, 0.125f, 0.57421875f, + 0.060546875f, 0.98828125f, -0.64453125f, 0.27734375f, 0.031982421875f, 0.2060546875f, + -0.28515625f, -0.0576171875f, -0.296875f, 0.6953125f, 0.9296875f, 0.4140625f, -1.0546875f, + 0.275390625f, -0.46484375f, -0.023193359375f, -0.453125f, 0.2041015625f, -0.197265625f, + -0.0189208984375f, -0.443359375f, 0.34375f, 0.57421875f, -0.373046875f, 0.388671875f, + 0.392578125f, -0.3828125f, 0.5078125f, 0.10009765625f, -0.0849609375f, 0.73828125f, + -0.33984375f, 0.291015625f, -0.173828125f, 0.01531982421875f, 1.0234375f, -0.31640625f, + 0.4453125f, 0.328125f, 0.7265625f, 0.3984375f, -0.0888671875f, 0.349609375f, 0.55078125f, + 0.6484375f, 0.52734375f, -0.043212890625f, 0.2060546875f, -0.0286865234375f, + 0.0079345703125f, 0.81640625f, 0.10498046875f, 0.458984375f, 0.2216796875f, -1.1484375f, + -0.326171875f, -0.0206298828125f, -0.07421875f, -0.48828125f, 0.9921875f, 0.1962890625f, + 0.57421875f, 0.166015625f, -0.2412109375f, 0.5625f, 0.94921875f, -0.2109375f, 0.337890625f, + 0.447265625f, 0.279296875f, -0.34765625f, 0.423828125f, -0.337890625f, -0.890625f, + 0.41796875f, 0.484375f, 0.58984375f, 0.80859375f, -0.63671875f, -0.32421875f, + -0.306640625f, 0.3671875f, -0.546875f, -0.400390625f, 0.60546875f, -0.82421875f, + 0.5703125f, -0.056396484375f, 0.1943359375f, 0.298828125f, -0.2236328125f, -0.796875f, + 0.671875f, 0.92578125f, 0.11572265625f, -0.0380859375f, 0.73046875f, -0.09521484375f, + -0.416015625f, -0.44140625f, -0.361328125f, 0.60546875f, -0.498046875f, 1.953125f, + 0.08740234375f, 0.318359375f, 0.027099609375f, 0.79296875f, 0.0791015625f, 0.4140625f, + 0.8125f, -0.609375f, 0.373046875f, -0.62890625f, 0.40625f, -0.3671875f, -0.1962890625f, + -0.271484375f, -0.2431640625f, 0.478515625f, -0.08740234375f, 0.8359375f, -0.75390625f, + 0.11279296875f, 0.8984375f, -0.373046875f, 0.07421875f, -0.96484375f, 0.43359375f, + 0.240234375f, 0.42578125f, 0.1591796875f, -0.96875f, 0.1865234375f, 0.392578125f, + -0.0556640625f, -0.283203125f, -0.341796875f, -1.3515625f, -0.1044921875f, + -0.0191650390625f, 0.9375f, 0.451171875f, -0.33984375f, -0.00067138671875f, 0.353515625f, + -0.10400390625f, -0.02392578125f, 0.0556640625f, 1.2734375f, -0.61328125f, 1.109375f, + -0.041015625f, 0.05615234375f, 0.75f, -0.423828125f, -0.74609375f, 0.640625f, + 0.1083984375f, -0.87890625f, 0.421875f, -0.439453125f, 0.306640625f, 0.58203125f, + 0.74609375f, 0.6640625f, 0.7109375f, -0.58203125f, -0.34375f, 0.90234375f, -0.328125f, + 0.40234375f, 0.21484375f, -1.1171875f, -0.037109375f, 0.1923828125f, -0.396484375f, + 0.059326171875f, -0.44140625f, 0.53125f, 0.494140625f, 0.67578125f, 1.1875f, -0.416015625f, + 0.462890625f, -0.380859375f, 1.453125f, 0.4140625f, -0.58203125f, -0.07763671875f, + 0.130859375f, 0.41796875f +}; +inline constexpr float kOverBudgetKRaw[] = { + 0.21875f, 0.1767578125f, -0.76953125f, -0.0693359375f, 0.220703125f, -1.015625f, + 0.11865234375f, 0.2734375f, -1.3984375f, -0.330078125f, -0.625f, 0.62890625f, 0.5f, + -0.376953125f, -0.734375f, -0.98046875f, 0.62890625f, -0.7265625f, -0.013671875f, + -0.5078125f, -0.0029296875f, -0.455078125f, 0.2060546875f, -0.255859375f, -1.0234375f, + 0.054443359375f, 0.8671875f, -0.578125f, -0.048095703125f, -0.404296875f, 0.1953125f, + 0.365234375f, 0.037353515625f, 0.80859375f, 0.232421875f, -0.34375f, 0.53515625f, + 0.5234375f, -0.76171875f, 0.15625f, 0.93359375f, 0.6328125f, 0.2373046875f, -0.1826171875f, + -0.1904296875f, -0.42578125f, 0.455078125f, 0.92578125f, 0.62890625f, -0.99609375f, + 0.14453125f, 0.373046875f, 1.0390625f, 0.2080078125f, -1.3984375f, 0.064453125f, + 0.38671875f, -1.1953125f, 0.193359375f, 0.1396484375f, -0.55859375f, 0.94921875f, + -0.3359375f, -0.80078125f, -0.1318359375f, 0.330078125f, 0.203125f, 0.1494140625f, + -0.36328125f, 0.1708984375f, 0.734375f, -0.2890625f, 0.51953125f, 0.640625f, 0.1181640625f, + -0.345703125f, -0.2119140625f, -0.376953125f, 0.61328125f, 0.3671875f, 0.50390625f, + -0.248046875f, -0.185546875f, 0.5859375f, 0.53515625f, 0.765625f, -0.828125f, 0.80078125f, + -0.2294921875f, -0.3203125f, -0.56640625f, 1.2421875f, -0.65625f, 0.55078125f, + 0.275390625f, 0.0693359375f, -0.50390625f, 0.609375f, -0.640625f, -0.7421875f, 1.046875f, + -0.609375f, 0.609375f, -0.080078125f, -0.1259765625f, 0.498046875f, -0.2392578125f, + 0.25390625f, -0.0810546875f, 0.232421875f, 0.33203125f, 0.047607421875f, -0.7265625f, + 0.203125f, -0.55078125f, -0.55078125f, -0.61328125f, -0.765625f, 1.1796875f, 1.0703125f, + -1.4453125f, -0.67578125f, -0.02294921875f, -1.140625f, 0.3984375f, 0.2080078125f, + -0.2421875f, -0.859375f, -1.7578125f, -0.0179443359375f, 0.056396484375f, 0.212890625f, + 0.103515625f, -0.1806640625f, -0.62890625f, -0.5f, 0.1943359375f, -0.1298828125f, + -0.2041015625f, 0.259765625f, -0.40625f, 0.5703125f, 0.578125f, -0.322265625f, 0.3203125f, + 0.0908203125f, 0.0240478515625f, -0.140625f, -0.1884765625f, 0.8828125f, -0.2392578125f, + -0.37109375f, 0.201171875f, 0.734375f, 0.56640625f, 0.279296875f, -0.29296875f, + -0.49609375f, -1.3046875f, 0.6484375f, -0.15234375f, 0.19140625f, -0.14453125f, + -0.357421875f, 0.60546875f, 0.0732421875f, 0.142578125f, -0.443359375f, -0.29296875f, + 0.52734375f, 0.337890625f, -0.173828125f, -0.205078125f, 0.0634765625f, -0.427734375f, + -0.6953125f, 0.478515625f, -0.039794921875f, -0.3046875f, 0.05712890625f, -0.0380859375f, + -0.263671875f, -0.041748046875f, 0.1552734375f, -0.34375f, -1.2890625f, 0.2451171875f, + 0.578125f, -0.01300048828125f, 0.44140625f, -0.341796875f, 0.91796875f, -0.84375f, + 1.3203125f, -0.203125f, 1.25f, -0.2890625f, 0.8828125f, 0.059326171875f, 0.0908203125f, + -0.62890625f, -0.04345703125f, 1.140625f, 0.734375f, -0.52734375f, -0.953125f, + 0.0184326171875f, 0.00136566162109375f, 0.7734375f, 0.396484375f, 0.90625f, 0.109375f, + -0.404296875f, -0.451171875f, 0.30859375f, 0.306640625f, 0.16015625f, -0.96875f, + 0.5078125f, 0.609375f, 0.28125f, -0.0849609375f, 0.060791015625f, -0.58203125f, 0.421875f, + 1.4140625f, 0.11572265625f, 0.73046875f, -0.0771484375f, -0.0654296875f, -0.75f, + -1.3046875f, 0.271484375f, 0.1064453125f, -0.1455078125f, 1.203125f, 0.08447265625f, + -0.0986328125f, -0.796875f, 0.83203125f, -0.65234375f, -0.3203125f, -0.75390625f, + -0.78515625f, -0.8125f, -0.0240478515625f, 0.5390625f, 0.134765625f, -0.291015625f, + -0.404296875f, -0.96875f, 0.345703125f, 0.7265625f, 0.458984375f, 0.087890625f, + -0.1962890625f, 0.412109375f, -0.1982421875f, 0.36328125f, 0.65625f, 0.10302734375f, + 0.357421875f, -0.875f, -0.251953125f, -0.423828125f, 1.21875f, -0.7578125f, -0.224609375f, + -0.373046875f, 0.1630859375f, 0.7578125f, -0.5390625f, 0.890625f, -0.28125f, -0.4140625f, + 0.337890625f, 0.35546875f, -0.5f, 0.90625f, -0.1328125f, -0.75390625f, 0.15234375f, + 0.287109375f, -0.2431640625f, -0.66015625f, -0.470703125f, 0.27734375f, 0.26171875f, + -0.66015625f, 0.050537109375f, -0.671875f, -0.78125f, 0.00982666015625f, -0.236328125f, + 0.8125f, 0.58984375f, 0.1298828125f, -1.0546875f, 0.392578125f, -1.25f, 0.56640625f, + 0.181640625f, -0.21484375f, -0.09521484375f, 0.2412109375f, -0.09619140625f, 0.609375f, + 0.003692626953125f, 0.322265625f, -0.1376953125f, 0.43359375f, 0.019775390625f, + 0.59765625f, 0.19921875f, 0.1923828125f, -0.16796875f, 0.51953125f, -0.345703125f, + -0.00738525390625f, -0.138671875f, 0.466796875f, 0.2392578125f, -0.57421875f, + 0.055419921875f, 0.37109375f, -0.3984375f, 0.78125f, 0.30078125f, 0.8515625f, -1.1015625f, + 0.205078125f, -0.953125f, 0.2421875f, -0.59765625f, -1.328125f, 0.06689453125f, + 0.11083984375f, 0.421875f, -0.6171875f, -0.63671875f, 0.1611328125f, -0.6171875f, 1.1875f, + 0.09423828125f, -0.1640625f, -0.25390625f, -0.50390625f, 0.5390625f, 0.10986328125f, + 0.66796875f, -0.7421875f, 0.80078125f, 0.48828125f, -0.80859375f, 0.5859375f, 0.26171875f, + 0.28125f, 0.267578125f, -0.0810546875f, -0.9921875f, 0.70703125f, 0.0240478515625f, + -0.25390625f, -0.255859375f, -0.98046875f, -0.2275390625f, 0.20703125f, 0.41015625f +}; +inline constexpr float kOverBudgetCos[] = { + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 0.5390625f, 0.99609375f, 1.0f, 1.0f, + 0.5390625f, 0.99609375f, 1.0f, 1.0f, -0.416015625f, 0.98046875f, 1.0f, 1.0f, -0.416015625f, + 0.98046875f, 1.0f, 1.0f, -0.98828125f, 0.95703125f, 1.0f, 1.0f, -0.98828125f, 0.95703125f, + 1.0f, 1.0f, -0.65234375f, 0.921875f, 1.0f, 1.0f, -0.65234375f, 0.921875f, 1.0f, 1.0f, + 0.283203125f, 0.87890625f, 1.0f, 1.0f, 0.283203125f, 0.87890625f, 1.0f, 1.0f, 0.9609375f, + 0.82421875f, 1.0f, 1.0f, 0.9609375f, 0.82421875f, 1.0f, 1.0f, 0.75390625f, 0.765625f, + 0.99609375f, 1.0f, 0.75390625f, 0.765625f, 0.99609375f, 1.0f, -0.1455078125f, 0.6953125f, + 0.99609375f, 1.0f, -0.1455078125f, 0.6953125f, 0.99609375f, 1.0f, -0.91015625f, + 0.62109375f, 0.99609375f, 1.0f, -0.91015625f, 0.62109375f, 0.99609375f, 1.0f, -0.83984375f, + 0.5390625f, 0.99609375f, 1.0f, -0.83984375f, 0.5390625f, 0.99609375f, 1.0f, + 0.004425048828125f, 0.453125f, 0.9921875f, 1.0f, 0.004425048828125f, 0.453125f, 0.9921875f, + 1.0f, 0.84375f, 0.361328125f, 0.9921875f, 1.0f, 0.84375f, 0.361328125f, 0.9921875f, 1.0f, + 0.90625f, 0.265625f, 0.9921875f, 1.0f, 0.90625f, 0.265625f, 0.9921875f, 1.0f, 0.13671875f, + 0.1689453125f, 0.98828125f, 1.0f, 0.13671875f, 0.1689453125f, 0.98828125f, 1.0f, + -0.7578125f, 0.0693359375f, 0.98828125f, 1.0f, -0.7578125f, 0.0693359375f, 0.98828125f, + 1.0f, -0.95703125f, -0.03076171875f, 0.98828125f, 1.0f, -0.95703125f, -0.03076171875f, + 0.98828125f, 1.0f, -0.275390625f, -0.130859375f, 0.984375f, 1.0f, -0.275390625f, + -0.130859375f, 0.984375f, 1.0f, 0.66015625f, -0.228515625f, 0.984375f, 1.0f, 0.66015625f, + -0.228515625f, 0.984375f, 1.0f, 0.98828125f, -0.32421875f, 0.98046875f, 1.0f, 0.98828125f, + -0.32421875f, 0.98046875f, 1.0f, 0.408203125f, -0.41796875f, 0.98046875f, 1.0f, + 0.408203125f, -0.41796875f, 0.98046875f, 1.0f, -0.546875f, -0.5078125f, 0.9765625f, 1.0f, + -0.546875f, -0.5078125f, 0.9765625f, 1.0f, -1.0f, -0.58984375f, 0.9765625f, 1.0f, -1.0f, + -0.58984375f, 0.9765625f, 1.0f +}; +inline constexpr float kOverBudgetSin[] = { + 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.83984375f, 0.10009765625f, + 0.010009765625f, 0.00099945068359375f, 0.83984375f, 0.10009765625f, 0.010009765625f, + 0.00099945068359375f, 0.91015625f, 0.19921875f, 0.02001953125f, 0.0019989013671875f, + 0.91015625f, 0.19921875f, 0.02001953125f, 0.0019989013671875f, 0.1416015625f, 0.294921875f, + 0.030029296875f, 0.00299072265625f, 0.1416015625f, 0.294921875f, 0.030029296875f, + 0.00299072265625f, -0.7578125f, 0.390625f, 0.0400390625f, 0.003997802734375f, -0.7578125f, + 0.390625f, 0.0400390625f, 0.003997802734375f, -0.95703125f, 0.48046875f, 0.050048828125f, + 0.0050048828125f, -0.95703125f, 0.48046875f, 0.050048828125f, 0.0050048828125f, + -0.279296875f, 0.56640625f, 0.06005859375f, 0.0059814453125f, -0.279296875f, 0.56640625f, + 0.06005859375f, 0.0059814453125f, 0.65625f, 0.64453125f, 0.06982421875f, + 0.006988525390625f, 0.65625f, 0.64453125f, 0.06982421875f, 0.006988525390625f, 0.98828125f, + 0.71875f, 0.080078125f, 0.00799560546875f, 0.98828125f, 0.71875f, 0.080078125f, + 0.00799560546875f, 0.412109375f, 0.78515625f, 0.08984375f, 0.00897216796875f, 0.412109375f, + 0.78515625f, 0.08984375f, 0.00897216796875f, -0.54296875f, 0.84375f, 0.10009765625f, + 0.010009765625f, -0.54296875f, 0.84375f, 0.10009765625f, 0.010009765625f, -1.0f, 0.890625f, + 0.10986328125f, 0.010986328125f, -1.0f, 0.890625f, 0.10986328125f, 0.010986328125f, + -0.53515625f, 0.93359375f, 0.11962890625f, 0.011962890625f, -0.53515625f, 0.93359375f, + 0.11962890625f, 0.011962890625f, 0.419921875f, 0.96484375f, 0.1298828125f, + 0.01300048828125f, 0.419921875f, 0.96484375f, 0.1298828125f, 0.01300048828125f, 0.9921875f, + 0.984375f, 0.1396484375f, 0.01397705078125f, 0.9921875f, 0.984375f, 0.1396484375f, + 0.01397705078125f, 0.6484375f, 0.99609375f, 0.1494140625f, 0.0150146484375f, 0.6484375f, + 0.99609375f, 0.1494140625f, 0.0150146484375f, -0.287109375f, 1.0f, 0.1591796875f, + 0.0159912109375f, -0.287109375f, 1.0f, 0.1591796875f, 0.0159912109375f, -0.9609375f, + 0.9921875f, 0.1689453125f, 0.0169677734375f, -0.9609375f, 0.9921875f, 0.1689453125f, + 0.0169677734375f, -0.75f, 0.97265625f, 0.1796875f, 0.0179443359375f, -0.75f, 0.97265625f, + 0.1796875f, 0.0179443359375f, 0.1494140625f, 0.9453125f, 0.189453125f, 0.01904296875f, + 0.1494140625f, 0.9453125f, 0.189453125f, 0.01904296875f, 0.9140625f, 0.91015625f, + 0.19921875f, 0.02001953125f, 0.9140625f, 0.91015625f, 0.19921875f, 0.02001953125f, + 0.8359375f, 0.86328125f, 0.208984375f, 0.02099609375f, 0.8359375f, 0.86328125f, + 0.208984375f, 0.02099609375f, -0.00885009765625f, 0.80859375f, 0.21875f, 0.02197265625f, + -0.00885009765625f, 0.80859375f, 0.21875f, 0.02197265625f +}; +inline constexpr float kOverBudgetQNormW[] = { + 0.3671875f, 0.1845703125f, 0.5f, 0.515625f, -0.296875f, -0.54296875f, 0.0240478515625f, + -0.8125f, 0.447265625f, -1.125f, -0.032470703125f, -0.9296875f, -0.041748046875f, + 0.06884765625f, -0.2392578125f, -0.1796875f +}; +inline constexpr float kOverBudgetKNormW[] = { + 1.0234375f, -0.32421875f, -0.279296875f, -0.75f, 0.251953125f, 0.69921875f, -0.45703125f, + 1.5234375f, 0.71875f, -0.265625f, -1.53125f, 0.439453125f, -0.181640625f, -1.171875f, + 0.64453125f, 0.67578125f +}; +inline constexpr float kOverBudgetQPost[] = { + -0.51953125f, 0.4609375f, -1.0f, 1.671875f, -1.25f, 0.462890625f, -0.73828125f, -0.125f, + 0.734375f, 0.1845703125f, -0.1494140625f, 0.00946044921875f, 0.73046875f, -0.0693359375f, + -0.51953125f, 1.8984375f, -0.72265625f, -1.21875f, -0.318359375f, 0.019287109375f, + 0.85546875f, -0.59765625f, -0.2314453125f, -0.234375f, 1.734375f, 0.18359375f, -1.5546875f, + 0.05908203125f, 0.6484375f, 1.4609375f, -0.41015625f, -0.484375f, -0.73046875f, + -0.55859375f, -1.7734375f, -1.5f, 0.294921875f, 0.9375f, -0.9140625f, -0.04248046875f, + -0.64453125f, -0.052734375f, 0.5234375f, 0.1318359375f, -0.75f, 0.0019378662109375f, + 0.796875f, -1.1640625f, 1.390625f, -1.0078125f, -0.38671875f, 1.5f, 0.412109375f, + -0.65234375f, -0.234375f, -0.2021484375f, -1.5234375f, 0.1533203125f, -1.0078125f, + 0.01416015625f, 1.3984375f, 1.9140625f, -0.42578125f, -0.361328125f, -0.435546875f, + -0.58984375f, 2.59375f, 0.00086212158203125f, -0.71875f, -0.490234375f, 1.21875f, + 0.2294921875f, 0.2392578125f, -0.2890625f, 0.5625f, -0.0093994140625f, 0.0306396484375f, + 0.2333984375f, 1.234375f, -0.29296875f, 0.45703125f, -0.8828125f, -1.4296875f, -1.46875f, + 2.21875f, 0.130859375f, -0.326171875f, 0.00982666015625f, 0.5546875f, 0.06982421875f, + 0.466796875f, 0.158203125f, 0.7421875f, -0.8984375f, -1.1953125f, -0.111328125f, + 1.4453125f, -1.578125f, -0.267578125f, 3.78125f, 1.375f, -0.5f, 0.140625f, -0.240234375f, + -0.142578125f, -0.15625f, 1.0f, -0.01080322265625f, 0.62109375f, 0.068359375f, + 0.003997802734375f, 0.1865234375f, 1.15625f, 1.4375f, -0.859375f, 1.921875f, 0.240234375f, + 0.95703125f, -1.4375f, -0.11865234375f, -1.3671875f, -0.0556640625f, -0.107421875f, + 0.044189453125f, -1.328125f, 1.078125f, 0.5703125f, 0.265625f, 0.83203125f, 1.6875f, + 0.57421875f, -1.1484375f, -0.23828125f, 1.15625f, -0.049560546875f, 0.0654296875f, + -1.765625f, 0.095703125f, 0.3203125f, -0.050048828125f, -0.59375f, 0.1953125f, + 0.2353515625f, 1.8984375f, 0.080078125f, -1.8828125f, 0.052734375f, -2.234375f, -1.453125f, + 0.337890625f, 1.6328125f, 0.0927734375f, 1.171875f, -0.1259765625f, -0.1259765625f, + -0.0498046875f, -0.2158203125f, 1.7109375f, -0.03759765625f, -0.10302734375f, -0.92578125f, + 0.484375f, 3.09375f, -0.1552734375f, 1.15625f, -0.068359375f, 0.58203125f, -0.1953125f, + 1.0234375f, -0.06640625f, 0.53125f, -0.029052734375f, -1.9296875f, -1.2578125f, + -0.34765625f, -1.09375f, -0.55859375f, 2.1875f, 1.703125f, 0.248046875f, 1.5625f, + 0.2578125f, 1.4375f, -0.126953125f, -0.51171875f, 0.22265625f, 0.93359375f, + -0.000957489013671875f, 0.89453125f, -1.2265625f, 0.2353515625f, -0.66796875f, -0.921875f, + 0.75f, -2.640625f, 0.99609375f, 0.953125f, -0.484375f, -0.55859375f, 0.0966796875f, + -0.85546875f, -0.1796875f, -1.6640625f, 0.046630859375f, -1.078125f, 0.70703125f, + 0.0673828125f, 0.1455078125f, 0.609375f, 1.7578125f, -0.70703125f, 1.3671875f, 0.22265625f, + 0.7109375f, -0.1484375f, 0.03369140625f, 0.74609375f, -0.1708984375f, 1.125f, + 0.034912109375f, -2.09375f, 1.3046875f, -0.48046875f, -1.0f, -0.359375f, 0.5f, -3.609375f, + 0.375f, -0.6953125f, 0.609375f, 0.63671875f, -0.1455078125f, 0.7421875f, 0.038818359375f, + 0.8515625f, 0.10986328125f, 0.59375f, 0.76953125f, -0.96875f, -0.640625f, -1.3125f, + 0.04296875f, -1.7734375f, 1.0078125f, 0.1787109375f, 0.515625f, 1.296875f, 0.3125f, + 2.03125f, 0.0947265625f, 1.171875f, -0.017578125f, 0.6328125f, -0.1728515625f, 0.90234375f, + -1.0859375f, -0.328125f, -0.80078125f, 1.5234375f, -3.984375f, -0.546875f, 0.6171875f, + -0.82421875f, -0.12890625f, -1.0234375f, 0.052978515625f, 0.341796875f, -0.0517578125f, + -0.640625f, -0.76171875f, -0.458984375f, -0.3671875f, -0.275390625f, -1.765625f, + 1.2109375f, -0.177734375f, -0.53515625f, 0.1015625f, 0.068359375f, -0.2109375f, -2.46875f, + 0.10595703125f, 0.78515625f, -0.11767578125f, -0.7265625f, -0.11279296875f, 0.404296875f, + 0.87890625f, 0.890625f, -0.65625f, 0.16796875f, 1.203125f, 0.48828125f, -0.46484375f, + 0.3828125f, -0.1650390625f, 2.953125f, 0.0225830078125f, 1.953125f, -0.04638671875f, + 1.8125f, -0.68359375f, 0.39453125f, 0.2275390625f, 0.7265625f, -0.84765625f, -0.53515625f, + -0.05859375f, 0.76953125f, -0.478515625f, 1.4921875f, -0.134765625f, -0.435546875f, + 0.1015625f, 1.59375f, -0.1142578125f, 0.4375f, -0.96484375f, 0.06494140625f, 1.7890625f, + 1.6015625f, 0.98046875f, -1.6484375f, -0.400390625f, -2.59375f, -0.20703125f, -0.9921875f, + 0.212890625f, -0.6484375f, 0.09765625f, -0.30078125f, 0.03271484375f, -0.111328125f, + 1.4296875f, -0.51953125f, 0.71875f, 0.197265625f, 0.30078125f, -1.0078125f, 2.953125f, + 1.3984375f, 0.427734375f, 0.474609375f, -0.0673828125f, 1.375f, -0.1025390625f, + -0.2119140625f, -0.0634765625f, -0.0556640625f, 1.625f, -1.3125f, -1.0390625f, -1.0625f, + -0.1005859375f, -3.09375f, 0.0966796875f, 0.9453125f, -0.30078125f, 1.3125f, -0.26953125f, + -0.8515625f, 0.03515625f, -0.87890625f, 0.1005859375f, -0.263671875f, 0.5f, -0.6171875f, + -0.98828125f, 0.6328125f, -0.46484375f, -0.7265625f, 0.05615234375f, -1.0078125f, -0.375f, + 1.421875f, 0.439453125f, 1.7421875f, -0.0517578125f, 0.43359375f, -0.017333984375f, + 0.1796875f, -1.3359375f, 0.70703125f, -1.359375f, -0.9765625f, -0.1728515625f, 1.7265625f, + 1.3125f, 0.365234375f, 0.13671875f, 1.3515625f, -0.1591796875f, 0.328125f, 0.2734375f, + -0.5f, 0.0224609375f, -1.765625f, -0.70703125f, -0.34375f, -0.9921875f, -1.28125f, + 0.61328125f, 1.2890625f, -0.11328125f, 0.056640625f, 0.4921875f, -0.494140625f, + 0.11181640625f, -3.25f, -0.04296875f, -0.3203125f, -0.11474609375f, -0.73828125f, + 0.2001953125f, 1.265625f, 1.109375f, -0.89453125f, -0.46484375f, -0.703125f, -2.125f, + -0.349609375f, -0.51171875f, -1.25f, 0.37109375f, -1.609375f, 0.0966796875f, + -0.08642578125f, 0.12060546875f, -0.6640625f, 1.21875f, -0.1787109375f, -0.255859375f, + -0.9296875f, -0.8828125f, 1.5625f, -0.48828125f, -0.15625f, -1.234375f, 0.34765625f, + 0.28515625f, -0.146484375f, -0.3203125f, 0.095703125f, 0.0927734375f, 0.345703125f, + 0.4609375f, -0.29296875f, -0.2890625f, 1.2109375f, -0.25390625f, -0.875f, 1.03125f, + 2.03125f, -0.6015625f, -0.416015625f, 0.283203125f, -2.734375f, 0.046142578125f, + 0.6484375f, -0.0400390625f, 1.6640625f, 0.48046875f, -0.408203125f, 0.53515625f, + 0.7734375f, 0.53125f, -0.28515625f, -0.97265625f, 0.380859375f, -0.181640625f, -2.046875f, + -0.1455078125f, 1.2265625f, 0.1435546875f, -1.234375f, -0.021484375f, 1.9453125f, + -0.484375f, 0.28515625f, -0.8984375f, -1.4296875f, -1.390625f, -0.0498046875f, + 0.1455078125f, -1.65625f, -0.8203125f, -0.390625f, 0.169921875f, -2.734375f, + -0.0123291015625f, -0.4453125f, 0.10009765625f, -0.57421875f, 0.7421875f, 1.0546875f, + 1.0234375f, 0.703125f, -1.2578125f, 1.7890625f, -0.84375f, 0.8984375f, -1.6875f, + -0.435546875f, -0.44140625f, 0.90234375f, 0.1455078125f, 0.44140625f, 0.00029754638671875f, + 0.1533203125f, 0.6015625f, -0.87890625f, 0.296875f, -1.015625f, -0.421875f, -0.5390625f, + 2.34375f, 3.28125f, -0.76171875f, 0.734375f, -0.181640625f, 1.0703125f, 0.12109375f, + -0.484375f, 0.0693359375f, 1.1171875f, 0.23828125f, -0.1875f, 0.275390625f, 0.03857421875f, + -0.5234375f, -0.2470703125f, 1.6328125f, -0.6953125f, -0.41796875f, 0.0189208984375f, + -0.1328125f, -0.56640625f, -0.01324462890625f, -1.7734375f, -0.04248046875f, 1.0546875f, + -2.46875f, 0.1630859375f, -1.453125f, 1.6328125f, -0.423828125f, -1.3828125f, -1.890625f, + -1.1796875f, -0.232421875f, -1.0625f, -0.078125f, 0.435546875f, 0.1572265625f, + 0.197265625f, 0.0947265625f, -1.3984375f, 0.17578125f, -0.515625f, -0.73828125f, + -0.37109375f, -1.109375f, 3.125f, 2.671875f, -0.671875f, -1.1953125f, 1.375f, 0.228515625f, + 0.1796875f, -0.138671875f, 0.2255859375f, 0.0007781982421875f, -0.193359375f, + -0.2294921875f, 0.6171875f, -0.89453125f, -1.734375f, -0.158203125f, -0.40234375f, + 2.234375f, 1.953125f, 0.494140625f, 0.390625f, -0.162109375f, -1.0859375f, 0.11376953125f, + -0.828125f, 0.0177001953125f, 0.2890625f, 0.5390625f, -1.109375f, 1.0625f, 0.62890625f, + -1.3671875f, -1.34375f, 0.16796875f, -1.0234375f, -0.9921875f, -0.625f, 0.00628662109375f, + 2.078125f, -0.2578125f, -0.337890625f, 0.1162109375f, 0.54296875f, 0.203125f, 0.64453125f, + 0.482421875f, 0.51171875f, -0.2197265625f, 2.625f, 1.8203125f, -1.96875f, -0.515625f, + 1.140625f, 0.01141357421875f, -1.1796875f, -0.03564453125f, 0.498046875f, 0.0712890625f, + -0.515625f, -0.79296875f, -0.412109375f, -0.84765625f, -1.765625f, 0.0f, -2.46875f, + -1.09375f, 0.1171875f, 0.46875f, -0.28125f, -0.400390625f, 1.140625f, -0.07763671875f, + 1.5f, -0.058837890625f, 0.83203125f, -0.004180908203125f, -0.146484375f, -0.6875f, -1.125f, + -0.95703125f, 1.171875f, 0.2001953125f, -0.7421875f, -1.3203125f, 0.60546875f, + 0.051513671875f, -0.97265625f, -0.173828125f, -0.25f, 0.08251953125f, -1.2578125f, + 2.46875f, -0.138671875f, -0.90234375f, 0.2216796875f, -0.7578125f, -1.8125f, 1.421875f, + -0.63671875f, 1.265625f, -1.46875f, -0.14453125f, -0.384765625f, 0.019775390625f, + 0.26171875f, -0.0023345947265625f, -0.89453125f, 0.341796875f, -0.31640625f, 0.43359375f, + 1.078125f, -0.953125f, -0.19140625f, 2.40625f, 0.96875f, -0.93359375f, -0.76171875f, + 0.037353515625f, 0.29296875f, 0.06884765625f, 0.0289306640625f, 0.11474609375f, 2.09375f, + 1.4296875f, 0.41796875f, 0.390625f, 0.7421875f, -1.09375f, 1.015625f, -1.1015625f, + 1.171875f, 0.177734375f, 1.78125f, -0.09228515625f, 1.8515625f, -0.1484375f, 1.03125f, + 0.01507568359375f, 0.87109375f, 0.10791015625f, -0.2158203125f, -0.423828125f, + -0.76953125f, -0.455078125f, -1.9609375f, 0.0595703125f, 1.4921875f, -0.8984375f, + -1.7265625f, -0.11328125f, 0.9765625f, -0.05029296875f, 0.78125f, -0.09326171875f, + -0.59765625f, -1.3984375f, -0.2470703125f, -1.4453125f, -0.56640625f, 1.5078125f, + -0.51953125f, -0.0673828125f, 1.0546875f, 0.828125f, 2.078125f, 0.07958984375f, 1.390625f, + -0.05322265625f, 0.154296875f, -0.09130859375f, -1.03125f, 0.59765625f, 0.1611328125f, + 0.0869140625f, 0.59765625f, -0.6953125f, -2.71875f, -2.390625f, -2.046875f, -0.369140625f, + -1.0625f, 0.28125f, 0.1259765625f, -0.01385498046875f, 1.015625f, -0.02734375f, + -0.51953125f, -0.45703125f, -0.053955078125f, 0.71875f, -0.8671875f, 0.90625f, -0.7734375f, + -0.158203125f, -0.8515625f, 0.5390625f, -0.921875f, 0.140625f, 2.03125f, + 0.0030364990234375f, -1.640625f, 0.11865234375f, -0.515625f, 0.52734375f, 0.05029296875f, + -1.2109375f, 3.03125f, 0.03564453125f, -1.3515625f, -0.23828125f, -2.03125f, + -0.1123046875f, -1.4296875f, -0.2236328125f, 0.494140625f, 0.01373291015625f, 1.109375f, + -0.0103759765625f, 0.109375f, 0.72265625f, 0.83203125f, 1.0703125f, 1.234375f, -1.2109375f, + 2.171875f, -1.6171875f, -0.4765625f, -2.25f, 1.015625f, -0.16015625f, 2.0f, + -0.05224609375f, -0.97265625f, 0.08349609375f, 0.5f, 0.375f, 0.041259765625f, -0.546875f, + -0.66015625f, -0.3046875f, -1.9765625f, 0.83203125f, 0.72265625f, -0.671875f, -0.109375f, + -0.294921875f, 0.279296875f, 0.21484375f, -1.296875f, 0.0703125f, -0.171875f, 0.91015625f, + -0.08935546875f, -1.5859375f, 0.7421875f, 0.5703125f, 1.84375f, 1.359375f, 1.40625f, + 1.171875f, 1.828125f, 0.01165771484375f, -1.9765625f, -0.03857421875f, -0.71875f, + 0.00665283203125f, 0.609375f, 0.2470703125f, -0.92578125f, -0.193359375f, 2.40625f, + -0.29296875f, -0.734375f, 0.6171875f, 0.73046875f, -0.08984375f, -2.359375f, 0.09765625f, + 0.4453125f, -0.1201171875f, 1.59375f, 0.06298828125f, -0.82421875f, -0.025634765625f, + -0.0830078125f, 0.88671875f, 2.234375f, -0.046142578125f, -2.15625f, -0.8125f, 0.5078125f, + 0.08056640625f, -0.609375f, -0.0859375f, 2.078125f, 0.022216796875f, -0.88671875f, + 0.1474609375f, 0.6015625f, 0.7109375f, -1.015625f, -0.36328125f, -1.78125f, 0.02001953125f, + 0.8359375f, 4.09375f, 0.09765625f, 0.251953125f, 0.3203125f, 0.03271484375f, -0.8125f, + 0.1240234375f, -0.047607421875f, -0.05859375f, -0.6015625f, 0.04248046875f, 0.140625f, + -1.4296875f, 0.78125f, -0.177734375f, 1.046875f, -2.625f, 0.1708984375f, 0.181640625f, + -0.1708984375f, -0.2890625f, -2.203125f, 0.1455078125f, 0.8984375f, -0.0211181640625f, + -0.060302734375f, -1.1953125f, 1.2578125f, -0.95703125f, 0.0849609375f, -0.734375f, + -0.7890625f, 1.6015625f, -0.6640625f, 0.0810546875f, -1.625f, -0.33203125f, 2.375f, + -0.014404296875f, 0.4765625f, 0.057861328125f, 0.77734375f, -0.78125f, 0.7109375f, + -0.326171875f, 0.2333984375f, -0.11083984375f, -1.0078125f, -1.90625f, -0.23828125f, + -0.416015625f, 0.2392578125f, 0.15625f, 2.046875f, -0.142578125f, -0.41796875f, + 0.1943359375f, -0.369140625f, 0.2001953125f, 0.1865234375f, 0.90234375f, -0.240234375f, + -0.28125f, 0.05859375f, 3.515625f, 0.7890625f, 0.265625f, -1.328125f, 0.03955078125f, + 0.984375f, -0.11181640625f, 0.5625f, -0.107421875f, -0.60546875f, -1.6484375f, + -0.197265625f, -0.859375f, -0.80859375f, -0.4375f, 0.36328125f, 0.5234375f, -0.27734375f, + -0.9375f, -0.2138671875f, 0.058349609375f, 1.109375f, -0.041015625f, -2.734375f, + -0.053466796875f, -0.02978515625f, -1.4921875f, 0.37890625f, -1.1328125f, -1.828125f, + 0.96484375f, -1.7421875f, 2.34375f, 0.66015625f, 0.20703125f, -1.1484375f, -0.0810546875f, + -0.369140625f, 0.06103515625f, -0.73046875f, -0.022705078125f, 0.41015625f, -0.55859375f, + -1.0625f, -0.53125f, 0.92578125f, 0.373046875f, -2.03125f, 0.70703125f, -0.97265625f, + 0.71484375f, -0.0869140625f, 0.26953125f, 0.2412109375f, -0.169921875f, -1.625f, + 0.0208740234375f, 1.6484375f, -0.052734375f, -1.015625f, -0.640625f, -1.15625f, + 0.451171875f, 1.3671875f, -1.28125f, 0.8671875f, -0.2275390625f, -0.86328125f, 0.33203125f, + -2.640625f, 0.2216796875f, -0.60546875f, -0.05615234375f, -0.06591796875f, -0.1689453125f, + -0.375f, -0.1787109375f, -0.71484375f, -0.51171875f, 0.6328125f, -2.84375f, 0.890625f, + -0.30078125f, -0.8671875f, 0.015380859375f, 2.296875f, 0.0035858154296875f, 0.6328125f, + -0.08154296875f, -0.63671875f, 0.44921875f, -0.71875f, -1.5625f, 0.8203125f, 0.349609375f, + 1.4375f, -1.0390625f, 0.0146484375f, -0.71484375f, -0.8125f, -0.060546875f, 0.353515625f, + -0.0291748046875f, 0.96875f, -0.0732421875f, 2.59375f, 1.1171875f, 0.1044921875f, + 1.0859375f, 0.1318359375f, -0.08203125f, 1.296875f, 1.390625f, -0.25390625f, -1.5f, + 0.71875f, 0.2353515625f, 0.15625f, 0.1865234375f, -1.5859375f, -0.05029296875f, + 0.271484375f, -1.4921875f, -0.83203125f, -1.2421875f, -0.97265625f, -0.1123046875f, + -1.0625f, -0.7421875f, 0.57421875f, 2.015625f, 0.66796875f, 0.05419921875f, -2.125f, + -0.1328125f, -0.64453125f, -0.035400390625f, -1.0625f, -1.4375f, -1.296875f, + -0.1865234375f, 0.5234375f, 0.255859375f, -1.765625f, -3.09375f, -0.0546875f, 0.98828125f, + 0.033203125f, 0.09375f, -0.296875f, -0.017333984375f, 0.008056640625f, -0.173828125f, + 0.76953125f, 1.109375f, 0.58984375f, -0.00775146484375f, 0.7890625f, -0.28515625f, + 0.85546875f, -2.328125f, 0.734375f, -0.2373046875f, -1.125f, -0.2421875f, 2.46875f, + -0.1376953125f, 0.365234375f, -0.10107421875f, 0.11083984375f, -1.3359375f, 0.435546875f, + -0.80078125f, -0.376953125f, 0.8828125f, -0.013671875f, 0.00750732421875f, -1.265625f, + 0.859375f, 0.7734375f, 0.328125f, -1.25f, 0.2138671875f, -0.98046875f, 0.0233154296875f, + -0.64453125f, 0.70703125f, 0.5546875f, -0.27734375f, 1.890625f, 0.1103515625f, + 0.119140625f, -0.765625f, -0.3125f, -1.2890625f, -0.201171875f, 0.18359375f, 1.5390625f, + 0.166015625f, 0.447265625f, 0.09765625f, 0.51171875f, -0.1962890625f, 0.6171875f, + 0.1318359375f, -0.58203125f, 0.80859375f, -0.0576171875f, 0.41015625f, 2.015625f, + -0.154296875f, -0.8828125f, 0.021484375f, -0.294921875f, -0.02490234375f, 0.06103515625f, + 0.1396484375f, 0.734375f, 1.6171875f, -0.21875f, -0.8203125f, 0.5859375f, -0.703125f, + -3.40625f, -1.5625f, -0.2080078125f, 0.224609375f, -0.38671875f, 0.00537109375f, + -0.79296875f, -0.1064453125f, 1.328125f, 0.060791015625f, -0.025390625f, 0.640625f, + -1.09375f, 0.92578125f, -1.234375f, 0.30859375f, -2.15625f, 0.796875f, -0.515625f, + -0.6640625f, 0.10546875f, -0.341796875f, -0.1953125f, -0.05322265625f, 0.70703125f, + 0.09228515625f, 1.6796875f, -0.83203125f, 0.016357421875f, 0.01153564453125f, -0.79296875f, + -0.5f, 1.3671875f, 0.25390625f, -0.3515625f, -2.546875f, 0.2890625f, -0.09326171875f, + 1.6875f, -0.02099609375f, -0.578125f, -0.058837890625f, 0.6796875f, -0.044921875f, + 0.72265625f, 0.306640625f, 0.828125f, 0.1533203125f, 0.416015625f, -0.65234375f, 1.015625f, + 0.82421875f, -1.7421875f, -0.0498046875f, 0.2578125f, -0.0157470703125f, 0.734375f, + 0.119140625f, -0.359375f, 1.8828125f, -0.953125f, -0.1787109375f, 0.41796875f, + 0.0025634765625f, -3.90625f, -2.0f, -0.5546875f, -0.0673828125f, -1.890625f, + -0.028564453125f, -0.439453125f, 0.06689453125f, -0.421875f, 0.052734375f, 0.734375f, + -1.625f, 0.00250244140625f, 0.419921875f, 1.1796875f, -0.1484375f, -0.15625f, 3.96875f, + 0.1796875f, 2.203125f, -0.119140625f, 0.345703125f, -0.2138671875f, -0.0252685546875f, + 0.2451171875f, -0.0703125f, -0.2470703125f, 0.0250244140625f, 0.1279296875f, 0.4453125f, + 0.4296875f, 0.1640625f, 0.97265625f, 2.25f, 1.2265625f, -0.203125f, 2.015625f, 0.35546875f, + -0.328125f, 0.0859375f, 0.50390625f, -0.007354736328125f, 0.54296875f, -1.8359375f, + 0.0830078125f, -0.2451171875f, 0.70703125f, -0.185546875f, -1.8046875f, -1.6171875f, + 0.2890625f, 1.1953125f, 0.609375f, -0.400390625f, 0.000431060791015625f, 0.2177734375f, + 1.4921875f, -0.03271484375f, -0.322265625f, -0.75f, 0.1689453125f, 0.8359375f, + 0.024658203125f, -1.078125f, -1.65625f, 0.78125f, 0.158203125f, 1.921875f, -0.890625f, + -0.0045166015625f, -0.80078125f, -0.162109375f, 1.671875f, 0.05419921875f, -1.8828125f, + 0.546875f, -0.66015625f, -0.035400390625f, 0.08203125f, -0.625f, -1.0390625f, + -0.06982421875f, -1.7734375f, 0.39453125f, 1.3203125f, -0.1796875f, 1.4375f, -0.125f, + -0.9453125f, 0.09130859375f, 0.2451171875f, -0.2314453125f, 1.4296875f, -0.7109375f, + 0.7890625f, -0.2138671875f, -0.095703125f, 3.34375f, 0.58984375f, -0.58984375f, + 0.72265625f, 0.361328125f, 1.2421875f, 0.02392578125f, 0.73046875f, 0.08349609375f, + 1.34375f, 1.21875f, -0.07080078125f, 0.365234375f, -0.6171875f, -0.1904296875f, 2.859375f, + 0.318359375f, 0.1904296875f, -0.06689453125f, -1.8125f, -0.115234375f, -0.059326171875f, + 0.0184326171875f, -0.9375f, 0.138671875f, 0.375f, 1.21875f, 0.251953125f, -0.392578125f, + -1.25f, -1.25f, -0.4296875f, 0.93359375f, 0.859375f, 1.6640625f, -0.7578125f, + 0.1650390625f, -0.89453125f, 0.203125f, 0.73828125f, 0.062255859375f, 1.03125f, 1.578125f, + -0.88671875f, -0.486328125f, -0.24609375f, 0.203125f, -1.796875f, -1.1875f, -1.140625f, + 1.1015625f, 0.78125f, -0.04541015625f, 0.55078125f, -0.07275390625f, -0.421875f, + -0.109375f, 1.2578125f, 1.9296875f, 0.171875f, -0.06103515625f, -0.4921875f, -0.267578125f, + -0.73828125f, -0.99609375f, 1.4296875f, -0.349609375f, -0.92578125f, 0.51953125f, + 0.185546875f, -0.058349609375f, 0.03857421875f, 0.08203125f, 0.111328125f, 0.6484375f, + 0.90625f, -0.734375f, -0.310546875f, 0.92578125f, 1.2421875f, -1.0625f, 0.95703125f, + -1.109375f, -0.2197265625f, 0.1484375f, -0.2412109375f, -0.19921875f, -1.390625f, + 0.01513671875f, 1.640625f, -0.76171875f, 0.107421875f, -1.5078125f, -1.0859375f, + -0.43359375f, 0.98046875f, 0.439453125f, 1.234375f, 0.328125f, 0.97265625f, + -0.0093994140625f, -0.74609375f, 0.07763671875f, -2.390625f, -0.01336669921875f, + -0.033447265625f, 1.828125f, 0.625f, -0.5078125f, 0.0020904541015625f, -1.1875f, + -0.0244140625f, -0.0673828125f, -0.064453125f, -0.0078125f, -1.0703125f, 0.341796875f, + -0.09814453125f, -0.0115966796875f, 1.1953125f, -0.04931640625f, -1.1796875f, 1.1328125f, + 0.1357421875f, -1.1875f, -1.0078125f, 0.10546875f, 0.5078125f, 1.5546875f, -0.9296875f, + -1.0546875f, 1.4296875f, -0.1572265625f, -0.875f, -0.1982421875f, -0.55859375f, + 0.0498046875f, 0.361328125f, -2.09375f, -0.049560546875f, 0.27734375f, 0.8828125f, + -0.470703125f, -1.4765625f, 1.3046875f, -0.5546875f, -0.203125f, 1.6875f, -0.09716796875f, + 1.0859375f, 0.0771484375f, 2.28125f, 0.047119140625f, -0.90234375f, -0.134765625f, + 0.1611328125f, 0.5546875f +}; +inline constexpr float kOverBudgetBlockKeys[] = { + 3.34375f, -0.54296875f, -0.3203125f, -0.1494140625f, 2.453125f, -1.3671875f, -1.0859375f, + 0.65625f, -2.0625f, -0.671875f, -0.390625f, 0.01226806640625f, -0.265625f, 0.04833984375f, + -0.75390625f, -0.8984375f, 1.90625f, 0.9453125f, -0.984375f, -0.1728515625f, 0.9140625f, + -0.498046875f, 0.984375f, 4.21875f, -2.453125f, 0.11669921875f, 0.421875f, + 0.0157470703125f, -0.50390625f, -0.11767578125f, 1.7890625f, -0.703125f, -0.3515625f, + -0.67578125f, -0.30078125f, -0.0439453125f, -3.0f, 0.9296875f, -0.5625f, -3.796875f, + -0.5390625f, -0.150390625f, -0.65234375f, 1.765625f, -0.9765625f, -0.12890625f, -3.203125f, + 1.1875f, -1.265625f, 0.119140625f, 0.029052734375f, 0.28125f, -1.078125f, 1.6015625f, + 0.07275390625f, -1.5703125f, -0.67578125f, -0.7734375f, -0.228515625f, 3.34375f, + 0.373046875f, 0.09375f, -0.83203125f, 0.07421875f, -1.9140625f, 1.1953125f, -0.2578125f, + 0.060546875f, -1.96875f, -0.4375f, 0.90625f, 0.76171875f, -1.0390625f, 0.5078125f, + -0.08154296875f, -3.640625f, 0.057861328125f, 0.10888671875f, 1.796875f, -1.1484375f +}; +inline constexpr int32_t kOverBudgetSelected[] = { + 0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0, 1, + 2, -1, -1, -1, -1, -1, -1, -1, -1, 0, 1, 2, 3, -1, -1, -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, + -1, -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, 5, -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, 5, 6, -1, -1, + -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, -1, -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, -1, -1, 0, 1, 2, 3, + 4, 5, 6, 7, 8, 9, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 0, 1, 2, 3, 8, 9, 10, 11, -1, -1, + -1, 0, 1, 2, 3, 4, 5, 6, 7, 12, -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, 12, 13, -1, 0, 1, 2, 3, 8, + 9, 10, 11, 12, 13, 14, 8, 9, 10, 11, 12, 13, 14, 15, -1, -1, -1, 4, 5, 6, 7, 8, 9, 10, 11, + 16, -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, 16, 17, -1, 0, 1, 2, 3, 8, 9, 10, 11, 16, 17, 18, 0, 1, + 2, 3, 4, 5, 6, 7, -1, -1, -1, 8, 9, 10, 11, 12, 13, 14, 15, 20, -1, -1, 0, 1, 2, 3, 12, 13, + 14, 15, 20, 21, -1, 4, 5, 6, 7, 16, 17, 18, 19, 20, 21, 22 +}; +inline constexpr float kOverBudgetAttnQ[] = { + -0.8046875f, 0.138671875f, -1.3203125f, 1.5546875f, 0.173828125f, 0.1611328125f, + -0.9140625f, 0.8203125f, -0.0244140625f, -0.0654296875f, 0.984375f, -0.451171875f, + 1.15625f, 0.72265625f, 0.07275390625f, 1.7578125f, -1.3671875f, 1.0390625f, 0.88671875f, + -0.08740234375f, -0.8359375f, -1.515625f, -1.1328125f, 1.3203125f, -0.63671875f, + -0.66015625f, 2.3125f, -0.734375f, 1.140625f, -1.8671875f, -0.25f, 1.078125f, -0.65625f, + 0.640625f, -0.59375f, -0.392578125f, -0.55859375f, -0.7578125f, -0.345703125f, 0.5625f, + 1.3828125f, -0.3515625f, -1.1015625f, -0.24609375f, -1.125f, 0.3125f, -1.09375f, + -0.67578125f, 0.35546875f, 1.90625f, -0.49609375f, -0.54296875f, 0.86328125f, -0.25390625f, + 1.0625f, 1.09375f, 0.2119140625f, 1.828125f, -0.345703125f, -0.1650390625f, -0.9140625f, + -2.125f, -0.169921875f, 0.546875f, 1.03125f, -0.953125f, -0.76171875f, -0.0201416015625f, + -0.78125f, -0.125f, 0.416015625f, -0.439453125f, 1.71875f, 1.9140625f, 0.8046875f, + 0.1982421875f, 0.7734375f, -1.421875f, 1.8671875f, 2.953125f, -0.625f, 0.1982421875f, + 1.0625f, -0.123046875f, 2.1875f, -1.1796875f, -0.10107421875f, 0.90625f, -1.3671875f, + -0.38671875f, -1.2734375f, 1.7578125f, 0.78515625f, -0.166015625f, -1.1484375f, + -0.14453125f, -0.78515625f, -1.015625f, -0.392578125f, 0.875f, -0.5625f, 1.4765625f, + -1.4375f, -0.625f, 0.1748046875f, 1.1796875f, 0.10009765625f, -0.5546875f, 0.9453125f, + 0.255859375f, 1.390625f, 1.1640625f, -0.396484375f, 0.453125f, 0.349609375f, + 0.0233154296875f, 0.26171875f, 0.91796875f, 0.64453125f, -0.83984375f, 2.0f, 0.390625f, + -0.734375f, -0.62109375f, 0.68359375f, -0.625f, 0.322265625f, 0.080078125f, -0.52734375f, + -2.234375f, -0.1494140625f, 0.154296875f, -0.71484375f, -1.6640625f, 0.380859375f, + 2.015625f, 0.1611328125f, -0.87890625f, -0.30078125f, -0.9375f, -0.1005859375f, + 0.0537109375f, 0.13671875f, 0.049560546875f, 0.5625f, 0.94140625f, -0.50390625f, + 0.07861328125f, 0.04150390625f, -1.140625f, -1.453125f, 0.48828125f, 0.94140625f, + -0.1806640625f, 0.53125f, 1.1015625f, -1.1015625f, -0.40625f, 0.25390625f, 1.7890625f, + -0.5703125f, 0.330078125f, -1.3671875f, -1.40625f, -0.302734375f, 0.361328125f, 0.734375f, + 0.298828125f, 0.146484375f, -0.142578125f, -0.10595703125f, 1.2734375f, 0.3515625f, + 1.1796875f, -0.498046875f, -0.298828125f, -2.6875f, 0.90625f, -1.2109375f, 0.83203125f, + -0.484375f, -0.58984375f, 0.86328125f, 0.359375f, -0.86328125f, 1.359375f, -1.3203125f, + -0.1328125f, -0.63671875f, 1.1015625f, 0.171875f, 1.0f, 0.6328125f, -0.0196533203125f, + 1.03125f, -0.46484375f, 0.1845703125f, -1.5078125f, 0.8359375f, -0.85546875f, -1.0625f, + 0.279296875f, 1.078125f, -0.8046875f, 0.076171875f, 0.95703125f, 0.06201171875f, 1.484375f, + 0.2373046875f, 1.296875f, -0.65625f, -1.8515625f, 0.6875f, -0.3046875f, 0.376953125f, + 2.03125f, -0.59375f, -0.291015625f, 0.490234375f, -0.7265625f, -0.271484375f, -1.671875f, + -0.059814453125f, -1.03125f, 1.203125f, -0.30859375f, 0.004150390625f, -0.85546875f, + 1.546875f, 0.1201171875f, -0.2080078125f, -1.6171875f, 0.88671875f, 0.80078125f, + -0.65234375f, -0.275390625f, 2.4375f, -0.91015625f, -0.0634765625f, -0.2021484375f, + -0.3828125f, -1.15625f, 0.53515625f, 0.091796875f, -0.267578125f, 0.828125f, -0.494140625f, + -1.2578125f, -0.9921875f, -1.8515625f, 0.357421875f, -1.328125f, 0.3671875f, -1.390625f, + -1.5625f, 0.259765625f, 0.99609375f, -1.3984375f, -0.04052734375f, -0.51171875f, + -0.8203125f, 0.609375f, -0.421875f, -0.0213623046875f, -0.875f, 0.1767578125f, -0.4609375f, + -0.94921875f, -0.1416015625f, 0.251953125f, 0.2138671875f, 0.9296875f, -0.56640625f, + 0.21484375f, -1.96875f, -1.2734375f, 1.28125f, 1.6484375f, 1.546875f, -0.294921875f, + 0.1728515625f, -1.3125f, -0.6328125f, -0.03857421875f, 1.3359375f, 0.2451171875f, + 0.77734375f, -1.953125f, -0.390625f, -1.3125f, 1.296875f, 0.41015625f, 1.171875f, + -0.29296875f, 0.796875f, -0.73046875f, 2.25f, -0.5390625f, -0.765625f, -0.052490234375f, + 0.173828125f, 0.166015625f, 0.8828125f, -0.33984375f, 1.6640625f, 0.255859375f, + 0.06396484375f, 0.287109375f, 1.15625f, -1.625f, 1.6640625f, -1.15625f, 0.7890625f, + 0.546875f, -0.92578125f, -0.41796875f, 0.9921875f, -0.1259765625f, -1.5625f, -0.65234375f, + -0.75390625f, 0.61328125f, 0.01495361328125f, 0.66015625f, -0.119140625f, -0.361328125f, + -0.341796875f, -1.359375f, 0.68359375f, 0.5f, -2.359375f, 0.9375f, 1.171875f, 1.625f, + -0.6328125f, -1.4375f, 1.265625f, -1.0625f, -0.3515625f, -0.0576171875f, -0.333984375f, + 0.7890625f, 0.5234375f, -0.25f, 0.59765625f, 0.271484375f, 1.4296875f, 1.2421875f, + 0.03173828125f, -0.5703125f, -0.4296875f, -0.54296875f, -0.73046875f, -1.0546875f, + 1.234375f, -1.0625f, -1.0390625f, 0.80078125f, -1.515625f, 0.75390625f, 0.330078125f, + -1.1171875f, 1.578125f, 1.2734375f, -1.2890625f, 0.625f, 0.048828125f, 0.055908203125f, + 0.455078125f, -1.4375f, -1.1640625f, 0.1396484375f, -0.2109375f, 0.302734375f, 1.2265625f, + -1.828125f, 0.6484375f, -0.2021484375f, -0.03515625f, -0.84765625f, 1.21875f, -0.36328125f, + 1.2265625f, 0.08740234375f, -0.640625f, 0.6171875f, -1.375f, 1.1015625f, -2.125f, + -0.8203125f, 0.03515625f, 0.322265625f, -0.90625f, 0.8359375f, -0.6171875f, -0.2119140625f, + 0.953125f, 0.265625f, 0.279296875f, -0.97265625f, 0.11181640625f, 1.6953125f, 0.6875f, + 0.71875f, 1.015625f, 0.49609375f, 2.265625f, 0.1787109375f, 0.365234375f, -1.0f, 0.171875f, + 1.6328125f, 0.796875f, 0.474609375f, 0.279296875f, -1.0859375f, -0.044677734375f, + -0.98828125f, 1.0703125f, -1.578125f, -0.7265625f, -0.1767578125f, -0.83203125f, + -1.171875f, 0.357421875f, 0.03173828125f, 0.443359375f, 1.0f, -0.45703125f, 0.89453125f, + 0.47265625f, -0.66015625f, -1.328125f, 1.0f, -1.15625f, -0.5f, 0.384765625f, -1.2265625f, + 1.28125f, 0.58203125f, -1.15625f, 0.59375f, 0.1103515625f, 0.78125f, 0.60546875f, + -0.39453125f, -0.70703125f, 0.828125f, -0.90234375f, 2.53125f, -0.76171875f, 0.0771484375f, + -0.283203125f, 0.76953125f, -0.84765625f, 0.46484375f, -0.2001953125f, 0.1875f, + -0.193359375f, -0.04248046875f, -0.271484375f, -0.2236328125f, 1.78125f, 1.3984375f, + 0.251953125f, 0.89453125f, -0.318359375f, -1.0546875f, -1.2890625f, 1.3515625f, + -0.353515625f, 0.0654296875f, -0.181640625f, -0.259765625f, -0.7578125f, -0.490234375f, + 1.2421875f, 0.72265625f, 0.1396484375f, 0.059326171875f, -1.15625f, 0.1787109375f, + -0.0419921875f, 1.2109375f, 1.0625f, 1.671875f, -0.8984375f, 0.92578125f, 0.1103515625f, + -1.0859375f, 0.2138671875f, 0.5390625f, 0.828125f, -0.8515625f, 1.5078125f, -0.177734375f, + 0.353515625f, -1.2890625f, 1.265625f, 0.0194091796875f, -0.7265625f, 1.953125f, + -0.34765625f, 0.40234375f, -0.8828125f, -0.28515625f, 1.6171875f, 0.8125f, -0.296875f, + -0.703125f, 0.259765625f, 0.166015625f, -0.83984375f, -0.88671875f, 0.69140625f, + -0.953125f, 0.91015625f, -1.1171875f, -0.10302734375f, 0.8359375f, -0.2275390625f, + -0.5546875f, 0.439453125f, 0.66796875f, 1.109375f, -0.033935546875f, -0.86328125f, + -1.0078125f, 0.1494140625f, -1.515625f, 0.85546875f, 0.54296875f, -0.384765625f, + -0.85546875f, -0.44140625f, 0.59375f, 0.859375f, 0.337890625f, -0.13671875f, 0.9609375f, + -0.0791015625f, 0.93359375f, 1.796875f, -0.2333984375f, -1.2734375f, 0.0108642578125f, + -1.15625f, -1.140625f, -1.296875f, -1.171875f, 0.43359375f, -0.59765625f, 0.1826171875f, + 0.453125f, 1.4765625f, -0.251953125f, 0.734375f, 0.09326171875f, -1.890625f, 0.1943359375f, + -1.0234375f, 1.0234375f, -0.279296875f, 0.5546875f, 1.6640625f, 0.400390625f, -0.08203125f, + 0.75f, -0.2255859375f, 0.3671875f, -1.53125f, 1.2578125f, -0.984375f, -1.53125f, + -0.69140625f, -0.119140625f, -0.88671875f, 0.203125f, 0.7265625f, 0.00762939453125f, + -1.875f, 0.34765625f, 0.328125f, -1.375f, 0.75390625f, -0.296875f, 0.08935546875f, + -1.203125f, -0.734375f, 0.90625f, 1.0703125f, -0.77734375f, -0.4296875f, 0.6953125f, + 0.375f, -0.6953125f, 0.30859375f, 1.0859375f, -0.59375f, 1.484375f, -1.5234375f, + -0.30078125f, 0.5625f, 0.314453125f, 0.51171875f, 0.9375f, 0.7109375f, 0.1826171875f, + -0.1953125f, 0.6640625f, 0.7109375f, 0.498046875f, -0.765625f, -1.4921875f, 1.46875f, + 1.40625f, 0.16015625f, -1.203125f, 1.7265625f, 0.01220703125f, 1.046875f, 0.53515625f, + 1.703125f, 0.6640625f, -1.4765625f, -0.73046875f, -1.1015625f, -0.71875f, -1.046875f, + -0.130859375f, -0.53125f, -0.85546875f, -0.74609375f, -1.171875f, 0.1923828125f, + 0.1259765625f, 1.9609375f, -0.71875f, -0.333984375f, -0.640625f, -1.0625f, -0.8046875f, + 1.1328125f, 0.1669921875f, -1.3671875f, 0.455078125f, -2.171875f, -0.2265625f, 1.234375f, + 0.2197265625f, -0.578125f, 1.6328125f, 0.5390625f, 0.6484375f, 0.78125f, 0.263671875f, + -0.1611328125f, 1.640625f, -0.0927734375f, -0.251953125f, -0.125f, 0.8828125f, + -0.41796875f, 1.90625f, -1.8359375f, 0.040283203125f, -0.1220703125f, 1.6875f, -2.234375f, + -0.7265625f, -0.671875f, -0.302734375f, 0.1455078125f, 0.11328125f, 1.265625f, + -0.90234375f, 0.890625f, 1.4296875f, 0.80078125f, 1.71875f, 0.185546875f, 0.2421875f, + -1.3359375f, 0.201171875f, -1.5f, -0.6171875f, 0.625f, 0.2119140625f, -0.578125f, + -2.015625f, -0.01226806640625f, 0.58203125f, 1.171875f, 0.7890625f, -0.6484375f, + -1.5859375f, -0.07177734375f, -0.1884765625f, 0.53515625f, -0.421875f, -0.59375f, + -0.051025390625f, -1.296875f, -1.0703125f, 0.494140625f, -0.8515625f, -0.057373046875f, + -0.337890625f, 1.671875f, -1.7734375f, -0.84375f, 1.0f, -1.1328125f, -1.4765625f, + 0.81640625f, -0.8046875f, 0.59765625f, -1.359375f, 0.39453125f, 0.73828125f, -0.5234375f, + 0.77734375f, -0.416015625f, 1.1953125f, -1.25f, -0.7265625f, 0.486328125f, -1.0f, + 1.359375f, -0.6015625f, 1.078125f, -1.4375f, 2.09375f, 0.42578125f, 0.244140625f, + 1.4140625f, -1.0390625f, 0.0010528564453125f, -0.400390625f, -0.73046875f, -2.0f, + -0.75390625f, 1.5546875f, -0.26953125f, 1.421875f, 2.171875f, -1.421875f, -0.4140625f, + -1.9765625f, -1.3046875f, 1.3359375f, -0.8515625f, -1.375f, 0.67578125f, 1.0078125f, + 0.55078125f, 1.953125f, -1.078125f, -0.478515625f, 0.62890625f, -0.77734375f, -0.67578125f, + -0.09716796875f, 1.3359375f, 0.388671875f, 0.240234375f, -0.67578125f, -0.248046875f, + 1.25f, 2.25f, -0.07470703125f, -0.44921875f, 1.296875f, -0.072265625f, 1.09375f, + -0.4140625f, -0.79296875f, 0.6171875f, 1.609375f, -2.46875f, 0.2255859375f, -0.82421875f, + -2.125f, 0.41015625f, -1.1484375f, 0.1611328125f, 0.482421875f, 1.5234375f, -1.40625f, + -0.75390625f, 0.94921875f, 0.3203125f, 0.60546875f, 0.2294921875f, 0.083984375f, + -1.5234375f, -1.5234375f, 1.0625f, 2.640625f, 0.89453125f, -0.63671875f, -0.1728515625f, + 0.341796875f, 0.298828125f, -1.7890625f, -0.08349609375f, 0.1279296875f, -0.734375f, + 1.28125f, -0.44921875f, 1.375f, 1.34375f, 0.0576171875f, -0.953125f, -0.34765625f, + 1.0390625f, -0.08349609375f, 0.92578125f, 0.90234375f, -0.7578125f, 0.0849609375f, + 0.76953125f, -0.54296875f, 0.392578125f, -0.181640625f, -0.546875f, 0.482421875f, 2.125f, + 0.310546875f, -0.421875f, -1.4140625f, 0.019287109375f, 0.58203125f, -1.203125f, + -0.2099609375f, 1.2421875f, 0.2392578125f, -0.58203125f, 0.75f, -0.765625f, 0.275390625f, + -0.984375f, -2.390625f, 0.15234375f, 0.1201171875f, -1.84375f, -2.046875f, -0.83203125f, + -0.6875f, 1.40625f, 1.546875f, 0.6796875f, -1.0703125f, 0.142578125f, -0.44140625f, + 0.087890625f, 1.28125f, -0.66015625f, 1.09375f, 0.08349609375f, 1.5078125f, + -0.056396484375f, -1.1484375f, 0.443359375f, -1.7265625f, 0.453125f, 1.78125f, -1.484375f, + 0.2373046875f, 0.333984375f, 0.56640625f, 1.5625f, 0.47265625f, -0.48046875f, -0.76953125f, + -0.138671875f, 0.375f, -1.703125f, -1.0234375f, -0.248046875f, 0.5390625f, 0.439453125f, + -0.1083984375f, -0.1259765625f, 1.2578125f, 0.71484375f, -0.1630859375f, 0.080078125f, + -0.50390625f, 0.439453125f, 0.302734375f, 1.0078125f, 1.7890625f, -0.11376953125f, + -1.078125f, -2.125f, 0.62109375f, -1.3046875f, -1.25f, -0.25f, -0.431640625f, + -0.384765625f, -0.396484375f, -1.0234375f, 0.9296875f, 1.265625f, -0.84375f, 0.376953125f, + -0.138671875f, -0.0791015625f, 0.359375f, -1.8984375f, 1.0390625f, -0.1669921875f, + 0.142578125f, -1.1328125f, 1.1171875f, -0.5546875f, -0.0751953125f, -0.51171875f, + 1.265625f, -0.9140625f, 0.4296875f, -1.3203125f, 2.390625f, 0.259765625f, 1.6171875f, + -0.392578125f, -0.96875f, 0.263671875f, -0.75390625f, -0.0294189453125f, -1.4921875f, + 2.421875f, 1.0390625f, -0.1298828125f, -0.58984375f, -1.1484375f, 0.578125f, -1.1796875f, + 1.578125f, -0.46484375f, 0.462890625f, -0.08154296875f, -1.375f, 0.52734375f, -0.6171875f, + 0.5546875f, -0.232421875f, 0.81640625f, 2.25f, -1.9921875f, -0.6171875f, -0.72265625f, + -1.953125f, -1.078125f, -1.078125f, -0.322265625f, 0.05517578125f, 0.158203125f, + 0.150390625f, 0.376953125f, -0.46484375f, 2.140625f, 0.5390625f, -0.6796875f, 1.15625f, + 0.2431640625f, -0.9140625f, 0.055908203125f, -0.2431640625f, 0.263671875f, 0.62890625f, + 0.0859375f, -0.038330078125f, -0.82421875f, 0.2060546875f, 2.359375f, 2.03125f, -1.1875f, + 0.357421875f, 0.055419921875f, -0.765625f, -1.65625f, -0.47265625f, 0.58984375f, + 0.1513671875f, 1.796875f, -1.5625f, -0.419921875f, -0.5703125f, 1.1171875f, 0.38671875f, + -1.4921875f, -0.1865234375f, -0.59765625f, -0.68359375f, 1.3125f, -1.4765625f, + 0.0791015625f, 0.369140625f, -0.93359375f, -0.16796875f, -1.3046875f, -0.345703125f, + 0.92578125f, -1.0234375f, 0.76953125f, -0.734375f, -0.69140625f, -0.055908203125f, + -0.1064453125f, -1.1796875f, -0.5f, -0.6171875f, 2.28125f, -1.2734375f, -2.453125f, + 2.4375f, 0.5390625f, 0.484375f, 0.640625f, 0.416015625f, 0.625f, -0.9140625f, -1.2265625f, + 1.5f, 0.55859375f, 1.3203125f, 1.1015625f, 2.078125f, 1.5625f, 0.16015625f, 0.82421875f, + 1.2265625f, 0.6796875f, -0.337890625f, -1.4140625f, 1.703125f, -0.9921875f, 1.59375f, + -1.15625f, 0.00023746490478515625f, 0.2177734375f, 1.984375f, -1.4296875f, -0.34375f, + -0.201171875f, 0.078125f, 0.4765625f, 0.490234375f, 1.453125f, -0.57421875f, -1.4140625f, + 0.375f, -0.08251953125f, -0.4140625f, -0.640625f, 0.11181640625f, 1.2734375f, -0.26171875f, + 0.78125f, 0.71875f, 0.94921875f, -0.451171875f, 0.06103515625f, -0.10498046875f, 0.71875f, + -0.83203125f, 0.94921875f, -0.046630859375f, 1.1015625f, 1.765625f, 0.97265625f, + 1.0703125f, -1.5859375f, -0.45703125f, -0.11181640625f, 0.55078125f, 0.74609375f, + -1.5546875f, -0.703125f, -0.2392578125f, 0.5703125f, 0.0654296875f, 0.400390625f, + -0.58203125f, -0.439453125f, -0.7734375f, -0.314453125f, -0.1572265625f, -1.9765625f, + 1.0078125f, 0.4453125f, -0.32421875f, -0.2099609375f, 1.09375f, 1.2734375f, 0.5625f, + 1.3359375f, -1.578125f, -1.34375f, -0.70703125f, 1.6796875f, 1.7890625f, -0.8984375f, + 0.185546875f, -0.57421875f, 0.384765625f, 1.3984375f, 1.421875f, 2.03125f, -0.7734375f, + -0.890625f, -0.0732421875f, 2.34375f, 0.93359375f, 0.578125f, -0.462890625f, -1.109375f, + -0.103515625f, 1.2890625f, 0.453125f, -0.34375f, -0.1796875f, -1.046875f, -0.4609375f, + -1.78125f, -0.19140625f, -1.203125f, 0.0f, 0.89453125f, -0.5703125f, -0.9453125f, + 0.5546875f, 1.7578125f, 1.671875f, 0.91015625f, -0.921875f, -0.84765625f, 0.328125f, + -1.6640625f, -0.671875f, 0.310546875f, 0.64453125f, 2.21875f, 0.890625f, 0.08056640625f, + -0.58984375f, 0.1318359375f, 0.55078125f, -0.2412109375f, 0.419921875f, 0.1943359375f, + 0.33203125f, -0.5546875f, -0.76171875f, -1.15625f, -0.49609375f, 0.1279296875f, + 0.380859375f, 0.875f, 0.921875f, 1.671875f, 1.7265625f, 1.046875f, -0.474609375f, -0.5625f, + 0.765625f, -0.1728515625f, 1.078125f, 0.44921875f, -1.203125f, 0.384765625f, -1.8828125f, + -0.033203125f, 0.275390625f, -0.546875f, 0.26953125f, -0.44921875f, -0.1689453125f, + -0.52734375f, -0.84375f, -0.58984375f, 0.71484375f, 0.96484375f, 0.609375f, + -0.05810546875f, 0.4453125f, -0.875f, -0.69140625f, -0.28515625f, 1.40625f, 0.546875f, + -0.609375f, 0.5703125f, 0.703125f, 0.671875f, 2.265625f, 1.6484375f, -2.046875f, 1.203125f, + 0.76953125f, -1.1484375f, -1.046875f, 0.35546875f, -1.5390625f, 0.2734375f, 0.259765625f, + -1.203125f, -0.07421875f, -1.25f, -1.375f, 1.7890625f, -0.046630859375f, -0.7421875f, + -0.55859375f, -0.19921875f, 1.8203125f, -0.48828125f, 0.50390625f, -0.765625f, -0.6875f, + 0.7734375f, 1.6796875f, -0.0091552734375f, -1.0f, 0.10107421875f, 1.2265625f, + -0.10791015625f, -0.52734375f, -1.9296875f, 1.4140625f, 0.9375f, -1.78125f, -1.1796875f, + -0.53515625f, 0.53515625f, -0.2265625f, -0.337890625f, 1.6484375f, 0.8984375f, 1.375f, + 0.361328125f, 0.09326171875f, 1.046875f, -0.6484375f, -0.625f, -1.0390625f, -1.4609375f, + 0.62109375f, 0.73828125f, -0.3828125f, 0.53515625f, 0.6953125f, 1.0f, -1.671875f, + -0.443359375f, 1.3203125f, 1.0f, 1.7265625f, -0.77734375f, -0.47265625f, -0.0966796875f, + 0.154296875f, -0.19921875f, -0.578125f, 0.1455078125f, -1.3125f, 2.25f, 0.087890625f, + 0.1884765625f, 1.5f, -0.0888671875f, 0.609375f, -1.4453125f, 0.042236328125f, 1.1953125f, + -0.53515625f, 0.33203125f, -0.3203125f, -0.79296875f, -0.2734375f, 1.3515625f, + -0.09814453125f, -0.1640625f, 0.1611328125f, 0.498046875f, -0.37109375f, 0.58984375f, + -0.59375f, 1.296875f, -0.2138671875f, -0.8984375f, -1.2265625f, 0.78515625f, 0.328125f, + 0.07666015625f, 0.287109375f, 0.1220703125f, 1.5703125f, -1.1875f, -0.78515625f, 1.0f, + 0.8515625f, -0.0986328125f, -0.41796875f, -0.46484375f, -1.296875f, 1.015625f, 0.28125f, + 0.09912109375f, 0.58984375f, 0.035400390625f, -1.1796875f, -0.69140625f, 0.0673828125f, + -1.6328125f, 1.6484375f, 0.98046875f, -0.1328125f, -0.0810546875f, 1.6171875f, + 0.283203125f, 0.447265625f, 0.078125f, -0.97265625f, -0.125f, -0.26171875f, + -0.033935546875f, -0.0986328125f, 2.140625f, 0.0322265625f, 1.453125f, 2.8125f, 0.3515625f, + 2.578125f, 0.73046875f, 1.40625f, 0.2119140625f, 0.21484375f, -0.62890625f, + -0.09326171875f, -0.51171875f, -0.1142578125f, -0.400390625f, -0.99609375f, 0.66015625f, + 0.64453125f, 1.765625f, 0.1953125f, -0.91796875f, 0.765625f, -0.423828125f, 0.44921875f, + 1.40625f, 0.875f, 0.130859375f, -0.1826171875f, 0.7578125f, 0.59765625f, 1.2421875f, + -0.52734375f, 0.1533203125f, -0.7578125f, -1.03125f, -0.0155029296875f, -0.91015625f, + 0.6953125f, -1.3125f, -1.25f, 2.15625f, -0.16796875f, 0.04931640625f, 0.67578125f, + -1.578125f, -0.400390625f, 0.09716796875f, 1.0859375f, -0.236328125f, -0.197265625f, + -0.2138671875f, -0.9921875f, 0.142578125f, 0.109375f, -1.40625f, -0.83984375f, -1.2890625f, + -0.091796875f, 0.4296875f, -0.21875f, -1.078125f, -0.41015625f, -1.0625f, -0.453125f, + -1.21875f, 0.443359375f, -2.359375f, 0.205078125f, -0.60546875f, -0.61328125f, + 0.291015625f, 0.162109375f, 1.6640625f, 0.87109375f, 0.002288818359375f, -0.51171875f, + 0.01953125f, -0.0830078125f, 1.8671875f, -1.0234375f, -0.484375f, -0.9453125f, + 0.1318359375f, 0.01434326171875f, -0.515625f, 1.7890625f, -2.109375f, 1.4453125f, + -0.294921875f, 1.765625f, 1.046875f, -0.08740234375f, 2.09375f, 1.078125f, 1.171875f, + -0.484375f, -1.2265625f, 0.330078125f, 0.61328125f, -0.416015625f, 0.9609375f, -0.59375f, + 0.451171875f, 2.1875f, 0.001190185546875f, -1.6796875f, 0.875f, 0.439453125f, -0.56640625f, + -0.2412109375f, -0.68359375f, -0.59765625f, -0.298828125f, -1.40625f, 0.76171875f, + -0.59375f, -0.76953125f, 0.07373046875f, 0.5078125f, 0.4453125f, 0.1181640625f, + -0.427734375f, 0.361328125f, -1.875f, 0.3046875f, -0.06298828125f, -0.8359375f, 1.578125f, + -0.47265625f, 0.6015625f, -1.5859375f, 1.2734375f, 0.7265625f, -0.062255859375f, + -0.5078125f, 0.39453125f, 0.60546875f, -0.72265625f, -0.83203125f, 0.2216796875f, + -0.047119140625f, -1.8046875f, 0.84375f, 0.341796875f, 0.36328125f, -1.109375f, -1.265625f, + 1.0078125f, 0.041015625f, -1.125f, 0.4765625f, -0.93359375f, 1.828125f, -0.15625f, 1.375f, + 1.515625f, 1.7109375f, 0.91015625f, 1.4140625f, -0.25390625f, -0.6171875f, 1.1328125f, + -1.1640625f, -0.0693359375f, -1.2734375f, 0.3515625f, -0.609375f, -0.470703125f, + -1.953125f, -1.28125f, 0.1669921875f, -0.7890625f, 0.57421875f, 0.177734375f, 0.47265625f, + 0.43359375f, 1.03125f, -0.57421875f, -1.0625f, 0.091796875f, 1.359375f, 0.125f, + -0.416015625f, -0.6328125f, 1.71875f, -1.15625f, 0.74609375f, 0.462890625f, 0.1337890625f, + -1.2109375f, 0.578125f, -0.66796875f, 0.59765625f, 1.7578125f, -2.046875f, 0.2177734375f, + 0.80078125f, -1.5f, -1.75f, -0.26171875f, 0.1962890625f, -0.2255859375f, -1.2421875f, + -0.341796875f, -1.1015625f, -1.8359375f, -0.2109375f, -1.375f, 0.0f, -2.625f, -0.640625f, + -0.169921875f, 1.40625f, -1.2421875f, -1.3984375f, 0.306640625f, 0.0f, -0.2451171875f, + 1.9296875f, 0.052001953125f, 0.28125f, 0.88671875f, 0.8984375f, 0.8828125f, + 0.00119781494140625f, 0.28125f, 0.5234375f, -0.38671875f, -1.484375f, 0.2216796875f, + -1.015625f, 0.55859375f, 2.46875f, 0.0137939453125f, 0.47265625f, -2.5625f, 1.34375f, + 0.39453125f, 0.474609375f, 0.84765625f, -0.072265625f, -0.458984375f, 0.796875f, + 0.66015625f, -0.4921875f, -1.5078125f, -1.8125f, 2.765625f, 0.484375f, -1.625f, + -1.2421875f, 0.84375f, 0.1259765625f, -0.58984375f, -0.82421875f, 0.482421875f, -2.25f, + 1.078125f, 1.75f, -0.54296875f, 0.1494140625f, -0.369140625f, -0.46484375f, 0.65234375f, + -0.921875f, 0.388671875f, -1.984375f, -0.94140625f, -0.033447265625f, -0.64453125f, + -0.71484375f, 1.1484375f, 1.3515625f, 0.5703125f, 0.8671875f, -0.263671875f, 2.0f, + -1.3984375f, -1.328125f, 1.3359375f, 0.51953125f, -0.83203125f, 0.263671875f, + 0.0458984375f, 0.76953125f, -1.6953125f, 1.4765625f, 0.55859375f, -0.031494140625f, + 0.8671875f, 0.8984375f, -0.72265625f, -0.1923828125f, 0.134765625f, -1.296875f, 1.1015625f, + 1.375f, -0.8125f, -0.94921875f, 0.103515625f, 1.2890625f, -2.4375f, -0.05322265625f, + 1.09375f, -1.1953125f, -0.51953125f, -1.140625f, -1.5390625f, 1.1484375f, 0.423828125f, + -0.29296875f, 0.98828125f, 1.2578125f, 0.51171875f, -0.93359375f, -0.1923828125f, + 0.228515625f, 1.015625f, -1.3515625f, 1.1875f, -1.3671875f, -1.328125f, -1.1875f, + -0.234375f, 0.83203125f, -0.515625f, -0.041015625f, 2.09375f, -2.375f, 1.3125f, 1.078125f, + -0.091796875f, -1.2109375f, -0.34765625f, 0.21484375f, -0.039794921875f, -0.55859375f, + -0.359375f, 0.8203125f, -1.9140625f, 0.2255859375f, 0.90625f, -0.9375f, -1.59375f, + 0.353515625f, -0.275390625f, -0.72265625f, 0.7578125f, -0.1435546875f, -1.21875f, + -0.47265625f, -0.5703125f, 0.486328125f, 0.416015625f, -1.671875f, 0.80078125f, + 0.77734375f, -0.5f, 0.0225830078125f, 0.126953125f, -2.125f, 0.65625f, -1.2890625f, 1.5f, + -0.42578125f, -0.333984375f, -0.84765625f, -0.1748046875f, -0.314453125f, -0.248046875f, + -1.421875f, 2.140625f, -0.3515625f, -1.0546875f, -0.2578125f, 0.5703125f, -2.078125f, + 0.2890625f, -0.55859375f, 0.76953125f, -1.25f, -0.14453125f, 0.76171875f, -1.109375f, + -0.1875f, 0.1943359375f, 1.1015625f, -2.375f, 1.6015625f, -1.640625f, 0.73046875f, + 1.2890625f, 0.52734375f, 0.04345703125f, 0.58984375f, -0.333984375f, -0.94921875f, + 1.4140625f, 1.515625f, -1.2421875f, 0.06396484375f, 0.70703125f, 1.6953125f, -0.71484375f, + 2.578125f, -1.59375f, -1.7109375f, 1.59375f, 0.90234375f, 0.70703125f, 0.0830078125f, + 1.0078125f, 0.92578125f, -2.75f, 0.2333984375f, 1.03125f, 0.392578125f, -1.640625f, + 0.1318359375f, 0.3984375f, 0.423828125f, -0.88671875f, 0.458984375f, -0.74609375f, + -0.1279296875f, -1.734375f, -0.8046875f, 1.7890625f, 0.66796875f, -2.234375f, + -0.251953125f, -0.57421875f, 0.515625f, -0.10888671875f, 0.57421875f, 0.3046875f, + -0.5234375f, -0.5703125f, -2.109375f, -0.7578125f, -0.16796875f, 0.390625f, 0.404296875f, + 1.1171875f, 1.4609375f, 1.2421875f, 0.06640625f, 1.4375f, -0.294921875f, -0.369140625f, + 0.193359375f, 0.369140625f, -2.03125f, -3.09375f, 0.6484375f, 0.359375f, -2.484375f, + 0.03515625f, 1.2890625f, -0.02392578125f, 0.453125f, 0.1943359375f, -2.171875f, + 0.94921875f, -0.6484375f, 0.58984375f, -0.19921875f, -0.451171875f, -1.3515625f, + -0.7421875f, -1.1015625f, 0.66796875f, 1.359375f, 0.296875f, 0.00640869140625f, 2.59375f, + -1.78125f, -1.4609375f, -1.1015625f, -0.435546875f, 1.03125f, -0.89453125f, -1.3828125f, + -0.23046875f, 1.0546875f, -1.5859375f, 1.203125f, 2.5f, -1.1640625f, 0.8828125f, + 0.06640625f, -0.302734375f, -0.578125f, -0.05224609375f, 1.1171875f, -0.2392578125f, -2.0f, + -0.96484375f, 0.64453125f, -0.76953125f, -1.203125f, -0.59375f, -0.61328125f, 1.9140625f, + 1.015625f, 1.1640625f, 1.3828125f, 0.51953125f, -0.609375f, -0.09228515625f, + -0.08544921875f, -0.12060546875f, -0.9765625f, -0.412109375f, 0.5234375f, -0.1640625f, + -0.0186767578125f, 1.9921875f, 1.84375f, 0.314453125f, 0.07763671875f, 1.8515625f, + -1.3046875f, 1.09375f, -0.08642578125f, 0.546875f, 0.59765625f, -0.8125f, -0.70703125f, + -0.84765625f, 0.1728515625f, -1.09375f, 0.93359375f, -0.1767578125f, 0.9921875f, + -0.220703125f, -0.31640625f, -0.0791015625f, 0.5234375f, -0.431640625f, -1.28125f, + 1.0390625f, 1.28125f, 0.1328125f, 0.37109375f, -0.1708984375f, 0.703125f, -0.90625f, + -1.9140625f, 0.4609375f, -0.703125f, 0.8828125f, -0.72265625f, 0.2578125f, 0.46875f, + 0.32421875f, -1.1640625f, -0.07666015625f, -1.046875f, 2.03125f, -0.1025390625f, 0.0f, + 1.9140625f, -1.4140625f, 1.578125f, -0.875f, 1.2578125f, -1.109375f, 0.6875f, 0.0f, + -0.3046875f, -0.50390625f, -1.140625f, 0.4296875f, -0.63671875f, 0.2041015625f, + -0.87109375f, -0.03662109375f, 0.7734375f, -0.6484375f, 0.330078125f, -1.640625f, + 0.578125f, 1.171875f, -0.0400390625f, -1.46875f, 0.54296875f, 1.2109375f, 0.232421875f, + 1.3359375f, -0.64453125f, -0.8359375f, 0.2392578125f, 0.9453125f, -0.2421875f, + 0.0181884765625f, 0.3984375f, -0.96484375f, 1.203125f, -0.328125f, -0.65234375f, + 0.09326171875f, -0.765625f, 0.1767578125f, -0.9140625f, -0.12158203125f, 0.546875f, + -0.921875f, -0.498046875f, 1.3984375f, -0.03125f, 0.9453125f, 0.53515625f, -0.09326171875f, + -1.65625f, 0.4296875f, 0.1962890625f, -0.0908203125f, 1.359375f, -0.51953125f, -0.6015625f, + 0.7578125f, -1.09375f, 0.224609375f, -1.125f, -1.25f, 0.251953125f, -0.921875f, + -0.1806640625f, -0.025146484375f, -0.8359375f, -0.30859375f, 0.5078125f, 0.21875f, + 2.453125f, -0.412109375f, 0.1748046875f, 0.90234375f, -0.060791015625f, 0.6875f, + 0.71484375f, 0.828125f, -0.66796875f, 1.28125f, 1.109375f, 0.06689453125f, 0.486328125f, + -0.349609375f, -1.640625f, 0.828125f, 0.095703125f, -1.1328125f, -1.171875f, 0.66015625f, + 2.390625f, 0.89453125f, 0.01165771484375f, 0.62109375f, 0.60546875f, 1.0703125f, 1.1875f, + 0.049072265625f, -0.61328125f, 0.396484375f, -0.310546875f, 0.6484375f, 0.95703125f, + 0.10595703125f, -0.859375f, -0.7734375f, -0.51953125f, 1.3203125f, -0.390625f, + -0.33203125f, 1.3359375f, -0.7109375f, -2.21875f, -0.072265625f, -1.4921875f, 0.5859375f, + -0.51171875f, 1.3515625f, 0.63671875f, -0.33203125f, -1.9921875f, 0.2333984375f, + -0.94140625f, 0.302734375f, 1.15625f, -1.3984375f, 0.458984375f, -0.9765625f, + -0.365234375f, -0.890625f, -0.90234375f, -0.5078125f, 0.0f, -0.06396484375f, -0.4296875f, + -0.7265625f, -0.6328125f, 0.125f, -0.384765625f, 0.39453125f, -0.86328125f, -0.58984375f, + 0.5546875f, 0.63671875f, 0.1796875f, 0.087890625f, 0.875f, -0.57421875f, 0.1513671875f, + 1.34375f, -1.3828125f, -0.1015625f, -1.09375f, -2.359375f, -0.244140625f, 0.578125f, + -2.0625f, 0.94921875f, 1.4296875f, -1.0859375f, 1.5234375f, 0.337890625f, -1.3359375f, + 0.5546875f, -1.046875f, 1.421875f, -0.97265625f, 2.1875f, -0.8359375f, -1.1796875f, + 0.490234375f, 0.12890625f, -1.296875f, 1.21875f, -1.265625f, 0.1103515625f, 1.1875f, + 0.0011444091796875f, 0.0240478515625f, -0.294921875f, 0.58203125f, 1.921875f, 1.140625f, + 2.328125f, 0.56640625f, -1.0234375f, -1.2421875f, -1.765625f, 0.63671875f, -0.466796875f, + -0.99609375f, -0.478515625f, -0.52734375f, 0.212890625f, 1.3125f, -0.053466796875f, + 0.193359375f, 1.1328125f, 0.44921875f, 0.10693359375f, 0.298828125f, 0.09619140625f, + 1.0625f, -0.458984375f, -0.609375f, -1.0859375f, 1.0625f, -0.6171875f, -0.953125f, + 0.55078125f, 0.74609375f, 0.69140625f, 0.1259765625f, 0.3515625f, 0.2373046875f, + -0.9921875f, 0.921875f, -1.28125f, 0.37890625f, -1.3515625f, -0.5f, 0.3515625f, + -0.99609375f, -0.3359375f, 0.11962890625f, -0.058837890625f, 0.8125f, 0.71484375f, + -1.296875f, -0.11474609375f, -1.4921875f, -0.365234375f, -0.90625f, -0.5234375f, + -0.1494140625f, 0.1142578125f, 1.390625f, 1.2890625f, 0.80859375f, -0.02978515625f, + -0.03125f, -1.2421875f, 0.7578125f, 0.1630859375f, 0.1376953125f, -0.81640625f, + -0.3203125f, -0.197265625f, -0.55859375f, 0.91796875f, -1.0625f, -1.328125f, -2.09375f, + 0.83984375f, -2.328125f, -1.0390625f, -0.59765625f, 1.0625f, -0.08251953125f, -0.921875f, + -1.09375f, 0.255859375f, 0.00225830078125f, 2.53125f, 1.359375f, 0.4921875f, 0.85546875f, + 1.3046875f, -1.5234375f, -0.30859375f, -0.3125f, -0.052978515625f, 0.9765625f, -1.2265625f, + -0.85546875f, -2.34375f, -0.21484375f, -0.9765625f, 2.09375f, 2.3125f, 1.921875f, + -0.09912109375f, 1.8046875f, 2.421875f, 0.63671875f, -1.703125f, 0.380859375f, + -0.88671875f, -1.1796875f, -0.65625f, -0.287109375f, 0.60546875f, 0.8125f, -0.66796875f, + 0.28125f, -0.040771484375f, 0.421875f, -0.41796875f, 0.59765625f, -0.2373046875f, + 1.9140625f, 0.62109375f, -1.609375f, -1.953125f, -0.5546875f, 0.83984375f, 0.3984375f, + 0.0849609375f, -0.5546875f, 1.09375f, 2.21875f, -0.05908203125f, -0.078125f, -0.486328125f, + -0.98828125f, 1.765625f, 1.296875f, 1.125f, 0.1435546875f, 0.56640625f, -0.9765625f, + 0.2197265625f, -0.6015625f, 0.26171875f, 0.291015625f, 0.027587890625f, 0.5234375f, + 0.013916015625f, -0.2158203125f, 0.6484375f, 1.1953125f, 0.10888671875f, -0.78515625f, + -0.2138671875f, -0.90625f, 0.0113525390625f, -0.470703125f, 1.8046875f, -1.3515625f, + 1.53125f, 1.578125f, 0.044189453125f, -1.359375f, -1.7890625f, -1.0234375f, 0.3515625f, + -0.3515625f, 0.302734375f, -0.01409912109375f, 0.70703125f, 0.72265625f, -0.78125f, + -0.408203125f, -0.271484375f, -0.765625f, -0.69140625f, 0.61328125f, 0.82421875f, + 1.140625f, -1.2578125f, -1.640625f, -1.640625f, -0.009765625f, 1.875f, 0.72265625f, + -0.232421875f, -0.40234375f, 0.79296875f, 0.259765625f, 1.5859375f, -0.392578125f, + -0.298828125f, -1.21875f, 3.3125f, 2.328125f, -1.6640625f, -0.70703125f, -1.34375f, + -1.3515625f, 2.0625f, -0.875f, 0.478515625f, 0.0f, -0.283203125f, -0.80859375f, + 0.62890625f, -0.27734375f, -1.1328125f, 0.65234375f, 0.6796875f, 0.0f, 0.388671875f, + 0.140625f, -0.69921875f, 0.373046875f, 0.232421875f, 0.59375f, 1.40625f, 1.25f, + -1.4765625f, -1.484375f, -0.57421875f, -1.6328125f, -0.6328125f, -0.376953125f, 0.75f, + 1.1953125f, 1.6171875f, 0.40625f, 0.51953125f, -0.921875f, -0.466796875f, 1.0f, -2.015625f, + 0.13671875f, 0.86328125f, -0.427734375f, 0.4453125f, 1.7265625f, 0.640625f, 0.5f, + -2.65625f, 1.0859375f, -0.25f, 0.859375f, -0.91796875f, 1.6640625f, -0.91796875f, + 0.181640625f, 1.2421875f, 0.017822265625f, 0.265625f, 1.5546875f, 0.36328125f, 0.6875f, + 0.4140625f, 0.0810546875f, 0.030517578125f, 0.74609375f, 0.400390625f, 0.51171875f, + 0.271484375f, 1.2421875f, -1.734375f, -1.8046875f, 0.640625f, 1.0078125f, -0.56640625f, + 1.359375f, -0.48828125f, -1.4921875f, 0.055908203125f, -0.83984375f, 1.1328125f, + -0.7265625f, 0.77734375f, 2.15625f, 1.2890625f, 0.16796875f, -1.1953125f, 1.4296875f, + 0.259765625f, 1.0390625f, 0.4453125f, 1.0625f, 1.234375f, -0.5078125f, 0.267578125f, + -0.3203125f, -0.142578125f, 0.4296875f, -1.109375f, 0.10498046875f, -1.015625f, 0.84375f, + -1.1484375f, -0.455078125f, -0.38671875f, -0.2109375f, 0.68359375f, 0.232421875f, + -0.53515625f, -0.498046875f, -0.107421875f, 0.07958984375f, 0.21875f, -0.27734375f, + 0.36328125f, -1.7578125f, -2.28125f, -0.4765625f, -0.349609375f, -0.16015625f, + 0.392578125f, 0.296875f, 0.921875f, 1.03125f, -0.93359375f, 1.4375f, -0.494140625f, + -1.8203125f, -0.1845703125f, 0.67578125f, -0.267578125f, -0.298828125f, 0.08251953125f, + -1.2109375f, 2.078125f, 1.15625f, -0.41015625f, -0.1708984375f, 0.083984375f, -0.67578125f, + 1.7421875f, -0.81640625f, 0.361328125f, 0.478515625f, 0.1689453125f, -0.103515625f, + 1.7734375f, 1.359375f, 1.1796875f, 0.306640625f, 0.0751953125f, 0.62109375f, + -0.1435546875f, 1.15625f, 1.0625f, 1.2265625f, -0.11376953125f, -0.447265625f, -0.328125f, + -0.09912109375f, -1.2109375f, -1.078125f, -1.6640625f, -0.7265625f, -1.2578125f, + -0.337890625f, 0.3984375f, -1.6171875f, -0.71484375f, -1.0859375f, -0.86328125f, + -1.3984375f, -0.09765625f, -0.5859375f, -0.171875f, -0.72265625f, -0.44921875f, + 0.92578125f, -0.6015625f, 0.9765625f, -1.4609375f, -1.0859375f, -0.83984375f, -0.25f, + 0.79296875f, 0.00043487548828125f, 0.69921875f, -1.703125f, 0.1845703125f, 0.447265625f, + 3.0f, -1.0078125f, 0.0194091796875f, 0.8984375f, -0.578125f, 2.34375f, 0.61328125f, + 0.703125f, 0.392578125f, 0.30859375f, 1.203125f, -1.15625f, -0.2490234375f, 0.55078125f, + 0.306640625f, -0.1572265625f, 0.83984375f, -1.5703125f, -0.875f, 0.201171875f, + 0.431640625f, -0.1796875f, 0.984375f, -0.8515625f, -1.6328125f, 1.4453125f, -1.234375f, + -0.703125f, -0.453125f, 0.48046875f, -0.423828125f, 0.72265625f, -0.126953125f, + -0.322265625f, -1.21875f, -1.4140625f, 0.52734375f, 1.9375f, -0.87109375f, -0.78515625f, + -0.2470703125f, 1.0625f, -0.345703125f, 0.0252685546875f, 0.95703125f, -0.1806640625f, + 0.609375f, 1.46875f, -1.3515625f, 0.50390625f, -0.048583984375f, -0.25f, -0.18359375f, + 0.12158203125f, -0.9609375f, 1.3203125f, 0.294921875f, 0.1123046875f, -0.875f, 0.37890625f, + -1.2890625f, 0.6171875f, 1.2265625f, 0.703125f, 2.34375f, 0.640625f, -0.8359375f, + -0.59375f, 1.1015625f, 0.1875f, 0.2490234375f, 0.37890625f, 0.71875f, 2.390625f, + 0.28515625f, -0.60546875f, 0.87890625f, 2.15625f, 0.353515625f, 1.53125f, 0.62109375f, + -0.07958984375f, 0.7421875f, 0.703125f, -0.37890625f, 0.3203125f, -0.306640625f, 1.78125f, + 1.3203125f, 0.388671875f, -1.3671875f, -0.1875f, -0.18359375f, -0.0242919921875f, + -1.421875f, -2.015625f, -0.47265625f, 1.75f, 0.6953125f, 0.0673828125f, 0.74609375f, + 0.345703125f, -0.048828125f, 0.169921875f, 2.203125f, 0.287109375f, -1.3125f, + 0.05908203125f, -0.54296875f, 0.87890625f, -0.5546875f, 1.3515625f, 0.9765625f, 1.9140625f, + -0.26953125f, 0.1103515625f, 0.37890625f, -0.63671875f, -0.734375f, -1.125f, 1.046875f, + 0.734375f, 1.4921875f, -0.0458984375f, 0.8359375f, -1.7734375f, -0.66015625f, -0.5f, + 1.5703125f, -1.546875f, -1.3984375f, 0.306640625f, 1.1875f, 0.05712890625f, -0.75390625f, + -0.3125f, -1.515625f, 0.27734375f, 0.005096435546875f, 0.734375f, 0.953125f, 0.953125f, + -0.65625f, 1.5859375f, -0.26171875f, 0.546875f, -0.8046875f, 0.9921875f, -1.2265625f, + 1.78125f, 0.76953125f, 0.19921875f, -0.4453125f, 0.314453125f, -0.1650390625f, 1.28125f, + 0.24609375f, -1.8515625f, -0.8671875f, 0.298828125f, 0.1826171875f, 0.0703125f, + 0.49609375f, -0.703125f, 1.40625f, 0.4140625f, 0.3203125f, 2.390625f, 0.212890625f, + 0.8125f, -1.1015625f, 0.177734375f, 0.1611328125f, 0.32421875f, -1.21875f, 0.0f, + 0.047607421875f, -0.62890625f, -0.6875f, 0.71484375f, 0.392578125f, 0.6953125f, 2.53125f, + -0.201171875f, -0.1396484375f, 1.1953125f, -0.48046875f, -0.037109375f, 1.546875f, + 1.203125f, 0.56640625f, 0.90625f, 0.16796875f, -1.2578125f, -0.474609375f, -0.421875f, + -0.9140625f, -0.66015625f, -0.41015625f, 0.1337890625f, 1.8671875f, -1.671875f, + 0.1865234375f, 0.55859375f, -0.8515625f, 0.06787109375f, 0.62109375f, -0.8203125f, + 0.138671875f, -1.8125f, -0.94921875f, 1.2734375f, -0.515625f, 0.1640625f, + -0.0230712890625f, 1.2578125f, 0.11865234375f, -1.4609375f, -2.46875f, -1.2734375f, + 0.59375f, -0.4296875f, 1.0078125f, 0.061767578125f, -0.2734375f, -1.2734375f, -0.796875f, + 0.6484375f, 0.07275390625f, 1.1171875f, 0.93359375f, -0.5859375f, 0.875f, 1.0625f, + -0.65625f, 0.18359375f, 0.7265625f, 0.6796875f, 0.162109375f, -0.828125f, 0.265625f, + 1.5078125f, 0.50390625f, 0.11669921875f, 0.8828125f, 1.078125f, -0.322265625f, 1.4375f, + -0.515625f, 0.236328125f, 1.609375f, 0.703125f, 0.93359375f, 1.0078125f, 0.8359375f, + 1.5859375f, 0.89453125f, 1.0625f, 1.3828125f, 0.0f, 0.091796875f, 0.8671875f, -0.83203125f, + 0.8203125f, 1.234375f, -0.310546875f, -0.408203125f, 2.03125f, 0.35546875f, 0.9765625f, + -0.38671875f, 0.294921875f, -0.48828125f, -0.984375f, -1.296875f, -1.1171875f, 0.9921875f, + -0.3515625f, 0.408203125f, 0.1904296875f, 0.431640625f, -0.58984375f, -0.9453125f, + -0.30859375f, 0.01251220703125f, -1.796875f, 0.11279296875f, -0.1044921875f, -0.7265625f, + 1.53125f, -1.2421875f, -0.380859375f, -0.12451171875f, 0.73828125f, -0.4375f, -0.87890625f, + 0.29296875f, -0.625f, -0.298828125f, 0.4375f, -0.5625f, -1.5859375f, 0.87890625f, + -0.1240234375f, 0.1865234375f, 0.7265625f, 0.15234375f, 1.765625f, 0.044921875f, + -0.6484375f, 0.83203125f, 0.6015625f, 0.7578125f, 0.193359375f, 0.98828125f, -0.61328125f, + 1.5f, 0.3828125f, 0.49609375f, -0.6796875f, -0.85546875f, 1.2890625f, -0.92578125f, + -0.412109375f, -0.6484375f, -0.27734375f, -0.80859375f, 1.671875f, 0.0f, -0.87109375f, + 1.2578125f, -1.0234375f, 0.146484375f, -0.0213623046875f, 0.51171875f, -0.91796875f, 0.0f, + -0.7578125f, 2.21875f, 0.84375f, 2.046875f, 0.123046875f, 0.1962890625f, 0.2275390625f, + 0.62109375f, 0.83984375f, 0.263671875f, -0.31640625f, -0.51171875f, -1.09375f, 0.546875f, + 0.5546875f, 0.3125f, 0.041259765625f, -2.625f, -0.11279296875f, -1.328125f, -1.546875f, + 1.0546875f, 0.06982421875f, -0.07080078125f, -1.515625f, -0.091796875f, 0.98828125f, + -0.162109375f, -0.72265625f, 1.8671875f, -0.69140625f, 0.052734375f, -0.59765625f, + 0.90234375f, 0.12158203125f, 2.3125f, 0.2578125f, 1.390625f, 0.1806640625f, 1.046875f, + 0.6875f, -1.1796875f, -1.203125f, -1.3828125f, -1.21875f, 1.6953125f, -1.796875f, + -0.302734375f, 0.9296875f, 0.0380859375f, 0.98828125f, -0.8671875f, -0.6171875f, + -1.0546875f, 0.055419921875f, -0.72265625f, 0.04345703125f, 0.1953125f, -0.5078125f, + 0.796875f, -0.2353515625f, 1.7734375f, -1.4765625f, -0.953125f, -0.146484375f, + -0.1826171875f, 0.6953125f, -0.259765625f, -0.9453125f, -0.306640625f, -1.4453125f, + 0.2158203125f, -1.0f, 0.26171875f, 1.5625f, 1.234375f, 1.8359375f, 0.053466796875f, + 1.078125f, 0.79296875f, 0.67578125f, -0.65234375f, 0.6953125f, 1.84375f, -0.53515625f, + -0.3671875f, -1.046875f, -0.2451171875f, 0.71875f, 0.6015625f, -0.28515625f, -0.52734375f, + 0.08984375f, -0.36328125f, -1.03125f, 0.208984375f, -0.51953125f, -0.11572265625f, + -1.890625f, -0.9140625f, 0.59765625f, 1.0078125f, 1.4140625f, 0.2177734375f, 0.1435546875f, + -1.421875f, -0.7421875f, -0.049560546875f, -1.0234375f, -1.578125f, 1.5625f, 1.453125f, + 1.3984375f, 0.11328125f, 0.51953125f, 1.0703125f, -0.69140625f, -0.90625f, -0.177734375f, + -1.90625f, -1.375f, -1.1875f, -1.2734375f, -1.203125f, 0.099609375f, -0.322265625f, 1.75f, + 1.5703125f, -1.0546875f, 0.42578125f, -0.058349609375f, 0.494140625f, 0.400390625f, + -0.455078125f, -0.0101318359375f, 0.494140625f, 0.8515625f, -2.328125f, -2.0625f, + -0.3671875f, -0.8046875f, 0.1796875f, 0.2177734375f, 1.28125f, -0.6171875f, 0.8359375f, + 0.06640625f, -0.80078125f, -0.11328125f +}; +inline constexpr float kOverBudgetAttnK[] = { + -0.466796875f, -0.1982421875f, 0.330078125f, -0.310546875f, 1.65625f, -0.60546875f, + 0.1591796875f, 0.2216796875f, 0.59765625f, -1.0859375f, -1.1171875f, 1.5703125f, + -0.107421875f, -1.3125f, 0.64453125f, -0.234375f, -0.58984375f, -0.80078125f, -1.1328125f, + -0.75390625f, 0.061279296875f, 0.2109375f, 1.5703125f, -1.15625f, -0.41015625f, 0.40625f, + 0.64453125f, 1.1796875f, -1.6328125f, 1.40625f, -0.62109375f, -0.255859375f, -2.0625f, + -1.9375f, -0.7265625f, -0.96875f, -0.86328125f, -1.25f, -1.421875f, -0.67578125f, + -0.16015625f, 0.0625f, -0.125f, 0.248046875f, -0.27734375f, -0.17578125f, 1.375f, + -2.140625f, -0.73046875f, -0.23828125f, 0.55859375f, -0.4375f, -0.12353515625f, + -0.080078125f, 0.625f, -1.1015625f, 1.1015625f, -1.3046875f, -0.58984375f, 0.52734375f, + -1.40625f, -0.91015625f, 0.2060546875f, 0.357421875f, -0.9609375f, -0.302734375f, 1.0f, + -0.408203125f, -1.25f, 1.3359375f, -0.462890625f, 0.2109375f, 1.609375f, 0.427734375f, + 0.1484375f, -0.65625f, -1.1953125f, 0.71484375f, -1.0078125f, 0.8515625f, -0.60546875f, + 0.625f, -0.66015625f, 1.8046875f, 0.283203125f, -0.55078125f, -0.21875f, -0.74609375f, + 0.3203125f, 0.7265625f, 0.021240234375f, 0.96484375f, -0.59375f, 0.53125f, -1.4453125f, + 0.380859375f, 1.078125f, 0.53515625f, -0.8359375f, -0.302734375f, 0.2373046875f, + 1.1171875f, -0.33203125f, -0.6171875f, -0.78125f, 1.0f, -0.80078125f, -0.55859375f, + 0.5390625f, 0.87109375f, -0.498046875f, 0.158203125f, -0.51953125f, -0.80078125f, + 0.30859375f, 0.53515625f, -0.55859375f, -0.302734375f, -0.6796875f, 1.5703125f, 1.3515625f, + -1.484375f, 0.69921875f, 0.62109375f, -3.0625f, 1.3671875f, 0.61328125f, -0.201171875f, + 0.46484375f, 0.41015625f, 0.9140625f, 0.11865234375f, -1.84375f, -0.1806640625f, 0.734375f, + 1.5390625f, -1.7734375f, -0.296875f, -0.39453125f, 0.0390625f, 0.50390625f, 1.2109375f, + -0.498046875f, -0.244140625f, 0.00482177734375f, 1.3984375f, -1.140625f, -0.150390625f, + -0.392578125f, 0.03564453125f, 0.162109375f, 0.796875f, -0.7578125f, 0.0f, -1.09375f, + -0.6953125f, -1.2421875f, -0.94921875f, 1.2890625f, -1.2421875f, -0.2080078125f, + 0.205078125f, -1.296875f, 0.49609375f, 0.08447265625f, 0.5f, -0.57421875f, 0.3671875f, + 0.6796875f, -0.37109375f, -0.73046875f, 0.03662109375f, -1.21875f, -1.90625f, 0.29296875f, + -0.32421875f, 1.8671875f, 0.83203125f, -1.3046875f, -1.5078125f, -1.234375f, + -0.2392578125f, -0.62109375f, -1.7890625f, 0.83203125f, -0.52734375f, -0.059814453125f, + -1.3125f, 0.625f, -0.337890625f, -1.3515625f, -0.63671875f, 1.0234375f, 0.2275390625f, + 0.90234375f, 1.8203125f, 1.34375f, -0.017578125f, 0.91015625f, -0.07861328125f, + 0.392578125f, 0.0625f, 0.86328125f, 0.9140625f, -0.9765625f, 0.765625f, 0.60546875f, + -0.039794921875f, -0.58984375f, 0.4375f, -1.1953125f, 0.380859375f, 1.4453125f, + 0.1650390625f, -0.2333984375f, -0.58203125f, -0.16015625f, 1.21875f, -0.3046875f, + -1.1484375f, -1.3515625f, 1.609375f, -0.2890625f, -0.6328125f, 1.6796875f, -0.2275390625f, + 0.68359375f, -0.279296875f, -0.58203125f, -0.98828125f, -0.76953125f, 1.34375f, + 0.94921875f, -0.72265625f, -1.234375f, 1.609375f, 0.48046875f, -1.984375f, 1.546875f, + -0.345703125f, 1.0859375f, 1.0625f, -0.07568359375f, 1.140625f, 1.3984375f, 0.6171875f, + -1.1875f, 1.9609375f, -0.0703125f, 1.875f, -0.240234375f, 0.54296875f, 0.20703125f, + -0.578125f, -0.421875f, -1.15625f, 0.796875f, 1.5859375f, -0.34765625f, 0.69921875f, + 0.984375f, 0.498046875f, -1.359375f, -2.078125f, 0.353515625f, -0.625f, 2.328125f, + -0.384765625f, 1.03125f, 1.5078125f, -0.3671875f, 1.7734375f, -2.03125f, 1.234375f, + 0.197265625f, -0.05615234375f, -0.1455078125f, 1.515625f, 0.2275390625f, 0.91796875f, + 1.4140625f, 0.62109375f, -1.4921875f, 0.80078125f, -0.96484375f, 0.9609375f, 0.2275390625f, + 0.58203125f, -1.7265625f, 3.140625f, 1.5859375f, 0.91796875f, -0.40625f, 1.390625f, + -0.1982421875f, 0.58203125f, 0.0556640625f, -1.125f, 0.84765625f, 0.2294921875f, + -0.703125f, -0.267578125f, -1.0859375f, -0.0927734375f, 0.75390625f, -0.734375f, + 0.376953125f, 0.51171875f, 0.90234375f, -0.63671875f, 0.234375f, 1.234375f, 0.478515625f, + -0.30078125f, -0.6328125f, 1.328125f, -0.423828125f, -1.515625f, -0.5859375f, -0.27734375f, + -0.365234375f, 2.28125f, -2.0625f, 0.67578125f, -0.671875f, -1.6015625f, 0.193359375f, + 1.6796875f, -0.86328125f, -0.5859375f, 1.578125f, 2.71875f, -0.6640625f, 1.1953125f, + 0.78125f, -0.322265625f, -0.28515625f, 0.4765625f, -0.89453125f, -0.65625f, 0.349609375f, + -0.0947265625f, 0.9921875f, -0.32421875f, 0.365234375f, -0.345703125f, -1.5546875f, + -2.078125f, 0.2734375f, -1.078125f, 0.5625f, -0.1826171875f, -0.236328125f, 0.62109375f, + 1.4765625f, -0.73046875f, -1.0625f, -2.0625f, -0.48046875f, -1.5f, -0.75f, 1.609375f, + 0.21875f, 0.70703125f, -1.7109375f, -1.046875f, 0.0155029296875f, -1.0390625f, -1.7890625f, + -0.1806640625f, 1.140625f, 0.291015625f, 0.89453125f, 0.640625f, 1.578125f, -0.62109375f, + -0.30859375f, 1.78125f, -1.0f, 2.3125f, -1.3984375f, -1.34375f, 0.353515625f, + -0.462890625f, 0.072265625f, 0.40234375f, 0.451171875f, -0.6875f, -0.44140625f, -1.03125f, + 1.40625f, 0.00616455078125f, -0.71875f, -0.7265625f, -0.04345703125f, 0.3203125f, + -0.81640625f, -0.5234375f, 0.54296875f, -0.96875f, -0.5234375f, -0.625f, -2.0f, + 0.12060546875f, -0.66015625f, 1.1796875f, -0.96875f, -1.6328125f, -1.2421875f, -0.8203125f, + -0.54296875f, 0.8046875f, 0.462890625f, 1.1796875f, -0.65625f, -0.640625f, -0.02099609375f, + 1.4140625f, -1.2421875f, 1.3984375f, -1.0625f, -1.3125f, 1.109375f, -1.0390625f, -0.84375f, + -0.091796875f, 1.125f, 0.79296875f, -0.234375f, 1.2734375f, 0.81640625f, -1.28125f, + 1.578125f, -1.703125f, 0.216796875f, 0.11328125f, -0.337890625f, 0.427734375f, -1.0625f, + -2.125f, -0.1513671875f, 0.703125f, 0.12890625f, 1.59375f, -0.4453125f, 0.9140625f, + 1.359375f, 0.76171875f, 1.1796875f, 0.96875f, 0.177734375f, -1.5625f, 0.05615234375f, + 1.4453125f, 0.609375f, -1.2890625f, 0.08740234375f, 0.0f, -1.765625f, 0.349609375f, + -1.4921875f, 0.76953125f, 0.060302734375f, -1.6953125f, -1.4375f, -0.1728515625f, + -0.87890625f, -1.0625f, 1.34375f, -0.47265625f, 0.37109375f, -0.5390625f, 0.59375f, + -0.376953125f, -1.4140625f, -2.3125f, -1.1875f, -1.671875f, 0.90234375f, -0.7578125f, + -0.671875f, 0.388671875f, -1.7578125f, -0.5234375f, -1.7265625f, 0.1513671875f, + 0.2333984375f, -1.9765625f, -1.59375f, -0.16796875f, -0.21875f, 0.71875f, + 0.00167083740234375f, 1.4765625f, 0.388671875f, 1.6796875f, 1.0234375f, -0.443359375f, + -0.6328125f, 0.25f, 1.7890625f, -0.328125f, -1.796875f, 0.65234375f, -0.6953125f, + -0.76953125f, -1.3828125f, 1.0078125f, -0.400390625f, 1.4765625f, 0.55078125f, -0.78125f, + -0.88671875f, 0.82421875f, -0.26171875f, -0.5f, -0.92578125f, 0.67578125f, -0.48828125f, + -1.2109375f, -1.25f, -0.053466796875f, -0.62109375f, -0.70703125f, 0.25390625f, + -0.7109375f, 0.66796875f, 0.392578125f, -1.640625f, 1.03125f, 0.166015625f, -0.26171875f, + -0.11181640625f, -0.53125f, 1.6953125f, 1.1796875f, -1.34375f, 0.23046875f, 0.953125f, + -1.4296875f, -0.2890625f, -0.921875f, -2.03125f, 0.388671875f, 0.8359375f, -0.388671875f, + -0.5f, 0.0306396484375f, 0.453125f, -0.08740234375f, 0.7109375f, -0.8046875f, -1.40625f, + -0.625f, 0.70703125f, 1.28125f, -0.423828125f, 0.19921875f, -1.3828125f, 2.875f, + -0.97265625f, 1.453125f, 0.5234375f, -1.265625f, 0.60546875f, -0.5546875f, -0.86328125f, + -0.0703125f, 1.46875f, 0.82421875f, -2.0f, -0.58984375f, 0.58203125f, 1.2578125f, + -0.33984375f, -0.38671875f, 0.91796875f, -0.2001953125f, -0.208984375f, 0.4375f, + -0.5546875f, 1.65625f, -0.984375f, -0.98828125f, -0.91796875f, 0.65234375f, -1.1484375f, + 0.306640625f, -0.64453125f, 1.1640625f, 2.546875f, 0.50390625f, 1.234375f, 0.66796875f, + 1.875f, -0.578125f, 0.54296875f, -0.306640625f, 2.578125f, 1.0625f, -1.15625f, 1.4296875f, + -1.03125f, -0.1474609375f, 0.345703125f, -2.578125f, 0.19140625f, 0.322265625f, + 0.69140625f, 0.400390625f, 1.3359375f, -0.33203125f, -1.65625f, 0.330078125f, 0.35546875f, + 1.421875f, -1.078125f, 0.46484375f, -3.0625f, 0.66796875f, -0.076171875f, 0.36328125f, + -0.8125f, 0.10546875f, 0.1923828125f, -1.15625f, 0.287109375f, -0.03759765625f, -1.0625f, + -0.232421875f, 0.25390625f, 0.043701171875f, -1.171875f, 0.890625f, 0.234375f, -1.5078125f, + -0.21875f, -0.73828125f, 0.1787109375f, -1.1640625f, -1.1953125f, -0.78515625f, + -1.3359375f, 1.6484375f, -1.375f, 1.7265625f, -0.09814453125f, 0.515625f, 0.703125f, + 0.1123046875f, -0.69921875f, 1.15625f, -1.359375f, 0.474609375f, 1.3984375f, 0.466796875f, + 0.8046875f, 1.5703125f, -0.349609375f, -1.578125f, -0.9609375f, -2.265625f, -2.765625f, + 2.328125f, 0.65625f, -0.201171875f, 0.79296875f, -1.421875f, 0.031005859375f, -1.1875f, + 1.40625f, -0.8359375f, -1.8671875f, -0.69921875f, -0.5703125f, 0.578125f, -1.9140625f, + -1.3359375f, -0.51171875f, -0.53515625f, -0.0859375f, -1.046875f, -1.2421875f, -0.578125f, + 1.125f, -0.361328125f, -0.322265625f, 0.0172119140625f, -0.75f, -0.54296875f, -1.515625f, + 0.1806640625f, -1.8984375f, -1.9921875f, 0.23046875f, -0.28125f, 0.3515625f, -0.5390625f, + 0.59765625f, 0.50390625f, -0.609375f, 0.06396484375f, -1.0078125f, 0.44140625f, 1.421875f, + -0.0439453125f, 0.26953125f, 1.140625f, 0.1669921875f, -0.2431640625f, -1.5546875f, + -1.015625f, -0.2255859375f, -0.369140625f, -0.5234375f, 1.390625f, -1.015625f, + -0.232421875f, 0.00150299072265625f, 0.27734375f, 1.3125f, -0.62109375f, -0.953125f, + -0.390625f, 0.7890625f, -0.8828125f, -0.2119140625f, -0.166015625f, 0.400390625f, + -0.2001953125f, -0.1337890625f, 0.1396484375f, -0.33984375f, -0.068359375f, 0.7578125f, + -0.431640625f, -1.3125f, 0.439453125f, 0.220703125f, -0.984375f, 0.12451171875f, + 0.5859375f, 1.7421875f, -0.2216796875f, 0.1044921875f, -0.73828125f, -0.12109375f, + 0.2197265625f, 0.828125f, 1.7734375f, 1.921875f, 1.125f, -0.26953125f, 0.625f, 0.80859375f, + -1.2109375f, -0.439453125f, -0.55859375f, 0.671875f, -0.00286865234375f, 0.408203125f, + -1.046875f, -2.015625f, 0.71484375f, -0.76171875f, -0.0169677734375f, -0.48828125f, 0.125f, + 0.265625f, 1.3125f, -0.68359375f, 0.388671875f, 0.4140625f, -0.9296875f, 1.140625f, + 0.0277099609375f, 0.419921875f, 0.328125f, 0.34765625f, -0.23046875f, 1.0078125f, + 0.333984375f, 0.25390625f, 1.15625f, 1.671875f, 0.455078125f, 0.9765625f, 0.3359375f, + 1.8046875f, 0.8515625f, 0.11572265625f, -0.19140625f, 1.6015625f, 0.435546875f, -1.171875f, + -1.015625f, -0.287109375f, -0.3671875f, 0.134765625f, 0.357421875f, 0.5078125f, 0.984375f, + -0.361328125f, 0.1484375f, 0.173828125f, -1.0f, -0.8359375f, 0.41796875f, 0.26953125f, + -0.8125f, -0.85546875f, 0.44921875f, -0.3984375f, 0.56640625f, 1.8984375f, 0.4609375f, + -1.0390625f, -0.5546875f, 0.67578125f, 0.470703125f, -0.52734375f, -0.796875f, 1.625f, + 1.328125f, 0.2314453125f, -0.11376953125f, 3.03125f, -1.796875f, -1.515625f, 0.83984375f, + -0.765625f, -1.5f, -0.478515625f, 1.640625f, 0.953125f, 0.8125f, -1.15625f, 0.154296875f, + 0.71875f, 1.0234375f, 1.9453125f, -0.7109375f, -1.078125f, -0.380859375f, 0.055419921875f, + -0.19921875f, 0.361328125f, -0.7421875f, 0.0595703125f, 0.609375f, -0.12158203125f, + 0.6796875f, -0.07275390625f, -0.8828125f, -0.578125f, 1.78125f, -1.5859375f, -0.73046875f, + 0.6171875f, 0.828125f, -0.3984375f, 1.0546875f, 0.3671875f, 1.125f, 0.68359375f, 0.96875f, + 0.80859375f, -0.130859375f, 0.9296875f, -0.068359375f, 0.048583984375f, 0.193359375f, + -0.8828125f, 2.203125f, -0.5859375f, -0.4140625f, -1.5625f, 0.97265625f, -0.369140625f, + 0.224609375f, 0.443359375f, 1.6875f, -0.427734375f, 0.35546875f, -0.443359375f, + -1.4296875f, 0.7734375f, 1.53125f, 0.87109375f, -0.2158203125f, 1.53125f, 0.48046875f, + -0.625f, -1.4140625f, 1.359375f, 0.81640625f, -0.1728515625f, -0.68359375f, 0.11376953125f, + -0.7578125f, -1.5859375f, -1.3671875f, 0.09326171875f, 0.94921875f, 0.419921875f, + -0.84375f, -0.49609375f, 0.341796875f, -0.83203125f, -1.3046875f, 0.470703125f, + 0.94921875f, 0.43359375f, 0.65625f, 1.359375f, 1.125f, -0.1826171875f, -0.7734375f, + -0.90234375f, 0.345703125f, -0.78125f, -0.6953125f, 0.1669921875f, -0.486328125f, + 0.4140625f, -0.0234375f, -3.203125f, -0.13671875f, -2.125f, -1.0625f, 1.3359375f, + -1.6015625f, -0.2138671875f, -0.5703125f, 0.75390625f, 1.0546875f, -1.203125f, 0.62109375f, + 0.98828125f, -0.328125f, 0.400390625f, 0.73046875f, -0.96875f, 0.3203125f, 0.039306640625f, + -1.9375f, -0.91015625f, 0.69140625f, -0.51953125f, 2.015625f, -1.2421875f, -1.09375f, + -1.046875f, -0.98046875f, 0.48046875f, 2.265625f, -1.6484375f, -1.375f, 0.703125f, + -1.546875f, -1.0703125f, 0.41015625f, 0.251953125f, 1.5234375f, -0.52734375f, -0.546875f, + -0.859375f, 0.087890625f, 0.5f, 0.765625f, -0.578125f, 1.2421875f, 0.404296875f, + 0.216796875f, -0.337890625f, 0.69921875f, 0.3046875f, -0.07568359375f, -0.33203125f, + -0.6015625f, 0.7890625f, -0.2216796875f, 0.69921875f, 0.439453125f, 1.7421875f, + 0.384765625f, -0.57421875f, -0.4609375f, 0.6171875f, -1.21875f, 0.1748046875f, -0.640625f, + -1.0546875f, 0.388671875f, -0.2373046875f, 0.1640625f, -1.328125f, -1.328125f, -1.0859375f, + 0.58203125f, 1.046875f, -0.28125f, -1.5703125f, -0.625f, -0.1240234375f, 2.078125f, + 0.640625f, 1.3203125f, -1.0625f, 0.5234375f, -0.291015625f, 1.453125f, -0.58203125f, + -0.158203125f, -0.29296875f, -0.1396484375f, 0.90234375f, -0.76171875f, -0.30859375f, + -1.28125f, 0.30859375f, 1.5546875f, 0.3203125f, -0.765625f, -0.203125f, -0.06298828125f, + 0.08544921875f, -0.2138671875f, 0.162109375f, -0.2138671875f, -0.1591796875f, -0.79296875f, + -1.1171875f, 0.50390625f, 0.0213623046875f, -0.1337890625f, 0.65625f, -0.1337890625f, + 0.4140625f, -1.046875f, -0.21484375f, -0.90234375f, -1.0546875f, 0.640625f, -1.5234375f, + -0.125f, -0.578125f, -1.7109375f, -0.6171875f, 0.671875f, 0.494140625f, 1.0703125f, + -1.03125f, 0.00012111663818359375f, 1.625f, -0.796875f, 1.1015625f, -0.453125f, 1.375f, + 0.2138671875f, 0.7578125f, 0.2578125f, 1.8671875f, 1.9296875f, -0.4765625f, 0.09423828125f, + 0.0f, 1.6953125f, 1.1953125f, 0.671875f, -0.12109375f, 0.482421875f, 0.384765625f, + -0.263671875f, -0.0390625f, -0.9921875f, -1.25f, -0.6796875f, 0.2138671875f, 0.25f, + -1.1640625f, -0.306640625f, -0.6953125f, -0.4609375f, 0.2578125f, 0.71875f, 1.6953125f, + -0.5234375f, 1.3359375f, 2.40625f, 0.671875f, 0.396484375f, 0.0263671875f, 1.640625f, + 0.734375f, 0.8046875f, -0.212890625f, -1.75f, 2.453125f, 0.359375f, 0.1513671875f, + -0.7109375f, -1.4296875f, 2.09375f, -1.4453125f, 1.4140625f, 0.171875f, -0.034912109375f, + -2.21875f, -1.7421875f, 0.546875f, 2.59375f, 0.59375f, -0.9609375f, 0.134765625f, + -0.62109375f, 1.5f, 1.21875f, 0.82421875f, -0.84375f, -0.26171875f, 1.3671875f, + -0.205078125f, 0.2490234375f, 0.90625f, -0.72265625f, -0.8203125f, 2.21875f, 0.63671875f, + -1.3671875f, 0.3828125f, -1.5234375f, -2.46875f, 0.0233154296875f, 1.375f, 0.9140625f, + -0.283203125f, -0.65625f, -1.3828125f, 0.275390625f, -0.7578125f, -0.48828125f, -1.09375f, + 1.25f, -1.5546875f, 0.50390625f, -0.12890625f, 0.765625f, -0.4296875f, 1.59375f, + -0.1357421875f, 0.37890625f, -0.19921875f, 0.045654296875f, -0.48828125f, -0.2333984375f, + -0.515625f, -1.0390625f, 0.1708984375f, -1.0f, 0.2392578125f, -0.66015625f, -0.484375f, + -0.330078125f, -1.6328125f, -0.703125f, -0.87109375f, 0.90625f, -0.2890625f, -0.2265625f, + -0.111328125f, -0.73046875f, 0.050537109375f, -0.035400390625f, 0.5703125f, -1.1640625f, + -0.75f, -1.8984375f, -0.056396484375f, 0.625f, -0.193359375f, -0.40234375f, -0.60546875f, + -0.5390625f, -0.040771484375f, -0.486328125f, 1.078125f, -1.1171875f, -0.5703125f, + -1.5546875f, -0.419921875f, 0.74609375f, 0.40234375f, -0.8984375f, -1.078125f, + -0.458984375f, -0.494140625f, 0.12353515625f, -0.62890625f, 1.9375f, -0.173828125f, + -0.041259765625f, -0.2314453125f, 0.39453125f, -0.27734375f, 0.1484375f, -1.09375f, + -0.166015625f, 0.038818359375f, 1.796875f, 0.828125f, -0.201171875f, 0.78125f, 0.44921875f, + 0.1904296875f, -0.431640625f, 1.7890625f, 1.3125f, -0.1513671875f, 2.296875f, + -0.357421875f, -0.302734375f, -0.55078125f, 0.66015625f, -0.640625f, 0.78515625f, + 0.76953125f, 1.21875f, -0.6640625f, 0.84765625f, -1.203125f, -1.390625f, 1.4921875f, + 1.0703125f, -1.46875f, 2.421875f, -0.625f, -1.515625f, 0.91796875f, 0.41015625f, 0.75f, + 0.2138671875f, 1.125f, -0.1572265625f, -1.1640625f, -0.3359375f, -0.91796875f, 1.0625f, + 1.4375f, 0.5625f, 1.3359375f, -0.037841796875f, 0.42578125f, -0.84375f, -0.421875f, 0.0f, + 0.396484375f, -1.2890625f, 0.40625f, -0.431640625f, 1.90625f, -1.109375f, -0.84375f, 0.0f, + -0.0294189453125f, 2.3125f, -0.07177734375f, -0.8828125f, -0.458984375f, -1.421875f, + 1.0390625f, 1.015625f, -1.1796875f, -1.3671875f, -0.474609375f, -0.5f, 3.078125f, + 0.5078125f, -0.337890625f, -0.59765625f, 0.62109375f, 0.515625f, -1.234375f, 0.66796875f, + 0.302734375f, 0.51953125f, 0.2275390625f, 0.79296875f, 0.484375f, 0.69921875f, 1.859375f, + -1.53125f, -1.84375f, 1.4453125f, 0.5546875f, 0.1376953125f, 0.67578125f, 0.76171875f, + 0.0380859375f, -0.7109375f, -0.22265625f, 0.000762939453125f, -0.4453125f, 1.140625f, + -0.76171875f, 0.83984375f, -1.015625f, 0.546875f, -1.4765625f, 1.578125f, 0.16015625f, + -1.6640625f, -0.462890625f, 0.057861328125f, -0.05517578125f, 0.376953125f, -1.7421875f, + 1.75f, -0.44921875f, -0.6328125f, -1.6328125f, -1.5390625f, -2.21875f, 1.3828125f, + 0.921875f, -0.224609375f, -0.11376953125f, 1.640625f, -0.1005859375f, 1.2421875f, + 0.055908203125f, 1.2421875f, -0.76953125f, 0.72265625f, -0.3359375f, -2.375f, -1.7890625f, + 1.3671875f, -1.484375f, 0.55078125f, 0.51953125f, 0.92578125f, -0.390625f, 0.53125f, + 0.625f, 0.1904296875f, 1.375f, 1.1640625f, 0.83984375f, -0.34765625f, 0.369140625f, + -2.59375f, 1.515625f, 0.6953125f, -1.0625f, 1.0078125f, -0.1337890625f, -0.1552734375f, + -2.25f, 0.1240234375f, 1.4921875f, 0.6015625f, 1.0078125f, -1.7578125f, 0.330078125f, + 0.306640625f, -0.1328125f, -1.203125f, 0.1474609375f, 0.423828125f, 0.416015625f, + -0.69140625f, 1.078125f, 1.375f, -1.1171875f, -0.8984375f, -0.58203125f, -1.609375f, + -0.10546875f, 0.63671875f, 0.267578125f, -0.494140625f, 0.59375f, 0.82421875f, 0.34375f, + 0.81640625f, -0.111328125f, 0.578125f, 0.5234375f, 1.203125f, 0.376953125f, 1.296875f, + -1.5390625f, -1.265625f, -1.3671875f, 1.0078125f, 1.265625f, -0.76171875f, 1.4765625f, + -0.7421875f, 1.3046875f, 1.375f, 0.6171875f, -1.140625f, 0.71484375f, -0.8046875f, + 0.369140625f, -1.765625f, 0.92578125f, -0.486328125f, 0.5390625f, 1.4453125f, 0.47265625f, + -0.32421875f, 0.63671875f, 0.037109375f, -0.177734375f, -0.068359375f, -0.2734375f, + -1.3515625f, -0.53125f, 0.98046875f, -0.71875f, 0.51953125f, -0.2578125f, 0.2890625f, + 1.140625f, 0.6875f, 0.94921875f, 0.703125f, -0.33203125f, -0.765625f, -1.703125f, + 0.06494140625f, -0.828125f, 0.65625f, 0.77734375f, -0.416015625f, -0.953125f, 0.98046875f, + 1.0546875f, -0.267578125f, 0.69921875f, -1.7578125f, -1.3203125f, 1.1328125f, + 0.2333984375f, 0.34375f, 1.0078125f, -0.376953125f, -1.6015625f, 0.79296875f, + -0.0400390625f, 0.8359375f, 0.1318359375f, -0.1884765625f, 1.1796875f, -0.4765625f, + 0.080078125f, -1.265625f, -0.2490234375f, -1.7578125f, 0.8671875f, -1.015625f, + -0.51171875f, 1.9375f, 1.125f, 0.671875f, 0.2255859375f, -1.1015625f, -0.62890625f, + 0.03271484375f +}; +inline constexpr float kOverBudgetAttnV[] = { + 0.134765625f, 1.15625f, 1.171875f, 0.38671875f, -0.21484375f, 0.494140625f, -1.0625f, + -1.9453125f, 0.263671875f, -2.09375f, -0.1494140625f, 0.271484375f, -1.8046875f, + 1.6171875f, 0.62109375f, 1.140625f, -0.462890625f, -1.4453125f, 1.1796875f, -0.2734375f, + -1.78125f, -0.59765625f, -0.189453125f, 1.109375f, 0.65234375f, -0.875f, -1.984375f, + -1.8046875f, -1.546875f, 0.69140625f, -0.412109375f, -0.859375f, -0.2333984375f, + -1.0546875f, -0.27734375f, 0.66796875f, -0.5078125f, 0.421875f, -0.1396484375f, + 0.09228515625f, 0.46875f, 0.1005859375f, -0.0478515625f, 0.1494140625f, 2.0625f, + -0.4765625f, -1.5859375f, 0.61328125f, 1.078125f, -1.2578125f, -0.78125f, -0.59375f, + -0.796875f, -0.79296875f, -0.9296875f, 0.322265625f, -0.50390625f, 1.0703125f, 0.75390625f, + -1.4140625f, -0.44921875f, -0.21484375f, -0.298828125f, -1.4140625f, 0.62109375f, + -0.66015625f, -1.6015625f, 0.53515625f, 0.53515625f, 0.11474609375f, -0.3203125f, + 0.890625f, -0.80078125f, -0.5078125f, -1.6953125f, -0.23046875f, 1.1484375f, -1.140625f, + -0.39453125f, 1.203125f, 1.578125f, -0.228515625f, 2.546875f, 1.6015625f, -1.2578125f, + 0.66015625f, 0.3984375f, 0.41796875f, -1.8984375f, -1.25f, 0.443359375f, -0.75f, + 1.8984375f, 0.29296875f, 0.326171875f, 0.89453125f, 0.75f, -0.177734375f, 0.15625f, + 0.0986328125f, -0.2255859375f, 0.1484375f, -0.28515625f, 1.328125f, -2.046875f, -2.015625f, + 0.47265625f, 2.078125f, -1.234375f, -0.90234375f, -0.1494140625f, 0.4375f, 1.1640625f, + 0.08349609375f, -0.74609375f, 0.5234375f, -2.109375f, 1.546875f, 0.74609375f, 0.8515625f, + 0.3203125f, -2.21875f, 0.1669921875f, -2.0f, -0.6796875f, -0.5546875f, -0.322265625f, + -1.953125f, 0.15234375f, 0.66015625f, -0.48828125f, 0.53515625f, 0.271484375f, -1.7890625f, + 1.0f, -0.0732421875f, 1.2109375f, -0.02197265625f, 0.66796875f, 1.3046875f, -0.1728515625f, + 1.3359375f, 1.6640625f, -1.3046875f, -0.859375f, -0.60546875f, 0.9375f, 0.89453125f, + 0.61328125f, 0.61328125f, -0.5078125f, 0.6484375f, 0.2197265625f, 0.7265625f, 0.234375f, + -1.296875f, 0.328125f, -0.6953125f, 0.609375f, 0.75390625f, 0.83203125f, -0.33203125f, + 0.70703125f, -0.126953125f, -1.1484375f, -0.8984375f, -0.046142578125f, -0.875f, -2.5f, + 0.361328125f, 1.0625f, 1.015625f, 0.0011138916015625f, -1.1796875f, -0.8203125f, 0.671875f, + -0.5859375f, -0.2451171875f, 0.4296875f, -0.314453125f, 0.73046875f, -2.640625f, + 0.80078125f, 0.9765625f, 1.2265625f, -0.4921875f, 0.2158203125f, 0.212890625f, 1.1015625f, + 1.390625f, -1.25f, -0.1552734375f, -0.671875f, 0.27734375f, 0.46875f, 1.765625f, + 0.451171875f, -0.0751953125f, 0.333984375f, 0.2138671875f, 1.3515625f, -1.4140625f, + 1.0625f, -1.5546875f, -0.8125f, 1.0703125f, 0.032958984375f, -0.81640625f, -0.2265625f, + -1.21875f, 0.193359375f, -1.375f, -0.0888671875f, -0.75390625f, -0.57421875f, 0.96484375f, + 0.263671875f, -0.73828125f, 2.71875f, 0.04443359375f, -0.125f, 0.875f, 0.48828125f, + -0.248046875f, 0.93359375f, -1.8359375f, 0.56640625f, 0.328125f, 1.078125f, 0.29296875f, + 0.044921875f, 1.53125f, -0.40234375f, 1.1640625f, 0.234375f, -0.25390625f, -0.2412109375f, + 0.306640625f, 0.146484375f, 0.76953125f, -0.1533203125f, -0.2255859375f, -0.306640625f, + 0.5859375f, -0.921875f, 0.2138671875f, 1.8046875f, 0.70703125f, -2.734375f, -0.64453125f, + -0.61328125f, -1.140625f, -0.412109375f, 0.04248046875f, -0.7109375f, 0.123046875f, + -0.435546875f, -1.15625f, -1.28125f, -1.234375f, -0.9921875f, -1.203125f, 0.361328125f, + -0.125f, -0.431640625f, 1.6328125f, -1.390625f, -0.375f, 0.35546875f, -0.265625f, + 0.7734375f, 1.2265625f, 0.984375f, 0.8515625f, -0.7578125f, 0.2236328125f, 0.1103515625f, + 0.33203125f, 1.109375f, -1.109375f, -0.423828125f, -0.74609375f, -0.51171875f, -2.21875f, + -0.8359375f, -0.90625f, 1.8515625f, -1.0625f, -0.63671875f, 1.4375f, -0.1201171875f, + 0.201171875f, -1.6796875f, -1.3984375f, 1.2890625f, -0.498046875f, -2.5625f, 1.0625f, + 0.69140625f, 0.5234375f, -0.87890625f, -0.173828125f, 0.6875f, 1.6328125f, -0.84375f, + 1.5625f, -0.7109375f, 0.205078125f, -1.5234375f, -0.26171875f, 1.0703125f, -1.71875f, + -2.0f, -1.875f, -0.703125f, 2.875f, 0.8046875f, 0.68359375f, 0.4765625f, -1.5f, + 0.2431640625f, 0.4921875f, 0.75390625f, 0.0478515625f, 1.8671875f, 0.5625f, 0.4609375f, + -0.1181640625f, 1.09375f, -0.40625f, 0.0f, 2.0f, 1.0546875f, -0.5625f, 0.90625f, -0.375f, + 0.3671875f, 0.55859375f, -2.578125f, 0.9453125f, 0.36328125f, -1.2421875f, -0.2177734375f, + 1.140625f, 1.0234375f, 0.392578125f, -1.1484375f, -0.9453125f, 1.09375f, -0.443359375f, + 0.00021076202392578125f, 0.1689453125f, 0.328125f, -0.98046875f, 0.341796875f, + -0.77734375f, -0.41015625f, 0.73046875f, 1.40625f, -1.640625f, 0.69921875f, 1.8359375f, + 0.77734375f, -0.34765625f, 1.2421875f, 2.203125f, 0.54296875f, 0.58984375f, 2.421875f, + 1.046875f, 1.703125f, -0.97265625f, 1.890625f, 1.8515625f, -1.2890625f, 1.21875f, + -1.4296875f, -0.06787109375f, 1.703125f, 0.384765625f, 1.0703125f, 1.1015625f, -2.578125f, + 1.4765625f, -0.193359375f, -0.349609375f, -0.64453125f, -1.8046875f, -0.84765625f, + 0.169921875f, -1.15625f, 0.34375f, -0.388671875f, -1.6171875f, -0.84765625f, 2.171875f, + 0.2314453125f, 0.0517578125f, 1.046875f, 0.95703125f, -1.03125f, -0.65625f, 0.5546875f, + -0.470703125f, -1.1328125f, 0.341796875f, -0.28515625f, -0.921875f, 0.93359375f, + -1.8828125f, 0.1240234375f, -0.1923828125f, -0.052001953125f, -1.7421875f, 0.59765625f, + 0.890625f, 1.8203125f, 2.109375f, 1.0859375f, 0.000797271728515625f, 0.7265625f, -1.0f, + -1.4921875f, -0.0269775390625f, -0.060546875f, -0.26953125f, 1.0859375f, 1.6484375f, + 0.053955078125f, 0.41015625f, -0.045166015625f, 0.58203125f, 0.322265625f, -0.58984375f, + -0.828125f, -2.1875f, -1.609375f, -0.11474609375f, 1.640625f, 0.482421875f, 0.291015625f, + 0.08447265625f, 0.703125f, 1.0234375f, -0.91015625f, -2.046875f, -0.2099609375f, 2.171875f, + 0.18359375f, -0.69921875f, 0.5546875f, 0.1005859375f, 1.65625f, 0.9921875f, 0.08544921875f, + 0.5f, -0.171875f, 1.953125f, -0.035888671875f, -0.146484375f, 0.04052734375f, -0.25390625f, + -0.6484375f, 0.7890625f, 1.296875f, -1.5234375f, -0.91796875f, -1.203125f, 0.84765625f, + -0.4609375f, -0.625f, -0.515625f, 0.86328125f, 0.466796875f, 0.330078125f, 0.208984375f, + 0.50390625f, 1.6484375f, 2.234375f, -0.4921875f, 0.73828125f, 1.5078125f, -2.71875f, + -1.2734375f, -0.11865234375f, 0.4453125f, -0.125f, -0.578125f, 0.2431640625f, 0.71484375f, + -0.734375f, -0.220703125f, -2.109375f, -0.16015625f, 0.447265625f, 0.404296875f, + -0.423828125f, -0.60546875f, -0.828125f, -0.5703125f, -1.2109375f, -0.01708984375f, + -0.08642578125f, -2.421875f, 0.6640625f, -0.02783203125f, -0.251953125f, -0.494140625f, + 0.54296875f, 0.08642578125f, -0.47265625f, -0.49609375f, -0.51171875f, 1.53125f, + -1.015625f, -0.2099609375f, -0.21484375f, 1.3203125f, 0.56640625f, -0.341796875f, + -1.1171875f, -1.2578125f, 0.146484375f, -0.6015625f, -0.302734375f, 0.703125f, + -0.384765625f, -1.8671875f, 1.2578125f, -0.6953125f, 0.83984375f, 0.90234375f, + 0.248046875f, -0.390625f, 0.34375f, 0.0f, -0.796875f, 0.2490234375f, -0.4609375f, + -1.4140625f, -1.0859375f, 0.453125f, -0.265625f, 0.0f, 0.12109375f, 1.328125f, -1.046875f, + -2.140625f, -0.53125f, -0.2275390625f, 0.470703125f, 0.1376953125f, -0.0311279296875f, + -1.1640625f, -0.318359375f, 0.373046875f, -0.435546875f, 0.51953125f, 0.369140625f, + -0.52734375f, -0.263671875f, -0.62890625f, 0.890625f, -1.4296875f, 0.384765625f, + -0.83984375f, 0.29296875f, -0.26171875f, -1.0625f, -0.88671875f, -0.890625f, 2.15625f, + 0.875f, 1.765625f, -0.494140625f, -0.345703125f, 0.71484375f, -0.267578125f, 0.373046875f, + 0.7109375f, -0.59765625f, 0.427734375f, 1.390625f, -0.279296875f, -1.203125f, 1.796875f, + -0.9296875f, -0.75f, -0.244140625f, -0.66796875f, 0.57421875f, 1.4140625f, 0.80078125f, + 2.34375f, 1.0078125f, -0.75390625f, 0.67578125f, -1.3046875f, -1.0703125f, -0.07568359375f, + 0.80078125f, -0.15234375f, 0.099609375f, 1.140625f, 0.3828125f, -1.171875f, -1.1640625f, + 0.271484375f, -1.265625f, -0.0220947265625f, -0.5078125f, 0.55859375f, 1.28125f, + -0.765625f, 0.1904296875f, 0.671875f, 0.16015625f, 0.1767578125f, -0.87890625f, + -0.94140625f, -0.328125f, 0.109375f, -0.1376953125f, -1.828125f, -0.1435546875f, + -0.0869140625f, 0.12109375f, 0.4296875f, -1.703125f, 0.53125f, 2.578125f, 0.1435546875f, + -0.2890625f, 0.5f, 0.23828125f, -2.1875f, 1.0f, 0.9921875f, 0.78125f, 1.140625f, + 0.19921875f, 1.1484375f, 0.8984375f, -1.0f, -1.6484375f, -0.765625f, 1.046875f, + -0.2158203125f, -0.2392578125f, -0.62890625f, 0.478515625f, 1.3671875f, -0.73828125f, + 0.91796875f, -1.1875f, 0.7734375f, 0.045654296875f, 0.14453125f, 0.06689453125f, 1.03125f, + 0.67578125f, 0.474609375f, 0.416015625f, -0.96875f, 0.23046875f, 0.328125f, -0.341796875f, + 0.546875f, 1.328125f, 0.333984375f, -0.875f, 1.0234375f, 0.63671875f, 0.625f, 0.703125f, + 0.69921875f, -1.484375f, -0.72265625f, 0.82421875f, 1.8203125f, 0.3203125f, -0.080078125f, + 0.8515625f, 0.57421875f, -0.255859375f, 1.25f, -0.486328125f, 0.26953125f, -1.5f, -1.4375f, + 0.125f, 0.9921875f, 0.2001953125f, 0.0419921875f, 0.9921875f, 0.036376953125f, + -0.69921875f, -0.462890625f, 0.0f, -0.87109375f, 0.375f, -0.2138671875f, -1.1171875f, + 1.5234375f, -1.0078125f, 1.5078125f, -0.2119140625f, -0.68359375f, -0.11474609375f, + -0.1103515625f, -0.796875f, 0.4921875f, 0.361328125f, 0.8515625f, -1.1640625f, + -0.96484375f, -0.36328125f, -0.60546875f, -0.061767578125f, 0.123046875f, 1.203125f, + -0.5078125f, 1.5703125f, -2.328125f, 0.984375f, 0.81640625f, -0.1796875f, -0.341796875f, + 2.125f, -2.34375f, -0.62109375f, 0.59765625f, -0.251953125f, -0.81640625f, 1.765625f, + 0.07080078125f, 1.7421875f, -0.35546875f, -0.21875f, 0.56640625f, 2.34375f, 0.79296875f, + 1.9609375f, -1.4609375f, 0.0f, 0.10888671875f, 0.30078125f, -0.53125f, 0.466796875f, + 0.1171875f, 0.9296875f, -0.91796875f, -0.8671875f, -0.55859375f, 0.46875f, -0.29296875f, + 0.77734375f, 1.3125f, 0.083984375f, -2.5625f, 0.85546875f, -0.09619140625f, -0.5625f, + 2.875f, -0.25390625f, -1.2265625f, 0.55859375f, -0.82421875f, 0.267578125f, -1.4765625f, + -0.287109375f, -0.1328125f, 1.453125f, -0.193359375f, -0.01092529296875f, -1.3125f, + -0.419921875f, 0.69140625f, -1.5703125f, 1.3046875f, -1.0546875f, -1.28125f, + -0.12451171875f, 0.74609375f, 1.1484375f, -0.011962890625f, 0.353515625f, -0.12890625f, + -0.8828125f, -0.431640625f, 0.23828125f, 1.0625f, 0.1416015625f, -0.48046875f, 0.984375f, + -0.70703125f, 0.154296875f, 0.35546875f, -1.8046875f, 0.1845703125f, 1.9375f, 0.5546875f, + 2.015625f, -0.373046875f, -1.5546875f, -0.326171875f, 0.2265625f, 1.3203125f, -0.91015625f, + -0.400390625f, -0.06689453125f, 1.046875f, 0.12353515625f, 0.08349609375f, 0.1669921875f, + 0.8359375f, -0.44140625f, 1.6328125f, -0.2060546875f, 0.427734375f, 1.0546875f, + -0.9296875f, -0.34375f, 0.291015625f, 0.52734375f, -1.34375f, 0.92578125f, -1.6328125f, + -1.265625f, 0.419921875f, -1.375f, -0.2109375f, 1.546875f, 0.97265625f, -1.6953125f, + -0.01507568359375f, 0.70703125f, -1.4453125f, 0.0341796875f, 0.421875f, -0.84765625f, + 1.1328125f, 0.376953125f, -0.60546875f, -0.84765625f, -1.1796875f, -0.90625f, 0.466796875f, + 0.72265625f, 0.265625f, -0.458984375f, 0.0257568359375f, -0.7265625f, -0.095703125f, + 0.671875f, 0.337890625f, -0.11474609375f, 0.400390625f, -0.064453125f, 0.54296875f, + 0.55859375f, 0.55078125f, -1.21875f, -0.19921875f, -1.640625f, 0.84765625f, -1.40625f, + -0.412109375f, 0.15625f, -1.1328125f, 0.294921875f, -0.640625f, -0.4453125f, -0.275390625f, + -1.484375f, 1.34375f, 0.703125f, -1.2265625f, 0.130859375f, 1.9296875f, 1.828125f, + -1.09375f, 0.8203125f, 0.30078125f, 0.734375f, -0.9609375f, 1.8671875f, 1.328125f, + -0.291015625f, -1.3515625f, -0.18359375f, -0.75f, 0.466796875f, 0.73828125f, 0.71875f, + -2.078125f, 0.7734375f, -0.275390625f, -0.44140625f, -1.15625f, -1.0546875f, 1.8125f, + -0.83984375f, -0.4296875f, -0.07421875f, -0.875f, 0.4375f, 0.86328125f, -0.181640625f, + 1.3359375f, 0.56640625f, 0.251953125f, -0.287109375f, -0.396484375f, 1.0546875f, + -0.169921875f, 0.310546875f, -1.0859375f, 0.7109375f, -1.5859375f, -0.4609375f, 0.7421875f, + 0.1826171875f, -0.78515625f, -0.51953125f, 0.76171875f, -0.7109375f, 0.431640625f, + 1.2578125f, 0.26953125f, -1.828125f, -0.169921875f, 1.3359375f, 0.4453125f, 0.734375f, + 0.984375f, -0.2001953125f, -0.81640625f, 0.58984375f, -0.1630859375f, -0.90625f, 2.546875f, + -0.94140625f, -0.28515625f, 0.1005859375f, -0.62109375f, 0.02197265625f, 0.90625f, + 0.037353515625f, -0.87109375f, -0.197265625f, 1.1796875f, -0.61328125f, -1.0f, + 0.462890625f, 0.11181640625f, 1.5625f, 0.028076171875f, -1.65625f, -0.5078125f, + -1.2734375f, -0.51171875f, 1.25f, 0.52734375f, -0.326171875f, -0.3203125f, 0.1650390625f, + 0.96875f, 0.263671875f, 0.859375f, 0.2177734375f, -0.59375f, -0.7734375f, -0.39453125f, + -1.0f, 1.125f, -0.17578125f, -0.040283203125f, -1.1171875f, 1.1015625f, 0.392578125f, + -1.28125f, -0.61328125f, 0.25390625f, -0.9609375f, 1.7578125f, -0.8359375f, -0.4765625f, + 0.41015625f, -0.50390625f, -0.0281982421875f, -0.45703125f, -2.75f, -0.3203125f, + -0.255859375f, 0.17578125f, -0.055908203125f, 0.86328125f, -0.96484375f, -0.2333984375f, + -0.56640625f, 0.671875f, -1.3984375f, -0.59375f, -0.306640625f, -1.2578125f, -0.26171875f, + -1.2734375f, -0.1435546875f, -1.3984375f, 0.8359375f, -0.6328125f, 1.2734375f, 0.86328125f, + -0.68359375f, -0.291015625f, 0.6484375f, 0.8203125f, 0.8359375f, -0.6875f, 0.56640625f, + -0.1103515625f, -1.375f, -0.921875f, 1.515625f, 0.640625f, 0.0830078125f, + 0.01055908203125f, 1.28125f, 0.2412109375f, -0.5859375f, 0.5234375f, -1.421875f, + -0.123046875f, 1.1640625f, -1.6640625f, -0.0830078125f, 1.0859375f, -0.208984375f, + -0.5234375f, -1.84375f, 1.3984375f, -0.89453125f, -1.0703125f, 0.08740234375f, 0.74609375f, + 0.251953125f, 1.2890625f, -0.171875f, 0.171875f, -0.54296875f, 0.60546875f, 0.19921875f, + -0.74609375f, 1.1328125f, -1.2890625f, 2.265625f, 1.359375f, -0.59765625f, -0.6796875f, + 1.734375f, -0.76171875f, -1.5390625f, -1.015625f, -1.34375f, -0.74609375f, -1.0390625f, + 1.0234375f, -0.75f, 1.1953125f, -0.1923828125f, -0.78125f, 0.3359375f, 0.51171875f, + -1.4140625f, 0.15625f, -0.5078125f, -0.177734375f, -1.375f, -0.3046875f, 1.5f, + -0.74609375f, 0.83203125f, 1.0390625f, 0.357421875f, -0.08984375f, -1.3125f, 0.416015625f, + 0.205078125f, 0.58203125f, 0.6015625f, 0.1943359375f, 0.138671875f, 1.4375f, 0.2890625f, + -1.890625f, -0.51171875f, -0.1689453125f, 0.76953125f, -0.408203125f, 0.453125f, + -1.734375f, 0.212890625f, 0.392578125f, 2.265625f, -0.44140625f, 0.1533203125f, + -0.1201171875f, -0.93359375f, -0.054443359375f, 0.9453125f, -0.1376953125f, 0.16796875f, + 0.52734375f, 0.0f, -1.0078125f, 0.796875f, -0.96875f, -0.150390625f, -1.5703125f, + 0.1513671875f, -0.216796875f, 0.10302734375f, 0.58984375f, -0.8203125f, 0.828125f, + -0.392578125f, 0.271484375f, 0.1298828125f, -0.515625f, 0.1923828125f, 1.640625f, + -0.11572265625f, 1.8828125f, 1.9921875f, 0.48046875f, -0.52734375f, -0.3046875f, + -0.6015625f, -0.671875f, 1.046875f, -0.53125f, 0.34765625f, 0.2890625f, 0.091796875f, + -0.2275390625f, -1.5625f, 0.3828125f, 0.208984375f, -0.14453125f, 0.01708984375f, + 0.14453125f, -1.0859375f, 0.609375f, 1.0234375f, -0.75390625f, -0.359375f, 1.171875f, + 0.1494140625f, -0.265625f, 0.76171875f, -1.515625f, 0.6796875f, 1.515625f, -1.140625f, + 0.451171875f, -1.1328125f, 1.7734375f, 0.216796875f, -0.62109375f, 0.3046875f, + 0.2490234375f, 0.1923828125f, -1.140625f, 0.734375f, -1.5546875f, -0.0208740234375f, + 0.169921875f, -0.55078125f, -0.03955078125f, -0.478515625f, 0.369140625f, 0.09033203125f, + -0.0712890625f, 0.73828125f, -1.1640625f, 0.65234375f, 0.1865234375f, -0.267578125f, + 0.3203125f, -0.70703125f, -0.90234375f, -1.078125f, -2.015625f, -0.9453125f, 1.2421875f, + 1.203125f, 0.039306640625f, -0.76953125f, 0.953125f, -0.90234375f, 0.10009765625f, + -0.55078125f, -0.1328125f, 0.087890625f, 1.3515625f, 0.041015625f, 1.03125f, 0.875f, + -1.4453125f, 0.70703125f, 1.890625f, -0.00848388671875f, 0.2353515625f, -1.09375f, + 0.2060546875f, -2.3125f, 0.431640625f, 0.400390625f, 0.76953125f, -1.4140625f, + -0.86328125f, -0.423828125f, -0.298828125f, 1.8984375f, -1.3046875f, -1.3515625f, + -1.390625f, 0.55859375f, 0.08203125f, 1.7265625f, -0.94921875f, -1.0859375f, 1.8671875f, + 0.8203125f, -0.158203125f, -1.5546875f, -0.38671875f, -0.1572265625f, -0.283203125f, + -0.80859375f, 0.0f, -0.92578125f, -2.8125f, 0.55859375f, -0.369140625f, -0.6328125f, + -1.3125f, -0.48828125f, 1.59375f, 1.140625f, 0.47265625f, -1.9296875f, -0.765625f, + -0.494140625f, 0.515625f, -0.484375f, -0.25390625f, 2.4375f, -0.0157470703125f, + 0.52734375f, -0.10791015625f, -0.30859375f, -0.328125f, -1.90625f, 0.337890625f, + -1.8984375f, 0.1455078125f, -0.38671875f, -0.06982421875f, -0.79296875f, -1.6484375f, + -0.484375f, -0.31640625f, 0.61328125f, 1.421875f, -0.671875f, 1.5078125f, -0.13671875f, + 1.796875f, -2.5f, 0.98046875f, -0.1845703125f, 1.234375f, -0.294921875f, 0.66796875f, + 0.318359375f, -1.296875f, -0.6796875f, 0.46484375f, -0.52734375f, -1.484375f, + -0.291015625f, -0.9140625f, 0.828125f, -0.416015625f, 1.234375f, 0.52734375f, -0.6953125f, + -0.2275390625f, -0.65234375f, 0.8671875f, 0.310546875f, 0.41015625f, -0.6796875f, + 1.0390625f, 0.1904296875f, 0.921875f, -2.0625f, -0.375f, 0.15625f, 0.4765625f, 0.578125f, + -1.9765625f, -1.890625f, -0.9375f, 1.4765625f, 1.2421875f, -1.1328125f, -0.9921875f, + -0.84375f, -0.77734375f, 0.421875f, -0.11669921875f, 1.2109375f, 0.703125f, 1.3671875f, + -0.69140625f, 0.78125f, -1.1328125f, -1.75f, 0.0f, 1.4765625f, -1.625f, -0.99609375f, + -0.57421875f, -0.337890625f, 0.71875f, -0.91796875f, 0.0f, -0.48046875f, -0.578125f, + 0.44921875f, -0.388671875f, -0.080078125f, -1.125f, -0.484375f, -0.146484375f, -1.921875f, + 1.0625f, 0.19140625f, -0.00408935546875f, -0.91015625f, 0.25f, -0.435546875f, 1.171875f, + -0.6171875f, 0.05224609375f, -1.453125f, 0.08837890625f, -0.87109375f, 1.359375f, + 1.484375f, 0.03564453125f, 1.2421875f, -1.015625f, 0.34375f, 0.49609375f, 0.6484375f, + -0.74609375f, -0.875f, -0.94921875f, -1.40625f, 0.2109375f, 1.96875f, -0.359375f, + 0.59765625f, -0.828125f, 0.283203125f, -0.7890625f, -0.06640625f, -0.6796875f, 0.55859375f, + 0.318359375f, 0.765625f, 0.703125f, -0.703125f, -1.1171875f, 2.890625f, -0.61328125f, + -1.28125f, -0.23046875f, 0.2177734375f, -0.72265625f, -1.0234375f, -1.3359375f, -0.65625f, + -0.431640625f, -0.71484375f, -0.287109375f, 1.2421875f, -0.953125f, -0.69140625f, + 0.7578125f, 0.306640625f, -0.4140625f, -1.2421875f, -0.21484375f, 0.64453125f, + -0.216796875f, 0.8515625f, 0.287109375f, -2.734375f, 0.34375f, -0.26171875f, -0.4296875f, + 1.3828125f, 0.60546875f, -0.703125f, 0.48046875f, -0.0830078125f, 0.1416015625f, + -0.828125f, 0.470703125f, -1.390625f, 0.0849609375f, -1.4453125f, -0.2177734375f, -0.375f, + -0.2333984375f, 1.0234375f, -0.1494140625f, -1.140625f, -0.37109375f, -1.5703125f, + 1.7421875f, -0.81640625f, 1.5703125f, 0.310546875f, -0.8203125f, 0.1376953125f, + -1.5390625f, 2.09375f, -0.39453125f, 1.859375f, 0.90625f, -0.1669921875f, 1.1953125f, + 1.34375f, 1.6875f, 0.99609375f, -1.25f, 1.2421875f, -1.3203125f, 0.37890625f, + 0.1181640625f, -0.953125f, -1.578125f, 0.041259765625f, -0.70703125f, 0.365234375f, + -1.8125f, 0.09033203125f, -0.08984375f, 1.3046875f, 0.2255859375f, 0.87109375f, -1.546875f, + 0.64453125f, -0.490234375f, 0.71875f, 0.71875f +}; +inline constexpr float kOverBudgetAttnOut[] = { + 0.134765625f, 1.15625f, 1.171875f, 0.38671875f, -0.21484375f, 0.494140625f, -1.0625f, + -1.9453125f, 0.263671875f, -2.09375f, -0.1494140625f, 0.271484375f, -1.8046875f, + 1.6171875f, 0.62109375f, 1.140625f, -0.462890625f, -1.4453125f, 1.1796875f, -0.2734375f, + -1.78125f, -0.59765625f, -0.189453125f, 1.109375f, 0.65234375f, -0.875f, -1.984375f, + -1.8046875f, -1.546875f, 0.69140625f, -0.412109375f, -0.859375f, 0.134765625f, 1.15625f, + 1.171875f, 0.38671875f, -0.21484375f, 0.494140625f, -1.0625f, -1.9453125f, 0.263671875f, + -2.09375f, -0.1494140625f, 0.271484375f, -1.8046875f, 1.6171875f, 0.62109375f, 1.140625f, + -0.462890625f, -1.4453125f, 1.1796875f, -0.2734375f, -1.78125f, -0.59765625f, + -0.189453125f, 1.109375f, 0.65234375f, -0.875f, -1.984375f, -1.8046875f, -1.546875f, + 0.69140625f, -0.412109375f, -0.859375f, -0.2333984375f, -1.0546875f, -0.27734375f, + 0.66796875f, -0.5078125f, 0.421875f, -0.1396484375f, 0.09228515625f, 0.46875f, + 0.1005859375f, -0.0478515625f, 0.1494140625f, 2.0625f, -0.4765625f, -1.5859375f, + 0.61328125f, 1.078125f, -1.2578125f, -0.78125f, -0.59375f, -0.796875f, -0.79296875f, + -0.9296875f, 0.322265625f, -0.50390625f, 1.0703125f, 0.75390625f, -1.4140625f, + -0.44921875f, -0.21484375f, -0.298828125f, -1.4140625f, -0.2333984375f, -1.0546875f, + -0.27734375f, 0.66796875f, -0.5078125f, 0.421875f, -0.1396484375f, 0.09228515625f, + 0.46875f, 0.1005859375f, -0.0478515625f, 0.1494140625f, 2.0625f, -0.4765625f, -1.5859375f, + 0.61328125f, 1.078125f, -1.2578125f, -0.78125f, -0.59375f, -0.796875f, -0.79296875f, + -0.9296875f, 0.322265625f, -0.50390625f, 1.0703125f, 0.75390625f, -1.4140625f, + -0.44921875f, -0.21484375f, -0.298828125f, -1.4140625f, 0.47405505180358887f, + -0.11097557842731476f, -0.763028621673584f, 0.490276962518692f, 0.3083977997303009f, + 0.22945398092269897f, -0.5447089076042175f, 0.03319453075528145f, -0.47894957661628723f, + -0.9873121976852417f, -1.2279183864593506f, -0.07870592176914215f, 0.2555760443210602f, + -0.3068152368068695f, -0.08746249228715897f, 1.1842284202575684f, 0.961034893989563f, + -0.5964075922966003f, 2.1335132122039795f, 1.0346662998199463f, -1.4160709381103516f, + 0.2798634171485901f, 0.22069193422794342f, 0.6270117163658142f, -1.1272225379943848f, + -1.1366207599639893f, -0.2906530499458313f, -1.0688791275024414f, 0.8567658066749573f, + 0.4134341776371002f, 0.10295651108026505f, 0.3642471432685852f, 0.4083267152309418f, + 0.13451586663722992f, -0.38819220662117004f, 0.4702153205871582f, 0.20703357458114624f, + 0.2807300090789795f, -0.6450172662734985f, -0.35008886456489563f, -0.33508631587028503f, + -1.2016552686691284f, -1.018986701965332f, -0.010865814052522182f, -0.14354552328586578f, + 0.06590943038463593f, 0.049801528453826904f, 1.175781488418579f, 0.6851869821548462f, + -0.7608605027198792f, 1.9487347602844238f, 0.781255841255188f, -1.4868147373199463f, + 0.10986717790365219f, 0.14123716950416565f, 0.7204568386077881f, -0.7824786305427551f, + -1.0859386920928955f, -0.6187669038772583f, -1.2114224433898926f, 0.3911239206790924f, + 0.4672839045524597f, 0.003176120575517416f, 0.12720268964767456f, 0.43528303503990173f, + -0.4583856463432312f, 0.01748701184988022f, 0.28083735704421997f, -0.3159068822860718f, + 0.2359456717967987f, -0.23858939111232758f, 0.9326192140579224f, -1.2417995929718018f, + -1.3383740186691284f, 0.30607813596725464f, 1.4608796834945679f, -0.1792762279510498f, + -0.7660809755325317f, -0.6091445684432983f, 0.4937552511692047f, 1.1365598440170288f, + -0.34576281905174255f, -0.7573447823524475f, 0.16590403020381927f, -1.689335584640503f, + 0.7980548739433289f, 0.20979353785514832f, 0.682171642780304f, 0.05653780698776245f, + -1.1661514043807983f, 0.3548222482204437f, -1.8124823570251465f, -0.6059305667877197f, + -0.44592729210853577f, -0.31476491689682007f, -1.7806087732315063f, 0.22664250433444977f, + -0.6444424390792847f, -0.07450544089078903f, 0.401629239320755f, -0.37578484416007996f, + 0.2939589321613312f, -0.20771804451942444f, 0.6704199910163879f, -0.7080776691436768f, + -0.8893929123878479f, 0.1956457793712616f, 1.051679015159607f, 0.5201978087425232f, + -0.6757460832595825f, -0.9139213562011719f, 0.5310494899749756f, 1.1183271408081055f, + -0.6303386092185974f, -0.7648035883903503f, -0.07112157344818115f, -1.410871982574463f, + 0.30162718892097473f, -0.14574483036994934f, 0.5698745250701904f, -0.11833073198795319f, + -0.4683348536491394f, 0.47934359312057495f, -1.6881682872772217f, -0.5570336580276489f, + -0.3738251328468323f, -0.30979233980178833f, -1.666239857673645f, 0.2066759169101715f, + 0.5941851735115051f, -0.30179935693740845f, 0.5062173008918762f, 0.20916059613227844f, + -1.1093486547470093f, 0.4352119266986847f, -0.3194490373134613f, 0.7783779501914978f, + -0.4857461154460907f, 0.21741147339344025f, 0.9140956401824951f, -0.3281816244125366f, + 1.0856084823608398f, 1.2070690393447876f, -0.5189440846443176f, -0.4817298352718353f, + -0.7227545976638794f, 1.1830227375030518f, 0.7539470791816711f, -0.08410581201314926f, + 0.38297584652900696f, -0.33407828211784363f, 0.7099024653434753f, 0.043069370090961456f, + 0.17077480256557465f, -0.17243598401546478f, -1.3284912109375f, 0.15607285499572754f, + -0.30318576097488403f, 0.3753328323364258f, 0.4567127525806427f, 0.33170419931411743f, + 0.4024735689163208f, -0.017267605289816856f, 0.4528855085372925f, 0.10830988734960556f, + 0.2444254457950592f, -0.6757704615592957f, -0.7219847440719604f, -0.12564684450626373f, + -1.3662422895431519f, -0.7384559512138367f, 0.11272744089365005f, -0.5442543625831604f, + 0.49272462725639343f, 0.2555248439311981f, 1.0623159408569336f, 0.34371015429496765f, + -0.9188776016235352f, 1.720984935760498f, 0.5324207544326782f, -1.4687118530273438f, + -0.03896450623869896f, 0.03423576056957245f, 0.8109341263771057f, -0.39500677585601807f, + -0.9584798812866211f, -0.911047637462616f, -1.3577073812484741f, -0.07758928090333939f, + 0.47196164727211f, -0.07102734595537186f, -0.08351591974496841f, 0.3741045594215393f, + -0.5599268674850464f, 0.09681771695613861f, 0.27946701645851135f, -0.528964102268219f, + 0.04141208156943321f, -0.17765603959560394f, 0.3793342113494873f, -1.1355547904968262f, + -0.6792417168617249f, 0.3865734338760376f, 1.0881736278533936f, 0.33785563707351685f, + -0.7895475029945374f, -0.8618556261062622f, 0.5561354160308838f, 0.7682297825813293f, + -0.5202028751373291f, -0.5172528624534607f, -0.0959341749548912f, -0.9984315633773804f, + -0.2528539001941681f, 0.08796176314353943f, 0.6659444570541382f, 0.1782664656639099f, + -0.5481976866722107f, 0.41153496503829956f, -1.3088102340698242f, -0.2196630984544754f, + -0.017079859972000122f, -0.5045480132102966f, -1.3663963079452515f, 0.16933761537075043f, + -0.7758832573890686f, 0.08199786394834518f, 0.37189990282058716f, -0.7265130281448364f, + -0.05182454735040665f, -0.11052808165550232f, -0.21618878841400146f, -0.6474175453186035f, + 0.13585536181926727f, 0.35832110047340393f, 0.5071682929992676f, 1.2458486557006836f, + -0.7363520860671997f, -1.2769291400909424f, 0.629257082939148f, 0.492657870054245f, + -0.864091694355011f, -0.3525823950767517f, -0.46509647369384766f, -0.29263758659362793f, + -1.3827574253082275f, -0.27345505356788635f, 0.5675880908966064f, 0.12945817410945892f, + 0.4300861060619354f, 0.5480841398239136f, -0.8551300764083862f, 0.09235423058271408f, + 0.3431086242198944f, -0.6354274153709412f, -0.9839130640029907f, 0.16279438138008118f, + 0.48640987277030945f, -0.382101446390152f, 0.6077641844749451f, 0.2647174894809723f, + -1.0717170238494873f, 0.48056066036224365f, -0.13741832971572876f, 0.7838989496231079f, + -0.4649313986301422f, 0.2208297699689865f, 0.7189646363258362f, -0.21904386579990387f, + 0.9612715840339661f, 1.085821270942688f, -0.5625867247581482f, -0.3775884211063385f, + -0.6981537938117981f, 1.1667653322219849f, 0.6891887784004211f, -0.037844184786081314f, + 0.3654223084449768f, -0.329415500164032f, 0.6940869092941284f, -0.056199487298727036f, + 0.10273804515600204f, 0.17043764889240265f, -1.1753740310668945f, 0.30421721935272217f, + -0.24678577482700348f, 0.42609721422195435f, 0.4969249963760376f, 0.2381237894296646f, + 0.17383143305778503f, -0.389445036649704f, 0.6613097190856934f, 0.27780428528785706f, + -0.11965811997652054f, -0.21671301126480103f, -0.1256142258644104f, 0.10707104951143265f, + -0.9798447489738464f, -0.5178970098495483f, -0.0225969385355711f, -0.1315222680568695f, + 0.3092314302921295f, 0.27465301752090454f, 0.5031219720840454f, 0.38346144556999207f, + -0.753150999546051f, 1.5957435369491577f, 0.5764309763908386f, -0.9354037642478943f, + 0.12112552672624588f, -0.04026445746421814f, 0.7166034579277039f, -0.5697980523109436f, + -0.7561395168304443f, 0.06000256538391113f, -1.0173636674880981f, 0.43616464734077454f, + 0.31778407096862793f, 0.19469037652015686f, 0.24266548454761505f, 0.7753372192382812f, + -0.5537744164466858f, 0.5837891101837158f, -0.0025598055217415094f, -0.7463773488998413f, + -0.5782663226127625f, -0.06053636595606804f, -0.3206349015235901f, -2.0179548263549805f, + 0.2443559318780899f, 0.840481162071228f, 0.8935456871986389f, -0.014954330399632454f, + -0.924224317073822f, -0.6838684678077698f, 0.6607766151428223f, -0.2822277843952179f, + -0.2738989591598511f, 0.1631188839673996f, -0.14355628192424774f, 0.18493282794952393f, + -1.806471824645996f, 0.8228521347045898f, 0.8928399085998535f, 0.5398259162902832f, + -0.5682981014251709f, 0.13711988925933838f, -0.24633270502090454f, 0.663433849811554f, + 0.9563626646995544f, -1.0452879667282104f, -0.3482591211795807f, 0.5975536108016968f, + -0.9994922280311584f, 0.30448171496391296f, 0.27076074481010437f, -0.011542700231075287f, + 0.061783380806446075f, -0.08820026367902756f, 0.6810502409934998f, -0.9591189622879028f, + 0.06535796821117401f, 0.37940818071365356f, 0.594258725643158f, 0.13539427518844604f, + -0.42916345596313477f, -0.5151833295822144f, 0.6414196491241455f, 0.3413390517234802f, + -0.4066840410232544f, -0.3723381757736206f, 0.11882087588310242f, -0.831160306930542f, + -0.286484032869339f, 0.7325819730758667f, 0.699409008026123f, -0.775926411151886f, + -0.5512104034423828f, 0.040594689548015594f, -1.130698800086975f, -0.1865195482969284f, + 0.11221662908792496f, -0.6329623460769653f, -0.7464855313301086f, 0.01991533488035202f, + 0.41498780250549316f, 0.08901869505643845f, 0.36449187994003296f, -0.06346466392278671f, + -0.3093266189098358f, -0.20247434079647064f, -0.7329149842262268f, 0.3551332950592041f, + -0.8902612924575806f, -0.20289167761802673f, 0.11218143999576569f, -0.7142385244369507f, + 0.9081801772117615f, 0.6616804599761963f, 0.44764596223831177f, -0.06983847171068192f, + -0.7845343947410583f, 0.9013829231262207f, 0.1186349168419838f, -0.7801446318626404f, + -0.09738517552614212f, -0.04438623785972595f, 0.6056398749351501f, 0.0705261155962944f, + -0.567663311958313f, -0.5032230615615845f, -1.4236836433410645f, -0.5062397122383118f, + 0.1651943027973175f, 0.30269551277160645f, -0.2925921380519867f, -0.015622163191437721f, + -0.5091054439544678f, -1.0515079498291016f, -0.2065974473953247f, -0.22749054431915283f, + -0.8608382344245911f, 0.25857046246528625f, 0.1174304261803627f, -0.09637736529111862f, + 0.4555607736110687f, -0.9059667587280273f, -0.015362851321697235f, 0.34355154633522034f, + -0.05645901709794998f, 0.602322518825531f, 0.6095126867294312f, 0.6677166223526001f, + 0.09312863647937775f, 0.600730836391449f, 0.6571090817451477f, -0.23097018897533417f, + 0.40441539883613586f, 0.4678586423397064f, -0.16873829066753387f, -0.6303695440292358f, + -0.5834164023399353f, -0.011418472044169903f, -1.5189650058746338f, 0.15905404090881348f, + -0.4079771339893341f, 1.0459486246109009f, -0.11084601283073425f, 0.17090848088264465f, + -0.3374492824077606f, 0.08801740407943726f, 0.3073859214782715f, -0.48875516653060913f, + -0.2205064594745636f, 0.23673680424690247f, 0.4029284417629242f, -1.2088459730148315f, + 0.13512353599071503f, 0.3842678368091583f, 0.6275655627250671f, 0.03716502711176872f, + -0.38768887519836426f, -0.31986144185066223f, 0.8712137937545776f, 0.24508847296237946f, + 0.010752437636256218f, -0.5745989680290222f, 0.08831017464399338f, -1.1628601551055908f, + -0.11851046234369278f, 0.5812955498695374f, 0.034497663378715515f, -1.018424391746521f, + -0.8035429120063782f, -0.04205707088112831f, -0.2693580985069275f, -0.055367179214954376f, + 0.11266741901636124f, -0.2564525902271271f, -1.156250238418579f, 0.20457269251346588f, + -0.2124498039484024f, 0.2258652299642563f, 0.21414901316165924f, -0.7545115947723389f, + -0.5400846004486084f, 0.33704712986946106f, -0.07339006662368774f, -1.5489825010299683f, + 0.5768868923187256f, 0.5683711767196655f, 0.5126123428344727f, 0.06175060570240021f, + -0.47302308678627014f, -0.3325941860675812f, 0.9528923034667969f, -0.19835840165615082f, + 0.10350494831800461f, -0.3150406777858734f, -0.028513707220554352f, -0.6235178709030151f, + -0.9283886551856995f, 0.7153098583221436f, 0.0013013286516070366f, -0.7952079772949219f, + -0.6788737773895264f, -0.09734447300434113f, 0.3631483018398285f, 0.38084229826927185f, + 0.5617910027503967f, -0.4268997013568878f, -0.8118834495544434f, 0.12096042931079865f, + -0.06324604153633118f, -0.6424999237060547f, 0.2117903232574463f, 0.08113209158182144f, + -0.5863171815872192f, 0.11479856818914413f, -0.053156156092882156f, 0.13965082168579102f, + -0.22920268774032593f, -0.5761309266090393f, 0.19477631151676178f, 0.0903497189283371f, + 0.23285315930843353f, 0.5889185667037964f, 0.38051724433898926f, 0.3825658857822418f, + -0.31855612993240356f, 0.9500046372413635f, 0.6581434011459351f, -0.4504004418849945f, + 0.24965476989746094f, 0.14603526890277863f, 0.32159674167633057f, -0.4539792835712433f, + -0.501499593257904f, -0.06956649571657181f, -1.302467942237854f, 0.28031405806541443f, + -0.13515722751617432f, 0.6035289168357849f, 0.10712101310491562f, 0.03865453600883484f, + 0.4355746805667877f, 0.22656802833080292f, 0.7641245126724243f, 0.216810941696167f, + 0.14421679079532623f, -0.3725733160972595f, -0.5566855072975159f, 0.35774871706962585f, + -1.3667746782302856f, -0.16846142709255219f, -0.22962138056755066f, -0.6680963039398193f, + 0.7257286310195923f, 0.28322479128837585f, 0.49067947268486023f, 0.10225027054548264f, + -0.9795578122138977f, 1.1003317832946777f, -0.02409190870821476f, -1.0278419256210327f, + -0.30079084634780884f, -0.1549203097820282f, 0.8706437945365906f, -0.06833754479885101f, + -0.8169984221458435f, -0.04720878601074219f, -1.0267726182937622f, -0.1900322288274765f, + 0.5355435609817505f, 0.07385143637657166f, -0.18389670550823212f, 0.5565211176872253f, + -0.6582973599433899f, 0.41233018040657043f, 0.11267463117837906f, -0.6405482888221741f, + -0.3609986901283264f, 0.012775760143995285f, -0.2171235829591751f, -1.4559533596038818f, + 0.35457029938697815f, 0.6563311219215393f, 0.6537231206893921f, 0.3505811393260956f, + -0.6814857721328735f, -0.7297898530960083f, 0.6871512532234192f, -0.0015805511502549052f, + -0.3538321852684021f, 0.003062809119001031f, -0.2038687914609909f, -0.018792632967233658f, + -1.4243801832199097f, 0.521765410900116f, 0.7176746726036072f, 0.07536222785711288f, + -0.2997317314147949f, 0.2230704128742218f, -0.3626021146774292f, 0.4491320550441742f, + 0.7222998142242432f, -0.9044352769851685f, -0.47677361965179443f, 0.3058870732784271f, + -0.9708606004714966f, 0.2604639232158661f, -0.32532522082328796f, -0.292137086391449f, + 0.4820544123649597f, 0.8973264694213867f, -0.8034776449203491f, 0.23205721378326416f, + 1.3044614791870117f, 0.5716648101806641f, -0.12356413155794144f, 0.990612804889679f, + 1.2687742710113525f, 0.11064939945936203f, 0.657378077507019f, 1.629033088684082f, + 0.4800627529621124f, 0.8530299663543701f, -0.6023254990577698f, 0.8232004642486572f, + 1.014742136001587f, -0.619846522808075f, 0.8764278292655945f, -1.3682783842086792f, + -0.10739564150571823f, 1.0715831518173218f, -0.06702680885791779f, 0.5789799094200134f, + 0.7011066675186157f, -1.7570565938949585f, 0.5924561619758606f, -0.3153534233570099f, + -0.7080892324447632f, -0.922727108001709f, -0.6653363704681396f, -0.5511132478713989f, + -0.986309826374054f, 0.3467424213886261f, -0.07700745761394501f, -0.051563192158937454f, + 0.9132165312767029f, -0.8952778577804565f, -0.14221453666687012f, 0.2195023149251938f, + 0.0045674266293644905f, 0.7648011445999146f, 0.7244593501091003f, 0.626754105091095f, + 0.4064604640007019f, -0.4019644856452942f, 0.21144729852676392f, 0.04593051224946976f, + 0.19719155132770538f, 0.6718732118606567f, -0.5636104345321655f, -0.24805906414985657f, + -0.5939117074012756f, -0.16638851165771484f, -1.7792322635650635f, -0.4763451814651489f, + -0.6577343940734863f, 1.4129914045333862f, -0.6467493176460266f, 0.05420286953449249f, + -0.3688022792339325f, -0.7673300504684448f, -0.25901633501052856f, -0.09210846573114395f, + -0.277850478887558f, -0.17840279638767242f, 0.06790294498205185f, -0.2004445642232895f, + -0.24751640856266022f, -0.9676609039306641f, 0.25858840346336365f, 0.3061240613460541f, + -0.16164369881153107f, 0.44269394874572754f, 0.8506069779396057f, 0.5530581474304199f, + -0.13586844503879547f, 0.685543954372406f, 0.5118535757064819f, -0.6669505834579468f, + 0.19268251955509186f, 0.3106062710285187f, -0.02461124211549759f, -0.42892247438430786f, + -0.9235017895698547f, -0.14668190479278564f, -1.2094160318374634f, 0.25956469774246216f, + -0.3342626988887787f, 0.7059651017189026f, 0.0394313670694828f, 0.6531504988670349f, + -0.6581044793128967f, 0.45441511273384094f, 0.07928690314292908f, -0.34926456212997437f, + -0.27397191524505615f, 0.05833880230784416f, 0.047806769609451294f, -1.3177028894424438f, + 0.41888055205345154f, 0.6352789402008057f, 0.6052389144897461f, 0.09308644384145737f, + -0.3867514431476593f, -0.4149143397808075f, 0.7081300616264343f, 0.08151618391275406f, + -0.16282109916210175f, -0.0204594936221838f, -0.11125867068767548f, -0.2400633841753006f, + -0.8353896737098694f, 0.7600412368774414f, 0.6871867775917053f, -0.414286345243454f, + -0.5413820743560791f, 0.12487572431564331f, -0.354497492313385f, 0.2966896891593933f, + 0.5260758996009827f, -0.8832938075065613f, -0.2927309572696686f, 0.3051953911781311f, + -0.4114215075969696f, 0.3054697513580322f, 0.15065762400627136f, -0.279171884059906f, + -0.391975462436676f, 0.564216136932373f, 0.2746603786945343f, -1.271131992340088f, + 1.0474594831466675f, 0.5245835185050964f, 0.1775088608264923f, -0.27393367886543274f, + 0.11812248826026917f, 0.22940455377101898f, 1.040003776550293f, -0.1905238777399063f, + 0.4788508117198944f, -0.22627441585063934f, 0.18793468177318573f, -0.7452923655509949f, + -0.12449031323194504f, 1.1271625757217407f, -0.08125371485948563f, -1.8782660961151123f, + -0.9807635545730591f, -0.32755839824676514f, 0.5305302143096924f, 0.28197795152664185f, + 0.4651734232902527f, -0.48749953508377075f, -0.39377957582473755f, 0.11695512384176254f, + -0.2754250466823578f, 0.40481945872306824f, 0.07376840710639954f, 1.243362545967102f, + 0.6091680526733398f, 0.058975689113140106f, 0.23206117749214172f, 0.18122491240501404f, + 0.7616679668426514f, -0.06307251751422882f, 0.39027443528175354f, 0.12984472513198853f, + -0.08915241062641144f, -0.03651172295212746f, 0.4498511254787445f, 0.7187243103981018f, + -0.9636116027832031f, -0.7918862700462341f, -0.5601437091827393f, 0.3779510259628296f, + -0.4846689701080322f, -0.44938796758651733f, -0.029808444902300835f, 0.7382371425628662f, + 0.10807140171527863f, 0.07277264446020126f, -0.2443360984325409f, 0.4524882733821869f, + 0.7832004427909851f, 1.383375644683838f, -0.20265953242778778f, 0.05995623394846916f, + -0.1447192281484604f, -0.1487491875886917f, -0.073786661028862f, 0.547607958316803f, + -0.1603822410106659f, 0.1563190370798111f, 0.07805128395557404f, 0.26086270809173584f, + 0.27510711550712585f, -0.25947684049606323f, 0.596296489238739f, 0.15973497927188873f, + 0.13175179064273834f, 0.51873379945755f, 0.27212896943092346f, 0.33143287897109985f, + -0.4980114698410034f, -0.1749369502067566f, 0.07077206671237946f, 0.08172274380922318f, + -0.09481344372034073f, -0.15266643464565277f, 0.05547310411930084f, 0.3172110915184021f, + -0.15284520387649536f, -0.06222972273826599f, -0.8392258286476135f, 0.2833206355571747f, + -0.0017102344427257776f, 1.0074207782745361f, 0.004109530709683895f, 1.0040134191513062f, + 0.6940479278564453f, 0.5099221467971802f, -0.11958321928977966f, 0.08031485974788666f, + -0.37246114015579224f, -0.357016384601593f, -0.2886791527271271f, -0.2634924650192261f, + 0.3254944682121277f, 0.7588571310043335f, 0.8682138919830322f, 0.24148912727832794f, + 0.4687095582485199f, -0.039406951516866684f, 0.6495793461799622f, 0.6673002243041992f, + -0.12229056656360626f, -0.2060908079147339f, -1.214479923248291f, -0.670218825340271f, + 0.15852248668670654f, 0.6545276641845703f, 0.43483754992485046f, -0.34149274230003357f, + -0.16777044534683228f, 0.6611889004707336f, 0.4281454086303711f, -0.16114795207977295f, + -0.6306483745574951f, -0.7531115412712097f, 0.9593201279640198f, 0.6332415342330933f, + 0.05864579975605011f, -0.21197551488876343f, -0.04580022394657135f, -0.29718586802482605f, + 0.07555583864450455f, -0.16240721940994263f, 0.8092191219329834f, -1.5678287744522095f, + -1.1805871725082397f, 0.32295089960098267f, 1.4793514013290405f, -1.0368294715881348f, + -0.6895912289619446f, -0.12336522340774536f, 0.5264229774475098f, 0.7566879987716675f, + -0.025588110089302063f, -0.6965243816375732f, 0.22840464115142822f, -1.737051010131836f, + 0.9229766130447388f, 0.5828476548194885f, 0.25931984186172485f, 0.1410982757806778f, + -1.589451789855957f, 0.10153551399707794f, -1.3623404502868652f, -0.3586464822292328f, + -0.3200659453868866f, -0.3462951183319092f, -1.5308380126953125f, -0.08327502012252808f, + 0.35808733105659485f, 0.2636767029762268f, 0.5424822568893433f, 0.2657417356967926f, + 0.16561751067638397f, -0.2569218575954437f, -0.3645927906036377f, 0.43422621488571167f, + -1.135851263999939f, 0.03054295852780342f, -0.03190584108233452f, -0.5781900882720947f, + 0.7152751684188843f, 0.3461851477622986f, 0.24718843400478363f, -0.03499390929937363f, + -0.9950652122497559f, 0.6104280948638916f, -0.3100389540195465f, -0.6737688183784485f, + -0.3548102080821991f, -0.27895694971084595f, 0.6188498139381409f, 0.25734564661979675f, + -0.6880052089691162f, 0.2272089272737503f, -0.775479793548584f, -0.20573221147060394f, + 0.3818489611148834f, 0.31442904472351074f, -0.1289101243019104f, -0.019834108650684357f, + 0.41780203580856323f, -0.21203908324241638f, 0.22220000624656677f, 0.358887255191803f, + -0.2261275202035904f, 0.2651781141757965f, -0.12034319341182709f, 0.43035751581192017f, + -0.5966606140136719f, 0.03032268024981022f, 0.5749114751815796f, -0.03961149975657463f, + 0.5210158228874207f, 0.6845589280128479f, -0.3717062175273895f, 0.04531615972518921f, + -0.5068879723548889f, 0.2857106029987335f, 0.4028048813343048f, -0.07891042530536652f, + -0.12387226521968842f, -0.16887034475803375f, 0.46841707825660706f, 0.1266188621520996f, + -0.1424398422241211f, -0.008458521217107773f, -1.019404411315918f, 0.11540728807449341f, + -0.147792786359787f, 0.3877210319042206f, 0.2083432972431183f, 0.348195880651474f, + 0.14628487825393677f, -0.36994776129722595f, -0.4196460247039795f, -0.4077725112438202f, + -0.0900408923625946f, 0.5092692375183105f, -0.24250227212905884f, -0.6883799433708191f, + 0.7167907953262329f, 0.29215681552886963f, 0.2111043632030487f, -0.38843396306037903f, + 0.3113866150379181f, 0.2584671974182129f, 0.7186911106109619f, 0.25048449635505676f, + 0.2680424749851227f, -0.012737352401018143f, -0.31030526757240295f, -0.4282568395137787f, + 0.16362611949443817f, 0.3644122779369354f, -0.3293208181858063f, -1.015744924545288f, + -0.7001276612281799f, 0.038187067955732346f, 0.41888272762298584f, 0.4685537815093994f, + 0.5203832983970642f, -0.7542248368263245f, -0.2593006491661072f, 0.5951202511787415f, + 0.32686537504196167f, -0.3884551525115967f, -0.6520037651062012f, -0.14074376225471497f, + -0.14357519149780273f, 0.1550052911043167f, 0.03675243258476257f, -0.5273981094360352f, + 0.3214477002620697f, 0.1910211145877838f, 0.31177064776420593f, -0.48916515707969666f, + 0.18305404484272003f, 0.1689586043357849f, 0.4488915503025055f, -0.04135023057460785f, + -0.175848126411438f, -0.010493399575352669f, -0.5582103133201599f, -0.44524529576301575f, + -0.2830869257450104f, 0.6693865656852722f, -0.20459258556365967f, -0.8257265686988831f, + -0.580522894859314f, -0.19866153597831726f, 0.4395535886287689f, 0.3521166145801544f, + 0.4905449151992798f, -0.6604574918746948f, -0.04066608101129532f, 0.036411985754966736f, + 0.08822466433048248f, -0.20956610143184662f, -0.06433288753032684f, 0.39180147647857666f, + -0.156889870762825f, 0.23665131628513336f, -0.002607153495773673f, 0.30433154106140137f, + -0.2166159600019455f, -0.27228280901908875f, 0.7659205794334412f, 0.2554369270801544f, + 0.10164103657007217f, 0.7116388082504272f, 0.04340958595275879f, 0.19004660844802856f, + -0.10940301418304443f, -0.17635513842105865f, 0.35454392433166504f, -0.03140521049499512f, + -0.2245149463415146f, -0.10971376299858093f, 0.26240089535713196f, 0.26686933636665344f, + -0.2630356550216675f, -0.1464148312807083f, -0.9048764705657959f, 0.302548885345459f, + -0.3999890983104706f, 0.49958252906799316f, 0.17058520019054413f, -0.26887574791908264f, + -0.3363683223724365f, -0.4910518527030945f, -0.29581716656684875f, -0.2125953733921051f, + -0.39885327219963074f, 0.21188069880008698f, -0.12696780264377594f, 0.006022256333380938f, + 0.34799477458000183f, -0.5470008254051208f, -0.18672555685043335f, -0.017054371535778046f, + 0.16728350520133972f, 0.5048250555992126f, 0.4807291328907013f, 0.5389289259910583f, + -0.033229175955057144f, -0.24178384244441986f, -0.05529379844665527f, + -0.047342367470264435f, -0.10220284014940262f, 0.32566601037979126f, -0.24707409739494324f, + -0.020163998007774353f, -0.5398797988891602f, -0.0005280787008814514f, + -1.3214551210403442f, -0.40837880969047546f, -0.23740871250629425f, 1.0602668523788452f, + -0.5058678984642029f, 0.5341110229492188f, -0.6254277229309082f, 0.36167389154434204f, + 0.11041107028722763f, -0.08200881630182266f, -0.31292295455932617f, 0.04303249716758728f, + 0.21452973783016205f, -1.0917692184448242f, 0.5094312429428101f, 0.463744580745697f, + 0.36889511346817017f, 0.01606285385787487f, -0.2880471348762512f, -0.3350456953048706f, + 0.5084245204925537f, -0.06925438344478607f, -0.20082958042621613f, -0.05085105448961258f, + -0.06057576462626457f, -0.4172911047935486f, -0.7614612579345703f, 1.0557268857955933f, + 0.5693895220756531f, -0.6325477361679077f, -0.41804954409599304f, 0.009387768805027008f, + -0.439547061920166f, 0.30733686685562134f, 0.530044436454773f, -0.6933425068855286f, + -0.08170264214277267f, 0.31492769718170166f, 0.2846527397632599f, -0.15894092619419098f, + -0.3520893156528473f, -0.18177905678749084f, -0.33050987124443054f, 0.015117231756448746f, + 0.15732347965240479f, -0.6988750696182251f, -0.1939268708229065f, 0.18622447550296783f, + 0.5296834707260132f, -0.15524591505527496f, -0.12831929326057434f, -0.19257059693336487f, + 0.13907712697982788f, 0.09023575484752655f, -0.31478485465049744f, -0.058192674070596695f, + -0.5168077349662781f, -0.6624992489814758f, -0.4231608510017395f, 0.7828724980354309f, + 0.1592288613319397f, -0.3794378638267517f, -0.4710177183151245f, -0.017347481101751328f, + 0.013850254006683826f, 0.30874550342559814f, 0.4768539071083069f, -0.39115479588508606f, + -0.20525184273719788f, 0.027457131072878838f, 0.5202342867851257f, 0.31727227568626404f, + 0.0070562660694122314f, -0.3000951409339905f, -0.27604469656944275f, 0.48744407296180725f, + -0.11892017722129822f, 0.1917044222354889f, -0.5441881418228149f, 0.16825038194656372f, + 0.44087010622024536f, -0.37070271372795105f, 0.5816771388053894f, -0.16165579855442047f, + 0.25672781467437744f, 0.16932372748851776f, -0.30253803730010986f, 0.19321919977664948f, + 0.3044843077659607f, 0.12256979197263718f, -0.06154022738337517f, -0.0066994186490774155f, + -0.008523472584784031f, 0.1707102209329605f, -0.12906593084335327f, 0.1643287092447281f, + -0.2685627341270447f, 0.5414547920227051f, 0.24318350851535797f, -0.009374771267175674f, + 0.3419133126735687f, 0.026718085631728172f, 0.17247779667377472f, 0.01812201552093029f, + -0.022323451936244965f, 0.5673023462295532f, 0.03444672375917435f, 0.2232545167207718f, + -0.17510905861854553f, 0.38465455174446106f, -0.2716033458709717f, -0.28139355778694153f, + 0.7381399273872375f, 0.2681874930858612f, 0.024812230840325356f, 0.649937093257904f, + 0.10233858227729797f, 0.34073275327682495f, 0.012529485858976841f, -0.6340211033821106f, + 0.4116799831390381f, -0.0605587437748909f, -0.4738684296607971f, -0.014262971468269825f, + 0.4313955307006836f, 0.3329358696937561f, -0.21001678705215454f, -0.45911991596221924f, + -1.0685330629348755f, 0.23777920007705688f, -0.2661783993244171f, 0.40272265672683716f, + -0.04702693596482277f, 0.5065898299217224f, 0.734368085861206f, -0.6063792109489441f, + -0.6010410189628601f, -0.47487151622772217f, -0.23869699239730835f, 0.10308986157178879f, + 0.04026699811220169f, -0.6386284232139587f, -0.08522013574838638f, 0.15661393105983734f, + 0.5571959614753723f, -0.6569101214408875f, -0.047412436455488205f, 0.0014737955061718822f, + 0.3750648498535156f, -0.14268018305301666f, -0.35291972756385803f, -0.188123881816864f, + -0.49688148498535156f, -0.6434450149536133f, -0.3412753939628601f, 0.4993920624256134f, + -0.4481845796108246f, -0.21332058310508728f, -0.6111479997634888f, -0.1316085010766983f, + 0.24001926183700562f, 0.36975106596946716f, 0.3729320168495178f, -0.5188049077987671f, + -0.3082001507282257f, 0.2500453293323517f, 0.3584001958370209f, 0.16858454048633575f, + 0.2165459245443344f, -0.7535542845726013f, -0.4905281960964203f, 0.5162887573242188f, + 0.44601374864578247f, -0.37445664405822754f, 0.3364827632904053f, 0.4944031238555908f, + 0.3771876394748688f, -0.048119667917490005f, 0.4963591992855072f, -0.19974493980407715f, + 0.43996894359588623f, -0.3147045373916626f, -0.35825955867767334f, -0.058220572769641876f, + 0.0757138580083847f, -0.4011352062225342f, -0.023589007556438446f, 0.7060518860816956f, + -0.08135643601417542f, -0.7536214590072632f, -0.42476511001586914f, 0.15118351578712463f, + -0.10649950802326202f, 0.3979780673980713f, 0.17925362288951874f, -0.6429068446159363f, + 0.38492512702941895f, -0.039637863636016846f, 0.8186268210411072f, 0.05923304706811905f, + 0.03840453922748566f, -0.4124068319797516f, -0.2330872118473053f, 0.41798609495162964f, + -0.17601153254508972f, 0.035936858505010605f, -0.6698760986328125f, 0.07776574045419693f, + 0.2232595980167389f, -0.4769856333732605f, 0.6625787019729614f, -0.11957532912492752f, + -0.020084219053387642f, 0.26634684205055237f, -0.4082069993019104f, 0.8184704184532166f, + 0.33483096957206726f, 0.09106092900037766f, 0.09893608093261719f, -0.0009211912984028459f, + 0.054328709840774536f, 0.06746330857276917f, -0.24102449417114258f, + -0.0008188950014300644f, -0.3937647342681885f, 0.30076658725738525f, 0.1668635457754135f, + -0.07214608043432236f, 0.3942160904407501f, 0.1377922147512436f, 0.503115713596344f, + -0.20089663565158844f, 0.12860682606697083f, -0.35060521960258484f, 0.14628618955612183f, + 0.5559941530227661f, -0.048192478716373444f, -0.5479832291603088f, 0.13678129017353058f, + -0.5141637921333313f, -0.20818275213241577f, -0.17906342446804047f, -0.02088446356356144f, + 0.01028110645711422f, 0.33248963952064514f, 0.7605072259902954f, 0.3821932375431061f, + 1.1366232633590698f, 0.09766270965337753f, 0.22744570672512054f, -0.18691033124923706f, + -0.20994949340820312f, -0.025196118280291557f, 0.11103303730487823f, -0.40392759442329407f, + -0.09904412180185318f, -0.04003768786787987f, 0.34199661016464233f, -0.2735242247581482f, + -0.22727981209754944f, 0.3874532878398895f, 0.7798921465873718f, 0.011668223887681961f, + -0.2225727140903473f, -0.020242389291524887f, 0.7288497090339661f, 0.11494404077529907f, + 0.9768214821815491f, -0.35555824637413025f, -0.1937469244003296f, -0.1533007025718689f, + 0.1632596105337143f, -0.04134052246809006f, 0.18370138108730316f, 0.12323915958404541f, + 0.20685692131519318f, -0.4916249215602875f, -0.5045111775398254f, -0.6160029172897339f, + 0.3436583876609802f, -0.3211502730846405f, 0.19522355496883392f, 0.1815038025379181f, + 0.4686097800731659f, -0.9531223773956299f, 0.06651858985424042f, -0.2325400859117508f, + -0.3192019760608673f, 1.1753125190734863f, 0.22577400505542755f, -0.0006504314951598644f, + -0.050322722643613815f, -0.33118322491645813f, 0.3679087460041046f, -0.5442244410514832f, + 0.0656610056757927f, 0.27711600065231323f, 0.5571757555007935f, -0.07555516064167023f, + 0.4003889262676239f, 0.6894688606262207f, -0.21355970203876495f, 0.2409370094537735f, + 0.0699319839477539f, -0.08889027684926987f, -0.029077818617224693f, 0.1869862824678421f, + -0.08126796036958694f, -0.16468574106693268f, -0.2653341293334961f, -0.48841461539268494f, + -0.014531880617141724f, 0.21362434327602386f, -0.621538519859314f, 0.17372898757457733f, + 1.2268763780593872f, 0.0250852070748806f, -0.9164780974388123f, -0.25433847308158875f, + -0.18553179502487183f, -0.6097862720489502f, 0.13870954513549805f, 0.18318963050842285f, + -0.24898631870746613f, 0.12123600393533707f, 0.23843270540237427f, -0.2765331268310547f, + -0.6353736519813538f, -0.13015618920326233f, 0.42163801193237305f, -0.011705624870955944f, + -0.22579044103622437f, 0.16743873059749603f, -0.11115960776805878f, -0.5712466239929199f, + -0.9085857272148132f, 0.8447943329811096f, 0.45915305614471436f, -0.4016120433807373f, + 0.40026968717575073f, 0.6300693154335022f, 0.47694167494773865f, -0.22918222844600677f, + 0.7380399107933044f, 0.6817950010299683f, -0.7282121181488037f, 0.08896227180957794f, + 0.06301464885473251f, 0.3086538016796112f, -0.2916901409626007f, -0.8330901861190796f, + -0.0032006418332457542f, -0.8228612542152405f, 0.781017005443573f, -0.3233967423439026f, + 0.23470647633075714f, 0.5005122423171997f, 0.021768750622868538f, -0.12458038330078125f, + 0.13908258080482483f, -0.15533404052257538f, 0.7963553667068481f, 0.17964786291122437f, + 0.04206337407231331f, -0.02658257447183132f, 0.360134094953537f, -0.03943195939064026f, + -0.253757506608963f, 0.9054529070854187f, 0.19211925566196442f, -0.08221887797117233f, + 0.49507686495780945f, 0.26007452607154846f, 0.284741073846817f, -0.36171263456344604f, + -0.7453933358192444f, -0.06045202910900116f, -0.01462505478411913f, -0.44725701212882996f, + -0.20083633065223694f, 0.23854206502437592f, 0.6238372325897217f, -0.25929656624794006f, + -0.08040202409029007f, -0.6730373501777649f, 0.325137197971344f, -0.07986194640398026f, + 0.6499130725860596f, 0.013557055033743382f, 0.17546051740646362f, 0.3763168454170227f, + 0.10734734684228897f, 0.03638329356908798f, -0.8393217921257019f, -0.46629348397254944f, + 0.5424652099609375f, -0.09374668449163437f, -1.233060598373413f, 0.5108470916748047f, + 0.4663965106010437f, 0.5572637915611267f, -0.13837380707263947f, 0.011991024948656559f, + 0.1011277362704277f, 1.0115456581115723f, 0.1263999044895172f, 0.589897096157074f, + -0.37738701701164246f, -0.13703931868076324f, -0.8103193044662476f, -0.01680792309343815f, + 0.43078410625457764f, -0.39601999521255493f, -1.0349551439285278f, -0.9569106698036194f, + 0.08323471993207932f, 0.6321536898612976f, 0.2010834813117981f, 0.27905261516571045f, + -0.3761727511882782f, -0.7371733784675598f, 1.317244291305542f, 0.23596252501010895f, + 1.2670769691467285f, -0.18412986397743225f, -1.157860279083252f, -0.3683636486530304f, + 0.1858825981616974f, 0.7011803984642029f, -1.0791300535202026f, -0.02834385074675083f, + 0.19797134399414062f, 0.8456324934959412f, 0.11133934557437897f, -0.1336931586265564f, + -0.07344797253608704f, 0.8236565589904785f, -0.2871275246143341f, 0.9078686237335205f, + -0.20536074042320251f, 0.15224677324295044f, 0.4424932599067688f, -0.9534909129142761f, + 0.05795537680387497f, 0.25616326928138733f, 0.15068921446800232f, -0.976055383682251f, + 0.5753369331359863f, -0.9183844327926636f, -0.5890903472900391f, 0.4481801390647888f, + -1.0693782567977905f, -0.32138389348983765f, 0.2983241081237793f, -0.2110815793275833f, + 0.18135379254817963f, 0.13647876679897308f, 1.2628377676010132f, 0.3747123181819916f, + 0.11127675324678421f, 0.23631010949611664f, 0.03171908110380173f, 1.0251275300979614f, + -0.039828330278396606f, 0.19766348600387573f, -0.013753875158727169f, -0.1939336210489273f, + -0.19327352941036224f, 0.47591447830200195f, 0.8366078734397888f, -0.9191758632659912f, + -0.6375211477279663f, -0.49680590629577637f, 0.33670774102211f, -0.390082448720932f, + -0.34522464871406555f, -0.001646929420530796f, 0.5865241885185242f, 0.2490273118019104f, + 0.06759701669216156f, -0.18545544147491455f, 0.47880756855010986f, 0.9194368124008179f, + 1.203451156616211f, -0.23423311114311218f, 0.15292476117610931f, 0.06206126511096954f, + -0.6225329041481018f, -0.6227844953536987f, -0.12640541791915894f, -0.39964789152145386f, + -0.4012860655784607f, 0.15121443569660187f, -0.058566540479660034f, -0.7052518725395203f, + -0.28230226039886475f, 1.031558632850647f, -0.16898562014102936f, 0.0862472653388977f, + 0.6071692705154419f, 0.4042854309082031f, -0.39902472496032715f, -0.5131970643997192f, + 0.39678797125816345f, -0.17752760648727417f, -0.6766841411590576f, 0.13496986031532288f, + -0.1230105459690094f, -0.1366419494152069f, 0.4662974178791046f, -0.8599501252174377f, + 0.2530156373977661f, -0.34761616587638855f, 0.10407125949859619f, -0.7038118243217468f, + 0.2236679494380951f, 0.43949562311172485f, 0.7067329287528992f, 0.3247920870780945f, + 0.4930388331413269f, -0.12206322699785233f, -0.6464152336120605f, -0.31361913681030273f, + 0.16710229218006134f, 0.5679914951324463f, -1.172094702720642f, -0.34729766845703125f, + 0.18545542657375336f, 1.0927492380142212f, -0.28952455520629883f, -0.21913287043571472f, + 0.09205657988786697f, 0.8724817633628845f, 0.3880108892917633f, 0.5302738547325134f, + -0.2960461378097534f, 0.12649981677532196f, -0.5091269016265869f, 0.08895653486251831f, + 0.5095667243003845f, 0.2804725170135498f, -0.26289102435112f, -1.3281580209732056f, + 0.30504173040390015f, -0.7069351077079773f, -0.3812716603279114f, 0.08043661713600159f, + -0.572374165058136f, -0.8036254048347473f, 1.1237082481384277f, 0.7833073735237122f, + 0.1415664255619049f, -0.36507633328437805f, -0.5122433304786682f, -0.36611080169677734f, + -0.25845399498939514f, -0.028455711901187897f, -0.7432114481925964f, 0.15268279612064362f, + 0.2824477255344391f, 0.7730991840362549f, -0.36901211738586426f, -0.20637424290180206f, + -0.10589263588190079f, 0.6551610827445984f, -0.1635896861553192f, 0.0007743912283331156f, + -0.3831789791584015f, -0.5825473666191101f, -0.35647186636924744f, -0.7948381304740906f, + 0.42981788516044617f, -0.19757798314094543f, 0.4358223080635071f, -0.416127473115921f, + 0.35802608728408813f, -0.25549542903900146f, -0.10588227957487106f, -0.008900387212634087f, + -0.7486977577209473f, 0.10606259852647781f, 0.025339746847748756f, 0.8243540525436401f, + -0.07774729281663895f, -0.02432101219892502f, -0.3867083191871643f, 0.43011873960494995f, + 0.1969851404428482f, -0.38655969500541687f, -0.2941063940525055f, -0.6593753099441528f, + 0.05376604571938515f, -0.13635636866092682f, -0.6026633977890015f, 0.6250115036964417f, + -0.060427237302064896f, -0.10744139552116394f, 0.42560890316963196f, -0.3235200345516205f, + 0.7858245968818665f, 0.014990752562880516f, 0.009434830397367477f, -0.3760318160057068f, + 0.03987026959657669f, -0.02704782970249653f, 0.06772609055042267f, -0.1636480987071991f, + -0.1760774403810501f, -0.502366304397583f, -0.2991757392883301f, 0.26906028389930725f, + -0.32254189252853394f, 0.09490648657083511f, 0.21019627153873444f, 0.21182577311992645f, + -0.040702223777770996f, 0.0892731100320816f, -0.15513236820697784f, -0.08556026220321655f, + 0.5998643040657043f, -0.3073670566082001f, -0.38705703616142273f, 0.37405240535736084f, + -0.4899393618106842f, 0.14859060943126678f, -0.4416752755641937f, -0.07880990207195282f, + 0.10858815163373947f, 0.3993547558784485f, 0.5966002345085144f, 0.43468546867370605f, + 0.8270280957221985f, -0.07107976824045181f, 0.21646514534950256f, -0.2925468385219574f, + -0.22947025299072266f, 0.0998540148139f, 0.33550724387168884f, -0.2464204579591751f, + 0.12087341398000717f, -0.005840559024363756f, 0.2951580584049225f, -0.25976091623306274f, + -0.5565623641014099f, 0.36314207315444946f, 0.38583114743232727f, -0.160760298371315f, + 0.02666308544576168f, 0.02036372758448124f, -0.20515839755535126f, -0.38914915919303894f, + 0.15718047320842743f, 0.1860467493534088f, -1.0371311902999878f, -0.11601860076189041f, + 0.2466573268175125f, 0.5755811929702759f, 0.0058876206167042255f, -0.2667483389377594f, + -0.488141268491745f, 0.30596843361854553f, 0.043746303766965866f, -0.3397168815135956f, + 0.08345984667539597f, 0.09737057238817215f, -0.1257927268743515f, -0.9160090684890747f, + 0.7981093525886536f, 0.6543830633163452f, 0.07669354230165482f, -0.5595042109489441f, + 0.19365207850933075f, -0.6363568305969238f, 0.4706505239009857f, 0.597342312335968f, + -0.6331154108047485f, -0.41559651494026184f, -0.09632350504398346f, -0.048610761761665344f, + -0.5983400344848633f, 0.19741825759410858f, 0.789680540561676f, -0.2702938914299011f, + 0.2734069228172302f, 0.7406113743782043f, -0.3172116279602051f, -0.27150237560272217f, + -0.232101172208786f, -0.047834362834692f, -0.3561805784702301f, 0.448498010635376f, + -0.3299601078033447f, -0.4905584156513214f, 0.19680778682231903f, -0.29181766510009766f, + 0.00016492677968926728f, 0.48957905173301697f, -0.8697939515113831f, 0.010497107170522213f, + 1.3232613801956177f, 0.587515652179718f, -0.43186473846435547f, -0.4400500953197479f, + 0.211727112531662f, -1.2719732522964478f, 0.6640509366989136f, 0.435154527425766f, + -0.19532637298107147f, -0.06197727844119072f, -0.05441476032137871f, 0.5214816927909851f, + -0.31980377435684204f, -0.28401127457618713f, -0.1064978539943695f, 0.19715240597724915f, + 0.04795083776116371f, -0.06732014566659927f, -0.3917539119720459f, 0.2714834213256836f, + -0.3622303903102875f, -0.17879512906074524f, -0.5217910408973694f, -0.18659566342830658f, + -0.2545938491821289f, 0.12507857382297516f, 0.7540612816810608f, -0.03764486685395241f, + 0.6041346788406372f, -0.27693772315979004f, 0.24162037670612335f, -0.28509846329689026f, + 0.08491276204586029f, -0.12424381077289581f, 0.267432302236557f, -0.23075971007347107f, + -0.17670002579689026f, -0.15196560323238373f, -0.08095184713602066f, -0.03226323425769806f, + -0.2685889005661011f, 0.10865277796983719f, 0.24682484567165375f, 0.0674993246793747f, + 0.0660565048456192f, 0.016734249889850616f, -0.46573832631111145f, 0.48259246349334717f, + 0.5330647230148315f, -0.1938147246837616f, -0.6517442464828491f, 0.6380144357681274f, + -0.4281212389469147f, -0.20182234048843384f, -0.3968639671802521f, -0.12098109722137451f, + 0.08749483525753021f, 0.6400931477546692f, 0.5919732451438904f, 0.8098844885826111f, + 0.7049689292907715f, -0.24383290112018585f, 0.42629170417785645f, -0.8212608098983765f, + -0.3029347062110901f, -0.26549482345581055f, 0.25061818957328796f, 0.0785837173461914f, + -0.05520016327500343f, 0.1883384734392166f, 0.08157448470592499f, -0.35870668292045593f, + -0.7145575284957886f, 0.21691565215587616f, 0.21544688940048218f, 0.15109112858772278f, + -0.38416787981987f, -0.0068653118796646595f, 0.6407197117805481f, 0.13022959232330322f, + 0.47329846024513245f, 0.3109179437160492f, -0.251838743686676f, -0.18475356698036194f, + -0.23241540789604187f, -0.16758331656455994f, -0.24595582485198975f, 0.4992983937263489f, + -0.09490983188152313f, -0.6101731061935425f, -0.23934146761894226f, -0.31341367959976196f, + 0.27929937839508057f, 0.14955265820026398f, 0.2290695309638977f, 0.02273491583764553f, + 0.35619860887527466f, -0.10218359529972076f, 0.036338381469249725f, -0.33820903301239014f, + 0.27230390906333923f, -0.14781324565410614f, 0.4826175570487976f, 0.15594805777072906f, + -0.6220239400863647f, -0.26521652936935425f, -0.3237963020801544f, 0.3261258006095886f, + -0.3543921113014221f, 0.07099086046218872f, 0.3335495889186859f, -0.4141715168952942f, + 0.410957396030426f, 0.6648834347724915f, 0.00226298114284873f, -0.14915265142917633f, + -0.3267055153846741f, -0.18826152384281158f, -0.2706129550933838f, 0.4417923092842102f, + -0.13852524757385254f, -0.7664065361022949f, -0.3418484628200531f, -0.5009276270866394f, + 0.28067973256111145f, 0.24306049942970276f, -0.40711385011672974f, 0.06008037552237511f, + 1.2783677577972412f, 0.07707283645868301f, -0.34009119868278503f, -0.16827650368213654f, + 0.08750004321336746f, -0.7929356694221497f, 0.6966761350631714f, 0.577951192855835f, + -0.1832721084356308f, 0.4553610384464264f, 0.14802221953868866f, 0.49810364842414856f, + 0.11356418579816818f, -0.5065097808837891f, -0.42153188586235046f, 0.011264470405876637f, + 0.5107285380363464f, -0.050495438277721405f, -0.4980171322822571f, 0.04811753332614899f, + -0.06067327409982681f, 0.2920294404029846f, -0.20897002518177032f, 0.31357645988464355f, + -0.2482505589723587f, 0.32320448756217957f, 0.3347584903240204f, 0.28591135144233704f, + -0.05173273757100105f, 0.16694393754005432f, 0.4082314074039459f, -0.02604116126894951f, + 0.10802900046110153f, -0.3543420433998108f, 0.24871638417243958f, -0.0551690049469471f, + -0.321806937456131f, -0.10170630365610123f, 0.3478531837463379f, -0.13953156769275665f, + 0.10044601559638977f, 0.1651950478553772f, 0.12960036098957062f, 0.37791475653648376f, + -0.14135724306106567f, -0.6073365807533264f, -0.20863258838653564f, -0.04061451554298401f, + 0.216165229678154f, 0.016729239374399185f, -0.4624423384666443f, -0.0199031513184309f, + -0.17057989537715912f, 0.18928463757038116f, -0.0191546231508255f, 0.2468564361333847f, + 0.06526995450258255f, 0.21011222898960114f, 0.12978290021419525f, 0.40099725127220154f, + -0.25365370512008667f, 0.03843764588236809f, 0.22812500596046448f, 0.07245387881994247f, + 0.22299228608608246f, -0.17778871953487396f, 0.23118750751018524f, -0.2982270121574402f, + -0.4401658773422241f, -0.4006398618221283f, 0.12478355318307877f, -0.48116984963417053f, + 0.441814661026001f, -0.0821511372923851f, 0.4161858558654785f, 0.013002391904592514f, + 0.05587095394730568f, -0.4855250418186188f, -0.3451950252056122f, 0.12183991819620132f, + 0.8598132729530334f, -0.894964873790741f, 0.1463768035173416f, 1.1310516595840454f, + 0.5623629093170166f, 0.017643408849835396f, 0.3899429738521576f, 1.3534598350524902f, + 0.38929158449172974f, 0.49848905205726624f, 1.1181762218475342f, 0.5119673609733582f, + 0.7480975985527039f, -0.7196075916290283f, 0.6071317195892334f, 1.0170568227767944f, + -0.2970138490200043f, 0.29035869240760803f, -0.9669429063796997f, -0.2728392779827118f, + 0.9067572951316833f, 0.5441610217094421f, 0.717409610748291f, 0.5266432762145996f, + -1.5074248313903809f, 0.9626047611236572f, 1.2323302030563354f, 0.6103440523147583f, + -0.8859409093856812f, -0.4092333912849426f, 1.0151773691177368f, 0.2795063853263855f, + 0.6768396496772766f, -0.8188841938972473f, -0.04683113098144531f, 0.06913090497255325f, + -0.11358339339494705f, -0.05678768455982208f, -0.4408646821975708f, 0.3232562839984894f, + 0.5149379372596741f, -0.3729676902294159f, -0.40816420316696167f, -0.43588489294052124f, + -0.03211471810936928f, -0.5068545341491699f, -0.01915191113948822f, 0.4790549576282501f, + -0.0028278883546590805f, -1.651803731918335f, 0.4777693450450897f, -0.23446297645568848f, + -0.21963977813720703f, 1.4039273262023926f, 0.1947948932647705f, -0.32510584592819214f, + -0.20863215625286102f, -0.2521999180316925f, 0.13348740339279175f, 0.012039562687277794f, + -0.2198227345943451f, 0.2441766858100891f, 0.23996753990650177f, 0.3881019949913025f, + 0.05266518518328667f, -0.38376545906066895f, -0.19882285594940186f, -0.31901806592941284f, + -0.09143021702766418f, -0.26950106024742126f, 0.34734928607940674f, -0.626097559928894f, + 0.19197878241539001f, 0.4145083725452423f, 1.0526310205459595f, -0.3466022312641144f, + 0.5374821424484253f, 0.10358438640832901f, -0.6342615485191345f, 0.06871465593576431f, + 0.37549784779548645f, 0.1314767748117447f, -0.25805479288101196f, -0.3561108708381653f, + 0.4002118706703186f, -0.7840244770050049f, 0.6489959955215454f, -0.34041887521743774f, + 0.4175305962562561f, -0.43664881587028503f, 0.17186009883880615f, 0.13249380886554718f, + -0.0691005066037178f, 0.1911209672689438f, 0.2303442358970642f, 0.17900124192237854f, + -0.21639034152030945f, -0.33777859807014465f, -0.09294271469116211f, -0.5661056041717529f, + -0.17517362534999847f, -0.11837778240442276f, -0.11655721068382263f, 0.13003885746002197f, + 0.28788718581199646f, 0.492314875125885f, 0.3453690707683563f, -0.4809058606624603f, + 0.5524118542671204f, -0.04600846394896507f, -0.6691011190414429f, -0.003267364576458931f, + 0.11445911973714828f, 0.35174092650413513f, -0.05686125531792641f, -0.5407987833023071f, + -0.04756605997681618f, -0.8490046858787537f, 0.13677294552326202f, -0.06102540343999863f, + 0.4894755482673645f, -0.34405049681663513f, 0.8078959584236145f, 0.17326456308364868f, + -0.12229807674884796f, -0.09529775381088257f, -0.0037976151797920465f, + -0.2567620575428009f, -0.2572486400604248f, -0.10313341021537781f, -1.103756308555603f, + -0.08068320155143738f, 0.1258392333984375f, 0.7998788356781006f, -0.27655643224716187f, + 0.056368228048086166f, -0.1704096496105194f, 0.4171367287635803f, 0.27931278944015503f, + 0.02734992839396f, -0.5854048728942871f, -0.1202973872423172f, -0.7164915204048157f, + -0.19408567249774933f, 0.12512372434139252f, 0.044074490666389465f, -0.03037148527801037f, + -0.6815215945243835f, 0.2156166434288025f, -0.035368580371141434f, 0.14683109521865845f, + 0.08904857933521271f, -0.6792382597923279f, -0.4514988958835602f, 0.6972554922103882f, + 0.15524688363075256f, -0.5900331735610962f, -0.20737391710281372f, 0.0666789785027504f, + 0.08181343227624893f, -0.26071205735206604f, -0.2220151722431183f, -0.257919579744339f, + 0.23278546333312988f, -0.09518257528543472f, 0.3925735056400299f, -0.031985804438591f, + 0.19576893746852875f, -0.25825151801109314f, 0.39228859543800354f, 0.42135313153266907f, + -0.3460484743118286f, -0.6032392978668213f, -0.4207472503185272f, -0.6652505993843079f, + -0.14844170212745667f, -0.20582067966461182f, -0.2750038802623749f, -0.08444196730852127f, + -0.06218747794628143f, 0.32963499426841736f, -0.3371811509132385f, 0.08280754834413528f, + -0.04497191309928894f, -0.6914868354797363f, -0.30968189239501953f, 0.05485693737864494f, + 0.5302237868309021f, 0.10639102756977081f, 0.0855746790766716f, -0.3997032642364502f, + -0.1341291218996048f, 0.49649474024772644f, -0.040979571640491486f, -0.06466677784919739f, + -0.4799630641937256f, 0.03065689653158188f, 0.1308591216802597f, -0.24697871506214142f, + 0.2297981232404709f, -0.3347514867782593f, 0.36923325061798096f, 0.5028753876686096f, + -0.15605232119560242f, 0.7693919539451599f, 0.324622243642807f, 0.057381320744752884f, + 0.07726196944713593f, 0.09613212943077087f, -0.10563689470291138f, -0.06854486465454102f, + -0.25765880942344666f, 0.3245580196380615f, -0.06287814676761627f, 0.7587410807609558f, + 0.09494870156049728f, -0.1230400800704956f, 0.3981558382511139f, 0.044513169676065445f, + 0.6811324954032898f, 0.13998186588287354f, 0.14610731601715088f, 0.006240556947886944f, + 0.7803986668586731f, 0.2171136885881424f, -0.9101721048355103f, -0.3578149974346161f, + -0.4791676700115204f, 0.24093949794769287f, -0.2303052693605423f, -0.05040895566344261f, + -0.44654738903045654f, 0.07136791944503784f, 0.14813725650310516f, 1.208767294883728f, + -0.38886675238609314f, 0.6543116569519043f, 0.08247247338294983f, -0.4550500810146332f, + -0.10845766961574554f, 0.4053417444229126f, 0.04241393133997917f, 0.003301314078271389f, + 0.015617990866303444f, -0.16417011618614197f, -0.78692626953125f, 0.25025972723960876f, + -0.34570881724357605f, -0.03743964806199074f, -0.5932687520980835f, 0.34881141781806946f, + -0.18716201186180115f, -0.026957044377923012f, 0.25000032782554626f, 0.15763449668884277f, + -0.03521738201379776f, 0.09655158966779709f, 0.3099806010723114f, -0.2571534514427185f, + -0.26140639185905457f, -0.05543451011180878f, 0.4030987620353699f, 0.09938368201255798f, + 0.4183371961116791f, 0.008939091116189957f, -0.1306128054857254f, -0.22973188757896423f, + -0.4569059908390045f, -0.22407282888889313f, -0.02363731898367405f, -0.3226625919342041f, + 0.03415939584374428f, 0.646252453327179f, 0.06004170700907707f, -0.18533262610435486f, + -0.25300198793411255f, -0.12023479491472244f, -0.24665622413158417f, 0.21922630071640015f, + 0.14248321950435638f, -0.2948630154132843f, 0.06332268565893173f, 0.8758218884468079f, + -0.8196413516998291f, 0.21089470386505127f, 0.2779034972190857f, 0.7404711842536926f, + 0.18119502067565918f, 0.27656930685043335f, 0.39561283588409424f, -0.41638773679733276f, + 0.2467481940984726f, 0.11487864702939987f, 0.12723571062088013f, 0.06027925759553909f, + 0.2728009819984436f, 0.10427459329366684f, 0.18590231239795685f, -0.30492740869522095f, + -0.35954156517982483f, -0.18012334406375885f, 0.14583809673786163f, -0.3217312693595886f, + 0.22032096982002258f, 0.8700504302978516f, -0.01914898492395878f, -0.8495885133743286f, + -0.4552241563796997f, -0.29328256845474243f, 0.016128458082675934f, -0.08096498250961304f, + -0.07367789000272751f, -0.44388166069984436f, -0.07881498336791992f, 0.16826149821281433f, + -0.10668208450078964f, -0.36661943793296814f, 0.08996971696615219f, 0.1740334928035736f, + 0.14061838388442993f, -0.4176405370235443f, -0.09189386665821075f, -0.1727631837129593f, + -0.6723613739013672f, -0.7774737477302551f, 0.1745104193687439f, -0.0069109289906919f, + 0.03611505404114723f, 0.21071484684944153f, 0.9078855514526367f, 0.49794262647628784f, + -0.6352639198303223f, 1.041084885597229f, 0.30637815594673157f, -0.9136573076248169f, + 0.034741032868623734f, 0.05598245561122894f, 0.2741699516773224f, -0.3031800091266632f, + -0.9250045418739319f, -0.19739428162574768f, -0.9629912376403809f, 0.2672995924949646f, + 0.15815384685993195f, 0.4821484386920929f, 0.06301682442426682f, -0.021579129621386528f, + -0.015606082044541836f, -0.18244123458862305f, 0.16984495520591736f, 0.37532278895378113f, + -0.1713021993637085f, 0.057753659784793854f, 0.04721484333276749f, 0.3741018772125244f, + -0.3831857144832611f, -0.21995827555656433f, 0.37593093514442444f, 0.06520190834999084f, + 0.23021525144577026f, 0.5226160883903503f, 0.18702594935894012f, 0.17663982510566711f, + -0.48334118723869324f, 0.15548022091388702f, 0.08952636271715164f, -0.23777168989181519f, + -0.13993266224861145f, -0.12650863826274872f, 0.30107253789901733f, 0.14219872653484344f, + -0.474923700094223f, 0.2449588030576706f, -0.8216109275817871f, 0.26436367630958557f, + -0.07064134627580643f, 0.6592618823051453f, 0.08359500020742416f, 0.6338295340538025f, + -0.2995147705078125f, 0.17025211453437805f, 0.000793169136159122f, -0.1378134936094284f, + 0.04602516070008278f, -0.002303259912878275f, 0.3151547908782959f, -0.852576732635498f, + -0.12707354128360748f, 0.4680027365684509f, 0.8473384380340576f, -0.11135178059339523f, + -0.06886235624551773f, -0.18094368278980255f, 0.6117734313011169f, 0.7629192471504211f, + -0.06406066566705704f, -0.2640898525714874f, -0.20914608240127563f, -0.8697301149368286f, + 0.45285409688949585f, 0.45638975501060486f, 0.5261545181274414f, -0.4951173663139343f, + -0.7837493419647217f, 0.3439026176929474f, -0.7314008474349976f, -0.13025982677936554f, + -0.09878755360841751f, -0.7245875000953674f, -0.4865923225879669f, 0.598723292350769f, + 0.6042801737785339f, 0.21225130558013916f, -0.032431479543447495f, -0.31782180070877075f, + -0.4721525013446808f, 0.029980197548866272f, -0.053849466145038605f, -1.0094914436340332f, + 0.25041139125823975f, 0.6240552067756653f, 0.8414186835289001f, -0.28102317452430725f, + 0.07052736729383469f, 0.08850985020399094f, 0.8300207853317261f, 0.28072482347488403f, + 0.2116929590702057f, -0.4083447754383087f, -0.554104745388031f, -1.0379745960235596f, + 0.13657432794570923f, 0.787078857421875f, -0.05660642683506012f, -0.6585412621498108f, + -0.7910969853401184f, 0.19156433641910553f, 0.508884847164154f, 0.01674194633960724f, + -0.23230083286762238f, -0.42490190267562866f, -0.06970442086458206f, -0.19646912813186646f, + 0.41579824686050415f, -0.14050468802452087f, -0.0958169549703598f, -0.14563429355621338f, + 0.10233160853385925f, 0.10450030118227005f, 0.010755028575658798f, -0.35685691237449646f, + 0.5110058188438416f, -0.366148442029953f, -0.3364986181259155f, -0.5255833268165588f, + -0.3251032829284668f, -0.2341415137052536f, 0.33868274092674255f, 0.6862916946411133f, + 0.1401270031929016f, 0.41534966230392456f, -0.2722761034965515f, 0.24871471524238586f, + -0.35144171118736267f, -0.042133230715990067f, -0.06933438777923584f, 0.22835670411586761f, + -0.054187923669815063f, -0.2136775702238083f, 0.019315214827656746f, + -0.050901349633932114f, -0.18475480377674103f, -0.23483304679393768f, 0.07313291728496552f, + -0.36190885305404663f, 0.0899462103843689f, 0.15504850447177887f, 0.10569758713245392f, + -0.16140881180763245f, 0.04084476828575134f, 0.38489800691604614f, -0.36304908990859985f, + -0.4357805848121643f, 0.5834512710571289f, -0.5182656049728394f, -0.0709775984287262f, + -0.5916424989700317f, -0.44964399933815f, -0.1492723524570465f, 0.5676036477088928f, + 0.6954147219657898f, 0.42057564854621887f, 0.1799532026052475f, -0.044140275567770004f, + 0.07495976239442825f, -0.3054050803184509f, -0.07843945175409317f, 0.08970359712839127f, + 0.08244586735963821f, 0.1870817393064499f, 0.007928984239697456f, 0.17378148436546326f, + -0.018975820392370224f, -0.21588557958602905f, -0.605201005935669f, 0.24244695901870728f, + -0.12700259685516357f, 0.2643260359764099f, 0.03982829675078392f, -0.10875667631626129f, + -0.0098826102912426f, -0.37230753898620605f, 0.18367156386375427f, 0.37846362590789795f, + -0.40725961327552795f, -0.1626349687576294f, -0.5824565291404724f, 0.32144400477409363f, + 0.12241137772798538f, -0.16323155164718628f, 0.1077992394566536f, 0.0302304457873106f, + 0.1366400122642517f, -0.031710460782051086f, 0.17127758264541626f, 0.23819495737552643f, + 0.2966558635234833f, -0.3384033441543579f, 0.8317639827728271f, 0.24288348853588104f, + 0.03895162045955658f, -0.5262442231178284f, 0.2778703570365906f, -1.0587685108184814f, + 0.02823498286306858f, 0.46356257796287537f, -0.22960764169692993f, -0.31631436944007874f, + -0.22489875555038452f, 0.21137022972106934f, -0.3423888087272644f, -0.040422145277261734f, + 0.32967326045036316f, -0.05014810338616371f, 0.4663337469100952f, -0.09438212215900421f, + -0.24248290061950684f, -0.08399742096662521f, -0.43082553148269653f, 0.17035949230194092f, + 0.1274539977312088f, -0.14289195835590363f, 0.17526768147945404f, -0.12283144891262054f, + -0.07376116514205933f, -0.7404740452766418f, 0.4319543242454529f, -0.01500926073640585f, + 0.6056957244873047f, 0.0629267692565918f, 0.5408875346183777f, -0.30627185106277466f, + 0.058143146336078644f, -0.3541799485683441f, -0.0141024524345994f, 0.1267186850309372f, + 0.24158290028572083f, 0.06680478155612946f, -0.23400020599365234f, -0.5467382669448853f, + -0.09498383104801178f, -0.09429619461297989f, -0.06768077611923218f, 0.4716065526008606f, + 0.15454278886318207f, -0.1374075561761856f, 0.02325879968702793f, -0.0838790312409401f, + 0.11530954390764236f, -0.1490313559770584f, -0.060450728982686996f, -0.08865965902805328f, + -0.6292425394058228f, 0.035946398973464966f, -0.14876477420330048f, 0.2928127646446228f, + 0.1581057459115982f, -0.32010796666145325f, 0.37932220101356506f, 0.09494031965732574f, + -0.2787003815174103f, -0.4683394134044647f, -0.051372479647397995f, 0.5179586410522461f, + -0.21640977263450623f, 0.0522945336997509f, 0.34227657318115234f, -0.5084550976753235f, + 0.009885052219033241f, 0.2579268217086792f, -0.1634274572134018f, -0.028448013588786125f, + 0.13567356765270233f, 0.21164393424987793f, -0.5504357814788818f, 0.3877277076244354f, + 0.12683266401290894f, -0.04599796235561371f, -0.15787510573863983f, -0.20454302430152893f, + -0.2485942840576172f, -0.30261582136154175f, -0.39618295431137085f, -0.10237300395965576f, + -0.2992267608642578f, -0.12179462611675262f, 0.05862067639827728f, 0.4558715522289276f, + 0.49318644404411316f, -0.28316250443458557f, 0.8934075236320496f, 0.3948947787284851f, + -0.7018772959709167f, 0.008871478959918022f, 0.19350804388523102f, 0.42821845412254333f, + -0.5009680390357971f, -0.17468640208244324f, -0.051448896527290344f, -0.5384035706520081f, + 0.05987801402807236f, 0.26254382729530334f, -0.1329416036605835f, 0.19075517356395721f, + 0.002006921684369445f, -0.5934259295463562f, -0.2790988087654114f, 0.1304180771112442f, + -0.11388646811246872f, 0.15932397544384003f, -0.08231991529464722f, 0.29834502935409546f, + -0.28769931197166443f, -0.18914566934108734f, -0.14139693975448608f, 0.3811367452144623f, + 0.8943368196487427f, -0.3350769877433777f, -0.733103334903717f, 0.4288082718849182f, + 0.7438012957572937f, -0.755225419998169f, -0.44139963388442993f, -0.16206666827201843f, + -0.3577750325202942f, -0.435754656791687f, -0.15965580940246582f, 0.44925275444984436f, + -0.2081562727689743f, -0.14114800095558167f, 0.3206135332584381f, -1.0114829540252686f, + -0.14336338639259338f, -0.08491828292608261f, -0.5027012228965759f, -1.1324446201324463f, + 0.7435458898544312f, -0.1505540907382965f, 0.5064373016357422f, -0.13465939462184906f, + -0.40224164724349976f, 0.05203772708773613f, 0.022681981325149536f, 0.8022064566612244f, + -0.7043471336364746f, -0.26872578263282776f, -0.06937218457460403f, 0.5089896321296692f, + 0.1377260386943817f, -0.0019696007948368788f, -0.26047953963279724f, 0.3104322552680969f, + 0.0855858251452446f, 0.16882142424583435f, -0.23841524124145508f, 0.1650375872850418f, + 0.20909762382507324f, -0.3913966417312622f, -0.09647224843502045f, 0.4004438519477844f, + 0.07412099838256836f, -0.8027051091194153f, 0.45467495918273926f, -1.0553187131881714f, + -0.330216646194458f, 0.0815214067697525f, -1.0498559474945068f, -0.6223840713500977f, + -0.3629399538040161f, -0.011116833426058292f, -0.01979384571313858f, -0.23612046241760254f, + -0.4025976359844208f, 0.4773606061935425f, -0.05579954385757446f, -0.37302425503730774f, + -0.29860836267471313f, -0.008714301511645317f, -0.17369703948497772f, 0.0741717740893364f, + -0.05772795155644417f, -0.23126894235610962f, 0.056466784328222275f, 0.4328966736793518f, + 0.1768350601196289f, 0.2692948877811432f, -0.11070919036865234f, -0.23626932501792908f, + -0.1808944195508957f, -0.0471813902258873f, 0.41253167390823364f, -0.21143393218517303f, + 0.04602552950382233f, -0.14970648288726807f, -0.34424805641174316f, -0.4230063855648041f, + -0.40506941080093384f, -0.3794063925743103f, 0.7920340299606323f, -0.4939767122268677f, + -0.4613063335418701f, -0.029549922794103622f, -0.007761359214782715f, -0.086216039955616f, + -0.12593524158000946f, 0.44612735509872437f, 0.04646465554833412f, -0.2726566195487976f, + -0.25621873140335083f, 0.3182326555252075f, -0.2464287430047989f, -0.001358022796921432f, + -0.04225081205368042f, -0.21481595933437347f, 0.0621643140912056f, 0.36715802550315857f, + 0.29086679220199585f, 0.2430940717458725f, -0.4624873995780945f, -0.23671072721481323f, + 0.011078139767050743f, -0.06392358988523483f, 0.3028055727481842f, -0.10176103562116623f, + 0.059716109186410904f, 0.021988535299897194f, -0.33655786514282227f, -0.2587900161743164f, + -0.506909966468811f, -0.1363111138343811f, 0.9405393004417419f, -0.35872355103492737f, + 0.5268040299415588f, -0.39617016911506653f, -0.04776310548186302f, -0.23046889901161194f, + 0.21662089228630066f, 0.31363946199417114f, -0.6202867031097412f, -0.3707340955734253f, + -0.43156731128692627f, -0.40593284368515015f, 0.42371150851249695f, 0.40143123269081116f, + 0.6822478175163269f, 0.4813128709793091f, 0.45337608456611633f, 0.05490715056657791f, + -0.20296885073184967f, -0.33285269141197205f, -0.986384928226471f, -0.35647183656692505f, + 0.36988380551338196f, -0.3554286062717438f, -0.3494953513145447f, -0.20584772527217865f, + 0.13929714262485504f, -0.21343939006328583f, -0.0005721314810216427f, 0.6549791693687439f, + -0.0898309275507927f, -0.035299111157655716f, -0.2771172523498535f, 0.45114865899086f, + 0.5786278247833252f, -0.08229086548089981f, 0.3528312146663666f, -0.2207300066947937f, + -0.014431103132665157f, 0.030746571719646454f, -0.3072320222854614f, -0.13286323845386505f, + -0.013863096944987774f, -0.030194414779543877f, 0.2320779412984848f, 0.28864142298698425f, + 0.7905232310295105f, 0.2362404316663742f, 0.04354792460799217f, 0.27758553624153137f, + -0.25825628638267517f, -0.3183424770832062f, -0.6835805177688599f, -0.7437856197357178f, + -0.09567305445671082f, -0.07029950618743896f, -0.08970872312784195f, -0.1045016348361969f, + 0.416835218667984f, -0.446024090051651f, 0.12570415437221527f, 0.6599895358085632f, + 0.07445510476827621f, -0.13402894139289856f, -0.5620865225791931f, 0.5679395794868469f +}; + +} // namespace qwen4_exp_qsa_goldens diff --git a/tests/vllm/models/test_qwen4_exp_qsa.cpp b/tests/vllm/models/test_qwen4_exp_qsa.cpp new file mode 100644 index 000000000..40454caac --- /dev/null +++ b/tests/vllm/models/test_qwen4_exp_qsa.cpp @@ -0,0 +1,622 @@ +// Qwen4-Exp W4 — Qwen Sparse Attention UNIT GATE (row MODEL-MM-QWEN4-EXP, +// issue #1991, .agents/specs/qwen4-exp-flash-next.md). +// +// The goldens in `fixtures/qwen4_exp_qsa_goldens.inc` are the output of the +// UNMODIFIED `Qwen4ExpTextQSAIndexer.forward` at transformers 5.16.0, this row's +// accepted ALGORITHM lane pin (vLLM registers no `qwen4_exp` at 6a5e8f5979, so +// there is nothing on the primary oracle to run). See +// `fixtures/gen_qwen4_exp_qsa_goldens.py` for exactly what was captured and what +// little was transcribed. +// +// TWO ORACLES HERE NEED NO WEIGHTS AND NO GPU, and both are used: +// +// (1) Below `indexer_budget + compress_ratio - 1` cached tokens every candidate +// is selected, so QSA must be BIT-IDENTICAL to dense attention. llama.cpp +// #27742 measures a max logit delta of 0.0 over all 2051 such rows. That +// gates the whole selection, ordering and masking path. +// (2) Above it, the selected token index sets must equal the transformers +// reference, INCLUDING the ragged tail, which is always attended. +// +// AND ONE GATE THAT IS NOT ABOUT CORRECTNESS. `keys_visited` asserts that the +// gather reads only the selected rows. It COUNTS reads at the key-row read; the +// first revision assigned it `sel.size()`, which made the assertion a statement +// about `indices` rather than about the loop, and a gather body doing the full +// dense work passed it (W4 fresh review, mutation M22c). A mask-only QSA is CORRECT and would pass +// every value comparison in this file while forfeiting the long-context speed +// lever the row is aiming at (llama.cpp #27739: a sparse mask over a dense cache +// costs the same as dense attention under CUDA flash attention, because +// `flash_attn_mask_to_KV_max` only scans back to the first tile that is not all +// -inf). Delete the `keys_visited` cases and this suite stops being able to tell +// the two implementations apart. +// +// SCOPE, honestly. Host/CPU reference math only. `Qwen4ExpTextModel` does not +// exist yet (W2/W3/W6a are three sibling waves and the registry wiring is W5), +// so nothing here enters through a production entry point; the spec lists the +// slice under `## Owed` and issue #1978 owns the wiring. No checkpoint, no GPU, +// no speed claim — the speed axis opens at G4, after W6a. +#include "vllm/model_executor/models/qwen4_exp_qsa.h" + +#include + +#include +#include +#include +#include +#include +#include + +#include "fixtures/qwen4_exp_qsa_goldens.inc" + +using namespace vllm::qwen4_exp; +namespace g = qwen4_exp_qsa_goldens; + +namespace { + +// The tiny config the goldens were captured under. Kept beside the goldens' +// own constants so a regenerated fixture with different shapes cannot silently +// keep passing against a stale literal. +QsaConfig GoldenConfig() { + QsaConfig cfg; + cfg.index_n_heads = g::kIndexNHeads; + cfg.index_kv_heads = g::kIndexKvHeads; + cfg.index_head_dim = g::kIndexHeadDim; + cfg.token_budget = g::kTokenBudget; + cfg.compress_ratio = g::kCompressRatio; + cfg.rotary_dim = g::kRotaryDim; + cfg.rms_norm_eps = g::kRmsNormEps; + return cfg; +} + +std::vector Slice(const float* p, int64_t n) { + return std::vector(p, p + n); +} + +double RelL2(const std::vector& a, const float* b, int64_t n) { + double num = 0.0, den = 0.0; + for (int64_t i = 0; i < n; ++i) { + const double d = static_cast(a[i]) - static_cast(b[i]); + num += d * d; + den += static_cast(b[i]) * static_cast(b[i]); + } + return std::sqrt(num) / std::max(std::sqrt(den), 1e-30); +} + +// One golden case, resolved from the flat arrays the fixture emits. +struct Case { + const char* name; + int64_t seq; + const float* q_raw; + const float* k_raw; + const float* cos; + const float* sin; + const float* q_norm_w; + const float* k_norm_w; + const float* q_post; + const float* block_keys; + const int32_t* selected; + const float* attn_q; + const float* attn_k; + const float* attn_v; + const float* attn_out; +}; + +const Case kSubBudget{"sub_budget", g::kSubBudgetSeq, g::kSubBudgetQRaw, + g::kSubBudgetKRaw, g::kSubBudgetCos, g::kSubBudgetSin, + g::kSubBudgetQNormW, g::kSubBudgetKNormW, g::kSubBudgetQPost, + g::kSubBudgetBlockKeys, g::kSubBudgetSelected, + g::kSubBudgetAttnQ, g::kSubBudgetAttnK, g::kSubBudgetAttnV, + g::kSubBudgetAttnOut}; + +const Case kOverBudget{"over_budget", g::kOverBudgetSeq, g::kOverBudgetQRaw, + g::kOverBudgetKRaw, g::kOverBudgetCos, g::kOverBudgetSin, + g::kOverBudgetQNormW, g::kOverBudgetKNormW, g::kOverBudgetQPost, + g::kOverBudgetBlockKeys, g::kOverBudgetSelected, + g::kOverBudgetAttnQ, g::kOverBudgetAttnK, g::kOverBudgetAttnV, + g::kOverBudgetAttnOut}; + +// q after `q_layernorm` and the partial rope, [seq, H, D]. +std::vector BuildQ(const Case& c, const QsaConfig& cfg) { + const int64_t H = cfg.index_n_heads, D = cfg.index_head_dim; + const std::vector raw = Slice(c.q_raw, c.seq * H * D); + // The norm is per index head, so [seq * H] rows of width D. + const std::vector normed = + QsaRmsNorm(raw, c.seq * H, D, Slice(c.q_norm_w, D), cfg.rms_norm_eps, + /*round_to_bf16=*/true); + // The rope's cos/sin are per POSITION and shared across the heads, so expand + // the [seq, rotary_dim] table to one row per (position, head). + std::vector cos(c.seq * H * cfg.rotary_dim); + std::vector sin(cos.size()); + for (int64_t t = 0; t < c.seq; ++t) { + for (int64_t h = 0; h < H; ++h) { + for (int64_t d = 0; d < cfg.rotary_dim; ++d) { + cos[(t * H + h) * cfg.rotary_dim + d] = c.cos[t * cfg.rotary_dim + d]; + sin[(t * H + h) * cfg.rotary_dim + d] = c.sin[t * cfg.rotary_dim + d]; + } + } + } + return QsaApplyRotaryLeadingHalf(normed, c.seq * H, D, cos, sin, cfg.rotary_dim, + /*round_to_bf16=*/true); +} + +// The side cache: one pooled key per complete block over the whole case. +std::vector BuildBlockKeys(const Case& c, const QsaConfig& cfg) { + const int64_t D = cfg.index_head_dim; + const int64_t complete = (c.seq / cfg.compress_ratio) * cfg.compress_ratio; + return QsaCompressNormRope(Slice(c.k_raw, c.seq * D), complete, + Slice(c.k_norm_w, D), + Slice(c.cos, c.seq * cfg.rotary_dim), + Slice(c.sin, c.seq * cfg.rotary_dim), cfg, + /*round_to_bf16=*/true); +} + +// The full per-query selection, ascending and -1 padded to index_width(). +std::vector SelectFor(const QsaConfig& cfg, + const std::vector& q, + const std::vector& block_keys, + int64_t query_idx) { + const int64_t H = cfg.index_n_heads, D = cfg.index_head_dim; + const int64_t kv_len = query_idx + 1; // causal, no interior masking + const int64_t nb = kv_len / cfg.compress_ratio; + std::vector q_row(q.begin() + query_idx * H * D, + q.begin() + (query_idx + 1) * H * D); + std::vector keys(block_keys.begin(), block_keys.begin() + nb * D); + const std::vector scores = QsaBlockScores(q_row, keys, nb, cfg); + const std::vector blocks = + QsaTopkBlocks(scores, nb, std::min(cfg.block_topk(), nb)); + return QsaSelectedTokenIndices(blocks, nb, kv_len, cfg); +} + +// Which refusal fired. Two config refusals can cover for each other, and +// `CHECK_THROWS` cannot tell them apart: it only reports that SOMETHING threw. +// VT_CHECK appends ` at :`, so match on the message text only — an +// exact-`what()` assertion would go stale on any edit above the line it names. +template +bool RefusesWith(F&& fn, const char* needle) { + try { + fn(); + } catch (const std::exception& e) { + return std::string(e.what()).find(needle) != std::string::npos; + } + return false; +} + +int64_t SelectedCount(const std::vector& idx) { + int64_t n = 0; + for (int32_t v : idx) { + if (v < 0) break; + ++n; + } + return n; +} + +} // namespace + +// ── The config invariants ──────────────────────────────────────────────────── + +TEST_CASE("qsa-config: the published shape resolves, and every upstream refusal fires") { + QsaConfig cfg; // the published Qwen3.8-Flash-Next values + CHECK(cfg.index_n_heads == 4); + CHECK(cfg.index_kv_heads == 1); + CHECK(cfg.index_head_dim == 128); + CHECK(cfg.token_budget == 2048); + CHECK(cfg.compress_ratio == 4); + CHECK(cfg.block_topk() == 512); + // budget + compress_ratio - 1: the incomplete trailing block is attended ON + // TOP of the budget, which is why the buffer is not just `budget` wide. + CHECK(cfg.index_width() == 2051); + QsaValidateConfig(cfg); + + // configuration_qwen4_exp.py:221 — "Qwen4-Exp QSA requires indexer_kv_heads=1." + QsaConfig two_kv = cfg; + two_kv.index_kv_heads = 2; + CHECK_THROWS(QsaValidateConfig(two_kv)); + + // :223 — "indexer_budget must be divisible by indexer_compress_ratio." + QsaConfig ragged = cfg; + ragged.token_budget = 2049; + CHECK_THROWS(QsaValidateConfig(ragged)); + + // :227-231 — the attention rope must FIT the index head. The probe is EVEN, + // so the evenness refusal below cannot cover for this one. `index_head_dim + 1` + // was the original probe and it is odd: EITHER refusal alone still threw on it, + // so deleting this one — the refusal upstream actually raises — left the suite + // green (W4 fresh review, mutation M3). + QsaConfig wide_rope = cfg; + wide_rope.rotary_dim = cfg.index_head_dim + 2; + CHECK(RefusesWith([&] { QsaValidateConfig(wide_rope); }, + "RoPE dimensions must fit the QSA index head")); + + // OURS, with no upstream counterpart: `rotate_half` needs an even span. The + // probe FITS the index head, so only this refusal can fire on it (M3b). + QsaConfig odd_rope = cfg; + odd_rope.rotary_dim = cfg.index_head_dim - 1; + CHECK(RefusesWith([&] { QsaValidateConfig(odd_rope); }, + "rotary_dim must be even")); + + QsaConfig zero = cfg; + zero.index_n_heads = 0; + CHECK_THROWS(QsaValidateConfig(zero)); +} + +// ── The side cache ─────────────────────────────────────────────────────────── + +TEST_CASE("qsa-cache: one key vector per compress_ratio tokens, 64 B/token/layer") { + const QsaConfig cfg; + const QsaSideCacheSpec spec = QsaMakeSideCacheSpec(cfg, /*elem_bytes=*/2); + // MLAAttentionSpec(num_kv_heads=1, head_size=128, tokens_per_state=4). + CHECK(spec.num_kv_heads == 1); + CHECK(spec.head_size == 128); + CHECK(spec.tokens_per_state == 4); + + // KEY-ONLY: one vector per state, not 2x for K+V. 2 * 1 * 128 / 4 = 64. + CHECK(spec.BytesPerTokenPerLayer() == 64); + // A quarter of what a per-token index cache would cost, which is the whole + // argument for compressing at all. + CHECK(spec.BytesPerTokenPerLayer() * spec.tokens_per_state == 2 * 128); + CHECK(spec.BytesForTokens(4096) == 4096 * 64); + + // Only a COMPLETE block produces a state: the compressor early-exits unless + // `(position + 1) % compress_ratio == 0`. The ragged tail costs no state. + CHECK(spec.StatesForTokens(0) == 0); + CHECK(spec.StatesForTokens(3) == 0); + CHECK(spec.StatesForTokens(4) == 1); + CHECK(spec.StatesForTokens(7) == 1); + CHECK(spec.StatesForTokens(8) == 2); + + // get_compressed_slot_mapping's per-token body: PAD_ID -1 off a boundary. + CHECK(QsaCompressedSlot(0, 4) == -1); + CHECK(QsaCompressedSlot(2, 4) == -1); + CHECK(QsaCompressedSlot(3, 4) == 0); + CHECK(QsaCompressedSlot(7, 4) == 1); + CHECK(QsaCompressedSlot(11, 4) == 2); +} + +// ── The indexer's own math, against the oracle ─────────────────────────────── + +TEST_CASE("qsa-indexer: the block score is relu(q.k) summed over heads, / sqrt(D)") { + // A HAND-DERIVED case, because the selection goldens cannot see this. Top-k is + // invariant under any positive rescale of every score, so inheriting + // DeepSeek-V4's extra `head_scale = n_heads ** -0.5` fold + // (models/deepseek_v4/attention.py:930) changes every score and moves no + // selection at all. A mutation proved that: it left the whole suite green + // until this case existed. The constant still matters — it is the logits' + // dynamic range, which is what an fp8 indexer kernel is quantised against. + QsaConfig cfg; + cfg.index_n_heads = 2; + cfg.index_head_dim = 4; + cfg.token_budget = 4; + cfg.compress_ratio = 2; + cfg.rotary_dim = 2; + + // q[h=0] = e0, q[h=1] = e1. + const std::vector q = {1, 0, 0, 0, 0, 1, 0, 0}; + const std::vector keys = { + 2, 3, 0, 0, // dots 2 and 3 -> relu sum 5 -> 5 / sqrt(4) = 2.5 + -1, -1, 0, 0, // dots -1 and -1 -> BOTH clamped -> 0 + 4, -2, 0, 0, // dots 4 and -2 -> only the first survives -> 4/2 = 2.0 + }; + const std::vector scores = QsaBlockScores(q, keys, 3, cfg); + REQUIRE(scores.size() == 3); + CHECK(scores[0] == doctest::Approx(2.5)); + CHECK(scores[1] == doctest::Approx(0.0)); + CHECK(scores[2] == doctest::Approx(2.0)); + // Without the ReLU block 1 would score -1 and block 2 would score 1.0, so the + // two are ordered differently as well as scaled differently. + CHECK(scores[0] > scores[2]); + CHECK(scores[2] > scores[1]); +} + +TEST_CASE("qsa-indexer: q_layernorm + partial rope match transformers 5.16.0") { + const QsaConfig cfg = GoldenConfig(); + for (const Case* c : {&kSubBudget, &kOverBudget}) { + CAPTURE(std::string(c->name)); // doctest stringifies a char* as a bool + const std::vector q = BuildQ(*c, cfg); + const int64_t n = c->seq * cfg.index_n_heads * cfg.index_head_dim; + REQUIRE(static_cast(q.size()) == n); + // Both sides are bf16-valued, so this is exact rather than a tolerance. + for (int64_t i = 0; i < n; ++i) CHECK(q[i] == c->q_post[i]); + } +} + +TEST_CASE("qsa-indexer: mean-pooled, normed, block-start-roped keys match the oracle") { + const QsaConfig cfg = GoldenConfig(); + for (const Case* c : {&kSubBudget, &kOverBudget}) { + CAPTURE(std::string(c->name)); // doctest stringifies a char* as a bool + const std::vector keys = BuildBlockKeys(*c, cfg); + const int64_t nb = c->seq / cfg.compress_ratio; + REQUIRE(static_cast(keys.size()) == nb * cfg.index_head_dim); + // Tight but not exact: the oracle's mean reduces in torch's order and ours + // in ascending token order over `compress_ratio` = 4 terms. Everything after + // the pool — the .to(dtype) round-trip, k_layernorm, the block-start rope — + // is reproduced operation for operation. + CHECK(RelL2(keys, c->block_keys, nb * cfg.index_head_dim) < 1e-6); + } +} + +TEST_CASE("qsa-indexer: the pool is a MEAN, not a sum — the /compress_ratio is pinned") { + // A HAND-DERIVED case, because no golden can see this constant. `k_layernorm` + // runs on the POOLED key, and RMSNorm is scale-invariant whenever its epsilon + // is negligible against the mean square. At the published eps = 1e-6, dropping + // the `/ compress_ratio` and storing a SUM therefore changes nothing any + // downstream value can observe — it left the whole suite green (W4 fresh + // review, mutation M12) — and `/4` is exact in binary floating point, so not + // even the bf16 round-trip catches it. The window is gated (M14) and the + // round-trip is gated (M13); the WEIGHTING was not. + // + // So probe where the norm is not scale-invariant. With eps dominating the mean + // square the norm is linear in its input, the pooled scale reaches the output, + // and mean versus sum is a factor of compress_ratio. Everything else is + // neutralised: cos = 1 and sin = 0 make the rope the identity, the norm weight + // is zero so `(1.0 + w)` is 1, and round_to_bf16 is off — which is what makes + // every expected value below EXACT rather than toleranced. + QsaConfig cfg; + cfg.index_n_heads = 1; + cfg.index_kv_heads = 1; + cfg.index_head_dim = 4; + cfg.compress_ratio = 2; + cfg.token_budget = 2; + cfg.rotary_dim = 2; + cfg.rms_norm_eps = 10.0f; + + // Two blocks of two raw keys each. Block 0 MEAN-pools to [4, 6, 8, 10]: mean + // square 54, + eps = 64, so the norm divides by exactly 8. Block 1 pools to + // [1, 3, 5, 5]: mean square 15, + eps = 25, so it divides by exactly 5. + const std::vector raw = { + 2.0f, 4.0f, 6.0f, 8.0f, // block 0, key 0 + 6.0f, 8.0f, 10.0f, 12.0f, // block 0, key 1 + 0.0f, 2.0f, 4.0f, 4.0f, // block 1, key 0 + 2.0f, 4.0f, 6.0f, 6.0f}; // block 1, key 1 + const std::vector w(4, 0.0f); + const std::vector cos(4 * cfg.rotary_dim, 1.0f); + const std::vector sin(4 * cfg.rotary_dim, 0.0f); + + const std::vector got = + QsaCompressNormRope(raw, 4, w, cos, sin, cfg, /*round_to_bf16=*/false); + const std::vector want = {0.5f, 0.75f, 1.0f, 1.25f, + 0.2f, 0.6f, 1.0f, 1.0f}; + REQUIRE(got.size() == want.size()); + for (size_t i = 0; i < got.size(); ++i) CHECK(got[i] == want[i]); + // An unnormalised SUM would pool block 0 to [8, 12, 16, 20], mean square 216, + // + eps = 226, and emit 8/sqrt(226) = 0.5322... where 0.5 is asserted. +} + +TEST_CASE("qsa-indexer: more selected blocks than the budget allows is refused") { + // `selected_blocks.size() <= cfg.block_topk()`. Deleting it left the suite + // green (W4 fresh review, mutation M34), because the index-width check a few + // lines later throws as well: compress_ratio * (block_topk + 1) is + // token_budget + compress_ratio, exactly ONE token past an index_width() of + // token_budget + compress_ratio - 1. No token count can separate the two, so + // the only available discriminator is WHICH refusal fires, and that is what + // this case asserts. + QsaConfig cfg; + cfg.index_head_dim = 4; + cfg.rotary_dim = 2; + cfg.token_budget = 8; + cfg.compress_ratio = 2; // block_topk() = 4, index_width() = 9 + const int64_t nb = cfg.block_topk() + 1; + const int64_t kv_len = nb * cfg.compress_ratio; + std::vector blocks; + for (int64_t b = 0; b < nb; ++b) blocks.push_back(b); + CHECK(RefusesWith([&] { QsaSelectedTokenIndices(blocks, nb, kv_len, cfg); }, + "more blocks selected than the budget allows")); + + // One block fewer is inside the budget and returns normally, so the refusal + // above is not this function simply refusing everything. + blocks.pop_back(); + const std::vector ok = + QsaSelectedTokenIndices(blocks, nb, kv_len, cfg); + CHECK(SelectedCount(ok) == cfg.block_topk() * cfg.compress_ratio); +} + +TEST_CASE("qsa-indexer: selected token sets equal the oracle, ragged tail included") { + const QsaConfig cfg = GoldenConfig(); + for (const Case* c : {&kSubBudget, &kOverBudget}) { + CAPTURE(std::string(c->name)); // doctest stringifies a char* as a bool + const std::vector q = BuildQ(*c, cfg); + const std::vector keys = BuildBlockKeys(*c, cfg); + for (int64_t qi = 0; qi < c->seq; ++qi) { + CAPTURE(qi); + const std::vector got = SelectFor(cfg, q, keys, qi); + REQUIRE(static_cast(got.size()) == g::kIndexWidth); + for (int64_t j = 0; j < g::kIndexWidth; ++j) { + CHECK(got[j] == c->selected[qi * g::kIndexWidth + j]); + } + } + } +} + +TEST_CASE("qsa-indexer: the ragged tail is ALWAYS attended") { + // Not a restatement of the case above: it names the property, and it fires on + // the queries where the tail is the only thing keeping the count off a + // multiple of compress_ratio. + const QsaConfig cfg = GoldenConfig(); + const Case& c = kOverBudget; + const std::vector q = BuildQ(c, cfg); + const std::vector keys = BuildBlockKeys(c, cfg); + int64_t ragged_queries = 0; + for (int64_t qi = 0; qi < c.seq; ++qi) { + const int64_t kv_len = qi + 1; + const int64_t tail = kv_len % cfg.compress_ratio; + if (tail == 0) continue; + ++ragged_queries; + const std::vector got = SelectFor(cfg, q, keys, qi); + // The last `tail` entries of the ascending buffer are the tail tokens, and + // they are there whatever the scores said. + const int64_t n = SelectedCount(got); + for (int64_t j = 0; j < tail; ++j) { + CHECK(got[n - tail + j] == static_cast(kv_len - tail + j)); + } + } + CHECK(ragged_queries > 0); // a gate that never fired is not a gate +} + +// ── Free oracle (1): sub-budget bit-identity with dense attention ──────────── + +TEST_CASE("qsa-oracle: below budget + compress_ratio - 1 EVERY candidate is selected") { + const QsaConfig cfg = GoldenConfig(); + const Case& c = kSubBudget; + REQUIRE(c.seq == cfg.index_width()); // the largest all-select context + const std::vector q = BuildQ(c, cfg); + const std::vector keys = BuildBlockKeys(c, cfg); + for (int64_t qi = 0; qi < c.seq; ++qi) { + CAPTURE(qi); + const std::vector got = SelectFor(cfg, q, keys, qi); + REQUIRE(SelectedCount(got) == qi + 1); + for (int64_t j = 0; j <= qi; ++j) CHECK(got[j] == static_cast(j)); + for (int64_t j = qi + 1; j < g::kIndexWidth; ++j) CHECK(got[j] == -1); + } +} + +TEST_CASE("qsa-oracle: a sub-budget GATHER is BIT-IDENTICAL to dense attention") { + // llama.cpp #27742 claims a max logit delta of 0.0 over all 2051 rows. This is + // the C++ statement of that claim: with every candidate selected the gather + // reduces over exactly the dense sequence, in exactly the dense order. + const QsaConfig cfg = GoldenConfig(); + const Case& c = kSubBudget; + const int64_t HQ = g::kNumAttentionHeads, HKV = g::kNumKeyValueHeads, + DH = g::kHeadDim; + const std::vector q_all = Slice(c.attn_q, c.seq * HQ * DH); + const std::vector k_all = Slice(c.attn_k, c.seq * HKV * DH); + const std::vector v_all = Slice(c.attn_v, c.seq * HKV * DH); + const std::vector q = BuildQ(c, cfg); + const std::vector keys = BuildBlockKeys(c, cfg); + + for (int64_t qi = 0; qi < c.seq; ++qi) { + CAPTURE(qi); + const int64_t kv_len = qi + 1; + const std::vector q_row(q_all.begin() + qi * HQ * DH, + q_all.begin() + (qi + 1) * HQ * DH); + // Dense: the causal prefix, nothing removed, through the INDEPENDENT walk. + // `want` must not come from the function under test. It did in the first + // revision of this case, and then the `==` said only that the gather is + // deterministic under two equal inputs: scaling its output by 2 left the + // case green (W4 fresh review, mutation M40). + std::vector dense(cfg.index_width(), -1); + for (int64_t j = 0; j < kv_len; ++j) dense[j] = static_cast(j); + int64_t dense_visited = 0, sparse_visited = 0; + const std::vector want = QsaMaskedAttention( + q_row, k_all, v_all, dense, kv_len, HQ, HKV, DH, &dense_visited); + const std::vector sel = SelectFor(cfg, q, keys, qi); + const std::vector got = QsaGatherAttention( + q_row, k_all, v_all, sel, kv_len, HQ, HKV, DH, &sparse_visited); + REQUIRE(got.size() == want.size()); + for (size_t i = 0; i < got.size(); ++i) CHECK(got[i] == want[i]); + // Below budget the gather is not merely correct, it is doing exactly the + // dense amount of work: every candidate is selected, so there is nothing to + // skip. The saving starts above the budget, in the case below. + CHECK(sparse_visited == dense_visited); + } +} + +// ── Free oracle (2): the consumer against the oracle's own masked attention ── + +TEST_CASE("qsa-consumer: the gather reproduces the oracle's masked attention") { + const QsaConfig cfg = GoldenConfig(); + const int64_t HQ = g::kNumAttentionHeads, HKV = g::kNumKeyValueHeads, + DH = g::kHeadDim; + for (const Case* c : {&kSubBudget, &kOverBudget}) { + CAPTURE(std::string(c->name)); // doctest stringifies a char* as a bool + const std::vector q_all = Slice(c->attn_q, c->seq * HQ * DH); + const std::vector k_all = Slice(c->attn_k, c->seq * HKV * DH); + const std::vector v_all = Slice(c->attn_v, c->seq * HKV * DH); + const std::vector q = BuildQ(*c, cfg); + const std::vector keys = BuildBlockKeys(*c, cfg); + std::vector out; + out.reserve(c->seq * HQ * DH); + for (int64_t qi = 0; qi < c->seq; ++qi) { + const std::vector q_row(q_all.begin() + qi * HQ * DH, + q_all.begin() + (qi + 1) * HQ * DH); + const std::vector sel = SelectFor(cfg, q, keys, qi); + const std::vector row = QsaGatherAttention( + q_row, k_all, v_all, sel, qi + 1, HQ, HKV, DH, nullptr); + out.insert(out.end(), row.begin(), row.end()); + } + // The oracle reduces in torch's order over the padded row; we reduce over + // the gathered subset. Same values, different summation order. + CHECK(RelL2(out, c->attn_out, c->seq * HQ * DH) < 2e-3); + } +} + +TEST_CASE("qsa-consumer: gather and mask agree value for value") { + // The mask reference is the red-first path. It must agree exactly, because it + // adds only exact zeros to the same accumulation in the same ascending order. + const QsaConfig cfg = GoldenConfig(); + const int64_t HQ = g::kNumAttentionHeads, HKV = g::kNumKeyValueHeads, + DH = g::kHeadDim; + const Case& c = kOverBudget; + const std::vector q_all = Slice(c.attn_q, c.seq * HQ * DH); + const std::vector k_all = Slice(c.attn_k, c.seq * HKV * DH); + const std::vector v_all = Slice(c.attn_v, c.seq * HKV * DH); + const std::vector q = BuildQ(c, cfg); + const std::vector keys = BuildBlockKeys(c, cfg); + for (int64_t qi = 0; qi < c.seq; ++qi) { + CAPTURE(qi); + const std::vector q_row(q_all.begin() + qi * HQ * DH, + q_all.begin() + (qi + 1) * HQ * DH); + const std::vector sel = SelectFor(cfg, q, keys, qi); + const std::vector gathered = + QsaGatherAttention(q_row, k_all, v_all, sel, qi + 1, HQ, HKV, DH, nullptr); + const std::vector masked = + QsaMaskedAttention(q_row, k_all, v_all, sel, qi + 1, HQ, HKV, DH, nullptr); + REQUIRE(gathered.size() == masked.size()); + for (size_t i = 0; i < gathered.size(); ++i) CHECK(gathered[i] == masked[i]); + } +} + +// ── The gate that a mask-only implementation CANNOT pass ───────────────────── + +TEST_CASE("qsa-consumer: the GATHER touches only the selected rows") { + // This is the wave's whole point, and it is not a correctness property. A mask + // over a dense cache is CORRECT and would pass every case above. llama.cpp + // #27739 names the mechanism by which it is also not faster. + const QsaConfig cfg = GoldenConfig(); + const int64_t HQ = g::kNumAttentionHeads, HKV = g::kNumKeyValueHeads, + DH = g::kHeadDim; + const Case& c = kOverBudget; + const std::vector q_all = Slice(c.attn_q, c.seq * HQ * DH); + const std::vector k_all = Slice(c.attn_k, c.seq * HKV * DH); + const std::vector v_all = Slice(c.attn_v, c.seq * HKV * DH); + const std::vector q = BuildQ(c, cfg); + const std::vector keys = BuildBlockKeys(c, cfg); + + // The two softmax passes: the max, then the weights and the reduction. Each + // reads every row it visits, so reading a row twice over — as a redundant pass + // would — is visible here (W4 fresh review, mutation M31). A single-pass + // online-softmax rewrite would legitimately halve this, and this is where that + // shows up and gets re-derived on purpose. + constexpr int64_t kReadsPerRowPerHead = 2; + + int64_t strictly_sparse_queries = 0; + for (int64_t qi = 0; qi < c.seq; ++qi) { + CAPTURE(qi); + const int64_t kv_len = qi + 1; + const std::vector q_row(q_all.begin() + qi * HQ * DH, + q_all.begin() + (qi + 1) * HQ * DH); + const std::vector sel = SelectFor(cfg, q, keys, qi); + // READS, not selected positions. Each consumer reads one key row per query + // head in each of its two softmax passes, so an honest gather comes out at + // `selected * HQ * 2` and any walk over the whole cache at `kv_len * HQ * 2`. + // `SelectedCount(sel)` alone was the expectation once, matching a counter + // that was assigned `sel.size()`; both sides were then the same quantity + // computed the same way, and a gather body that dot-products every cached + // row and masks the rest — QsaMaskedAttention's cost — passed this case (W4 + // fresh review, mutation M22c). + const int64_t want = SelectedCount(sel) * HQ * kReadsPerRowPerHead; + const int64_t dense = kv_len * HQ * kReadsPerRowPerHead; + int64_t gather_visited = -1, mask_visited = -1; + QsaGatherAttention(q_row, k_all, v_all, sel, kv_len, HQ, HKV, DH, + &gather_visited); + QsaMaskedAttention(q_row, k_all, v_all, sel, kv_len, HQ, HKV, DH, + &mask_visited); + CHECK(gather_visited == want); + CHECK(mask_visited == dense); // the mask cannot do better, by construction + if (want < dense) { + ++strictly_sparse_queries; + CHECK(gather_visited < mask_visited); + } + } + // Above `indexer_budget` the selection MUST discard blocks. A fixture that + // never crossed the budget would leave every assertion above trivially true — + // the exact failure the spec's `## Gates` G2 warns about. + CHECK(strictly_sparse_queries > 0); +}