Repository navigation
model: Flint (flint, 256 experts, sliding-window attention with sinks) #632
Description
Recursive SI's Flint, a MoE model with sliding-window attention.
- 49 layers, the first 2 dense; 256 routed experts, 6 per token, plus a shared expert; sigmoid router. Config: https://huggingface.co/RecursiveSI/flint/raw/main/config.json
- Attention: GQA 32/4, head 128, q/k norm, learnable sinks. Sliding window 513, three sliding layers to one full layer. The full layers are NoPE (
no_rope_layer_indices). Vocabulary 150,144, context 262,144. - License: the safetensors repo says apache-2.0 (https://huggingface.co/RecursiveSI/flint, created 2026-10-21). The unsloth GGUF header says
other,flint-preview.
GGUF: arch flint. https://huggingface.co/unsloth/flint-GGUF (Q8_0, UD-Q2_K_XL to UD-Q6_K_XL, IQ files; UD-Q4_K_XL is 25,054,688,704 B, so a 3090 needs the host tier), https://huggingface.co/RecursiveSI/flint-int4-gguf (q4_1), https://huggingface.co/lmstudio-community/flint-GGUF.
UD-Q4_K_XL routed stacks: gate/up IQ4_XS, Q5_K, Q4_K, IQ3_S; down Q5_K, IQ4_XS, Q6_K, Q8_0.
llama.cpp mainline merged it on 2026-10-21 (ggml-org/llama.cpp#30928), so mainline is the oracle.
Likely reuse [unverified, to check against the code in the design]: sliding-window attention with sinks (as MiMo-V2.6 uses), the sigmoid router and shared expert (GLM, MiMo), NoPE layers in GQA.
Plan: a design round reads the code first and fills the common-owner table of docs/contrib/new-model.md. Then the reader, the body and the serve seat.
Opened by the maintainer with help from an AI assistant (Claude).
Activity
Burntgogi commentedon Oct 22, 2026
I will take the design round requested here: pin the upstream config/GGUF headers and llama.cpp reference, compare them with current common owners, and fill all rows of the new-model common-owner table. I will document tensor/reader and placement requirements, proven reuse versus prerequisite lifts, tokenizer/template behavior, and verification/hand-off gates. This is a design contribution only, not a claim of Flint runtime support or GPU validation; no model-owned planner, expert leg, prompt loop or timing driver will be proposed as a shortcut.
Burntgogi commentedon Oct 22, 2026
Design round delivered in #676 (commit
c2ffbb8). It fills all nine common-owner rows, with pinned source/real-header evidence and an ordered reader → common prerequisites → state/tier → body/seat acceptance plan. This is design only; #632 remains open for implementation.The actual UD-Q4_K_XL header confirms 779 tensors, exact full/NoPE and SWA/RoPE masks, all four EOG IDs, and no router correction bias. The main attachment gaps are now concrete: NoPE QK-norm need generation; the complete head128 sink/SWA/NoPE operation; unbiased sigmoid256/top6 and its normalization policy; shared IQ4_XS in layer42; common card/tier down-form integration and six-slot F32 handoff; and paged/resident-seat plumbing. Existing common owners—not copied model planners or loops—remain the target.
Host probes also observed the Flint pretokenizer, block-set template and replace-filter refusals. The exact original template should be supported by the common tokenizer/Jinja owners, reusing the existing GLM tool grammar. The current ring formula needs1024 rows for window513; file23.334GiB alone is not proof of3090 non-fit. The GGUF license/provenance mismatch remains explicit.
Verification: bounded pinned-header read; independent Python/Rust metadata/byte agreement; config/template comparisons; nine ring-capacity contexts and cuts512/513; the documented reproduction recipe executed successfully; repository comment check passed. No full weights, model inference, GPU, two-card, HTTP-seat or timing claims. Temporary probes were removed. The PR's hand-off table names the required implementation gates before claiming Flint support.
docs: design Flint common-owner attachment (#632)
Burntgogi commentedon Oct 22, 2026
Following merged design #676, I will take the three CPU-side implementation slices in separate PRs: (1) common Jinja block-set capture and replace, checked against the unmodified official template/Jinja2; (2) the exact Flint pretokenizer and four-ID EOG contract, compared against the pinned official tokenizer and mainline; (3) the GGUF reader/description/coverage, with exact masks/shapes/types and named input refusals. Reader-only admission will retain Need::Program: this work does not claim a GPU body, serve seat, inference/performance support, or fixture redistribution license equivalence. I will use the existing common owners and keep the active GPU/chain/serve work separate.
Burntgogi commentedon Oct 22, 2026
Host implementation progress, in the claimed order:
No Flint GPU body, runtime admission, serve seat, fixture redistribution, payload equivalence or performance claim. Final reader PR/evidence will follow after the remaining checks.
Burntgogi commentedon Oct 22, 2026
The three claimed CPU/header slices are now published in order: #677 (common Jinja), #678 (Flint tokenizer/four EOG), #680 (Flint reader/coverage).
Combined smoke from the three exact PR commits: actual hashed GGUF header -> reader -> original stored template -> common renderer -> tokenizer, 51 contexts, byte-exact Jinja2 renders and official HF IDs. EOG=[150001,150003,150005,150018], no automatic BOS/EOS, program=None. Need::Program and numerical/site coverage refusals remain; tokenizer coverage gap disappears with #678 present.
Reader host suites: model463 pass/5 ignored, models31 unit+3 integration, placement175; targeted pinned-header gate passed and separate executable confirmed named malformed-header refusals. Final registration checks: gate_class0 findings, check-arch/comment/fmt pass; LC_ALL=C check-recipes passes145 blocks (225 recipe graph cases, 888 card shell stubs, 99 tree-commit checks). The initial Korean-locale diagnostic expectations failed; no unrelated tooling checks were weakened or changed.
CUDA-aware compile-only check also succeeded: cargo oxide build --arch sm_86 -- -p bloomery-gpu-gates --features glm5next,mimo2 --bin gate_routed_legs. It emitted shared-memory/unroll diagnostics; no NVIDIA card execution, inference, GPU correctness or performance claim.
The remaining runtime prerequisites in the design are still real: numerical operation bindings, quant/site and placement/tier/slot integration, provenance-cleared fixture/refset, body/app/seat and actual end-to-end hardware evidence. These PRs do not close #632.
midagedev commentedon Oct 22, 2026
Thanks for the design and the three CPU slices. The model port is yours; we will not open parallel work on it.
We already have three card-side pieces from the common prerequisites. They land on main in our next train, so you can build on them:
crates/gpu/src/flash_gqa_h128_swa.rs: GQA attention, head 128, sliding window with learnable sinks (just gate-gpu-flint-attn).crates/gpu/src/qk_rms.rs: the q/k RMS norm (same gate).crates/gpu/src/route_body.rs(just gate-gpu-flint-router).We removed our own Flint header reader and tokenizer from that train, so #678 and #680 own those parts. If any of the three pieces does not fit your plan, change it or tell us. The UD-Q4_K_XL file (sha256
63c2ecc4…) is on our box, so we can run card gates for your PRs.(Written with AI help.)
Burntgogi commentedon Oct 23, 2026
Readiness check completed on main
2a7005b8, with the updated handoff and corrected runnable header command in #748.Executed, not just source inspection: fetched only the exact 5,470,528-byte header from pinned revision
20468edfc38ab43d6914cc88b08cbecb05d0fc62, checked its published header SHA, and ran the realgate-flint-metarecipe: 4 reader tests + 1 pinned-header test passed. It read 779 tensors / 49 layers / 2 dense + 47 routed blocks, emitted 49 norm / 35 rotary / 49 sink needs, and still reported the missing operation/program bindings. No full weights, GPU, inference or performance claim.The test's file path must be passed as
BLOOMERY_BOX_ENV="BLOOMERY_MODEL=/absolute/path/flint-header.gguf", not the outerBLOOMERY_MODEL(whichbox.shinterprets as a tool-profile name). I reproduced that command mismatch and corrected the handoff.Current boundary:
max(sum_selected, 2^-14); ordinary shared IQ4_XS; selected card/tier down forms; six-slot F32 handoff; paging/resident-slot and body/app/seat integration.1723485602c1) and serve: keep driver calls away from the process exit's teardown #747 (f883e84f8d2c) do not contain the promised attention/QK-RMS/router train. I checked definitions and launch/shape contracts, not only filenames; their changes instead concern Metal/common-owner moves and serving shutdown.Could you provide the PR or fetchable commit/ref for the three promised CUDA pieces? I will consume those rather than duplicate unpublished work. Does their router already implement the floor and absent/selection-only bias, and does the attention gate cover genuine NoPE plus prompt/ring/slot paths?
Next bounded scope: first make the existing router description/selection contract distinguish normalization policies while preserving current model arithmetic and every unsupported-program/shape refusal. Reconcile that API with your train before binding the device row. Then separate common-operation PRs for shared IQ4_XS, selected down forms and six-slot F32 handoff, followed by paging/slots and thin body/app/seat integration. Device slices require your offered artifact/card runs before admission; no default or coverage shortcut is proposed.
The model port remains mine as agreed. #748 records the dependency and acceptance order; it does not implement the router policy or claim Flint runtime support.
Burntgogi commentedon Oct 24, 2026
The bounded host-policy slice is implemented in #772. Router/RouterRule now distinguish None, legacy SelectedSum and SelectedSumFloor; Flint describes max(sum_selected,2^-14). Existing arithmetic/metadata rendering are preserved, norm_arg rows cannot admit the floor, and no unsupported device row or Program is supplied. Host libraries816passed (5existingignored), the real pinned-header gate passed, and a standalone compiled reader/selector consumer exercised all47routed layers plus a legacy-vs-floor selection difference. The migrated GPU libraries/router gate also compile with the pinned oxide toolchain. Device execution is not claimed. The fetchable common train ref remains the next prerequisite for actual norm/attention/router bindings.
Burntgogi commentedon Oct 24, 2026
Contributor handoff — status after maintainer continuation
The maintainer has closed #632 and merged #781, #782 and #783. After those merges the maintainer merged current main into #799, resolved its former merge conflicts, made it ready for review and reported A6000 gates and serving checks. Their stated plan is to merge #799 after CI and review; current state is on the PR. The cc12.1 resource pins are explicitly provisional copies pending physical GB10 measurement.
On my earlier
182012b3090d2e00051bc736052ef6f583636921source, real Korean natural HTTP reached EOG but disagreed with aligned CPU natural generation at predict position 29 (CPU token 100, GPU token 230); strict text comparison failed. The independent same-input F64 checks and output-head counterfactual are documented in #799. They do not establish a unique causal operator or a contract violation. I could not find a confirmed solution and am handing off the investigation. I asked the maintainer or another contributor in this reply to evaluate the disagreement on the newer source and find a fix if it is a defect, or determine its proper disposition. My earlier-source mismatch is not asserted to persist or be resolved on maintainer head35f9c80be22d816381c8c3ab287e0262b51ccbe4.No production numerical code, expected text, tolerance, tie handling or precision was changed to waive the failure. Exact-source results and original failure/success receipts are preserved in the contributor evidence archive; all owned workers ended, the observer was reversed and the existing GPU1 service remained healthy. No new issue or PR was opened.