[gdevenyi]

Stacked on #81 (the rebuild). It adds:

item	why
merge FlashML-org#614 (head a8498c7): leak no semaphores when a signal stops the server	the production serve log had 186 leaked semaphore warnings over 181 guarded starts, and /dev/shm held 19 stale sem.mp-*. In this round's A/B, the two production-arm stops leaked and none of the 8 rebuilt-arm stops did
merge FlashML-org#593 (head 1d5ce54): refuse converting into a directory that already holds an FTW	this round converts a second checkpoint; a re-run into an existing FTW dir silently reopened shard 00000 under a stale index
feat(qwen4_exp): read llm-compressor (compressed-tensors) NVFP4 routed experts	the expert spec now takes its tensor names and global-scale direction from the checkpoint's quant config, as qwen3_5_moe does. RedHatAI/Qwen3.8-Flash-Next-NVFP4 then maps all 221184 expert tensors (73728 before); the nvidia checkpoint keeps the same spec (modelopt names, no reciprocal)
test(moe): the prefetch counter test's copy spies pass FlashML-org#610's blocks_per_bank through	4 GPU test failures after the rebuild

The new CPU test, test_compressed_tensors_experts_load_like_their_modelopt_twin, writes a toy llm-compressor checkpoint and its modelopt twin (weight_scale_2 = 1 / weight_global_scale, plus unused input scales) and asserts that the pieces are bit-identical. It fails before the change (found 18 expert tensors, expected 54).

Evaluated and not merged:

• perf(model): fuse hc combine with the next block's RMSNorm FlashML-org/FreeToken#590 (fuse hc combine with the next RMSNorm): bit-identical, but no measurable decode gain (+0.1%, 2 rounds).
• perf(linear): run a single-row bf16 projection as one GEMV FlashML-org/FreeToken#585 (single-row bf16 projections as one GEMV), 4 interleaved rounds:
  • decode gain is small: 32K +0.8%, 64K +0.8%, 128K +2.1%, 255K +0.3%;
  • it changes greedy output;
  • one of four cold runs gave a different think fingerprint than the other three, which the base never did.
• The rest of the round's PRs are skipped or deferred; the reasons are in the session report.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Bu6LgoxLR4wETqb7RPR2vt

[gdevenyi]

RedHatAI/Qwen3.8-Flash-Next-NVFP4 vs the nvidia checkpoint, served the FreeToken way. Production stays on nvidia.

Setup. RedHat was converted with this PR's loader:

FREETOKEN_FP8_DENSE=1 FREETOKEN_FP8_LMHEAD=1 python -m freetoken.checkpoint --model <RedHat dir> --out <RH FTW>

• Conversion took 500 s. The FTW is 70.45 GiB, with the same tensor counts and total bytes as the nvidia FTW.
• Its BF16 PLE side files were replaced with links to the nvidia FTW's fp8 table, so the disk backend reads the same rows.
• Everything outside the routed experts is byte-identical BF16 in both checkpoints and goes through the same FP8 requant, so the experts are the only variable.

Paired next-token NLL. The offline engine ran with production numerics (FP8 dense and lm_head, nvfp4 KV, disk PLE, host embedding, scored eviction). The corpus is frozen: 62 recent upstream source and doc files, 207,166 scored positions, plus a 131K-token sequence scored on its last 4K positions.

	nvidia	RedHat
mean NLL (nats/token)	0.99122	0.99812
top-1 accuracy	0.7796	0.7782
long-context tail dNLL		-0.0004

• dNLL (RedHat - nvidia) = +0.0069 nats/token, 95% block-bootstrap CI [-0.0023, +0.0172] (404 blocks of 512).
• Top-1 disagreement is 8.5%, and top-20 KL(nv || rh) is 0.079 nats: the two models differ, but RedHat is not better.
• Harness noise: two nvidia runs are bit-identical (dNLL 0, disagreement 0).

Why the model card's gap (99.1% vs 98.0% recovery) does not show up here:

• The card's numbers come from vLLM W4A4 with RedHat's calibrated input_global_scale. FreeToken serves these experts W4A16: b12x, the W4A4 backend, needs I >= 1024 and this model has I = 640.
• In weight space, RedHat's dequantized experts are further from the BF16 base than nvidia's on all 72 sampled projections: relative L2 9.48% vs 8.13%.

Speed and memory would be identical (the same bank bytes and layout), so there is no reason to switch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Bu6LgoxLR4wETqb7RPR2vt