5080deploy 9/9: merge upstream #389 (+#385), load-time FP8 dense projections

.github/ISSUE_TEMPLATE/config.yml

@@ -10,7 +10,7 @@ contact_links:
     url: https://discord.gg/MsA277cJzZ
     about: Ask on the Community Discord instead of opening an issue.
   - name: Usage questions (WeChat, CN)
-    url: https://github.com/FlashML-org/FreeToken/blob/main/assets/freetoken-wechatgroup.png
+    url: https://github.com/FlashML-org/FreeToken/issues/482
     about: Scan the QR code to join the Community WeChat group.
   - name: Development discussion
     url: https://join.slack.com/t/flashml/shared_invite/zt-3zpdh5j10-9dwTXrgLiqpVxizhA9KVbA

.github/issue-title-labeler.yml

@@ -0,0 +1,2 @@
+amd:
+  - '/\b(ROCm|HIP|RCCL|RDNA\s?\d?|CDNA\s?\d?|gfx\d{3,4}[a-z]?|MI\d{3}X?|RX\s?\d{4}|Radeon(?!\s*(\d{3}M|Graphics|Vega))|AMD[\s-]*(GPUs?|graphics|cards?|Instinct))\b|\bAMD[\s-]*显卡|\(amd\)|(?<!\w)A卡/i'

.github/workflows/issue-labels.yml

@@ -1,23 +1,35 @@
-name: Label issues
+name: Label issues and PRs
 
 on:
   issues:
     types: [opened, edited]
+  pull_request_target:
+    types: [opened, edited]
 
 permissions:
   issues: write
+  pull-requests: write
 
 jobs:
   label:
     runs-on: ubuntu-latest
     steps:
-      - uses: github/issue-labeler@v3.4
+      - if: github.event_name == 'issues'
+        uses: github/issue-labeler@v3.4
         with:
           configuration-path: .github/issue-labeler.yml
           enable-versioned-regex: 0
           include-title: 0
           repo-token: ${{ github.token }}
-      - if: github.event.action == 'opened'
+      - if: github.event.action == 'opened' || github.event.changes.title
+        uses: github/issue-labeler@v3.4
+        with:
+          configuration-path: .github/issue-title-labeler.yml
+          enable-versioned-regex: 0
+          include-title: 1
+          include-body: 0
+          repo-token: ${{ github.token }}
+      - if: github.event_name == 'issues' && github.event.action == 'opened'
         uses: actions/github-script@v7
         with:
           script: |

AGENTS.md

@@ -34,6 +34,8 @@ benchmarks/            end-to-end and micro benchmarks, see benchmarks/README.md
 docs/                  install, quickstart, CLI and model docs
 freetoken-kernel-cache/ companion wheel of prebuilt kernels, see its README
 scripts/               wheel build and release scripts
+  checkpoint_diagnose/ why an HF safetensors checkpoint does not load, and fixes for non-standard ones
+  gguf_check/          which GGUF files of a family load
 ```
 
 ## Development
@@ -54,6 +56,7 @@ Put a new test in the `tests/` directory that mirrors the module it protects, an
 
 - Search existing issues and PRs before starting. Items on the [Roadmap](https://github.com/FlashML-org/FreeToken/issues/79) are discussed with maintainers before implementation; features not on it start as an issue.
 - When helping the user draft an issue, follow the matching template in `.github/ISSUE_TEMPLATE/` (engine bug, model checkpoint, feature request) and fill in every required field: hardware, driver, FreeToken version, checkpoint ID, exact command, and the full log.
+- When a checkpoint fails to load, follow [scripts/checkpoint_diagnose/README.md](scripts/checkpoint_diagnose/README.md) before drafting a model checkpoint issue: it tells a FreeToken gap from a non-standard checkpoint, and repairs the latter.
 - One change per PR, linked to its issue, with the hardware, checkpoint ID and exact command it was tested with.
 
 ## Code comments

CONTRIBUTING.md

@@ -6,7 +6,7 @@ Thanks for helping make FreeToken better. This page covers how to report issues
 
 - [FAQ](https://github.com/FlashML-org/FreeToken/issues/84): kept up to date; most install and runtime problems are answered there.
 - [Roadmap](https://github.com/FlashML-org/FreeToken/issues/79): what we are working on next.
-- [Developer Slack](https://join.slack.com/t/flashml/shared_invite/zt-3zpdh5j10-9dwTXrgLiqpVxizhA9KVbA) for development discussion; [Community Discord](https://discord.gg/MsA277cJzZ) or [Community WeChat](https://github.com/FlashML-org/FreeToken/blob/main/assets/freetoken-wechatgroup.png) for usage questions.
+- [Developer Slack](https://join.slack.com/t/flashml/shared_invite/zt-3zpdh5j10-9dwTXrgLiqpVxizhA9KVbA) for development discussion; [Community Discord](https://discord.gg/MsA277cJzZ) or [Community WeChat](https://github.com/FlashML-org/FreeToken/issues/482) for usage questions.
 
 ## Reporting issues
 

README.md

@@ -7,7 +7,7 @@
 </div>
 
 <p align="center">
-| <a href="https://www.flashml.ai/"><b>Download</b></a> | <a href="https://arxiv.org/abs/2608.16157"><b>Paper</b></a> | <a href="https://join.slack.com/t/flashml/shared_invite/zt-3zpdh5j10-9dwTXrgLiqpVxizhA9KVbA"><b>Developer Slack</b></a> | <a href="https://discord.gg/MsA277cJzZ"><b>Community Discord</b></a> | <a href="https://github.com/FlashML-org/FreeToken/blob/main/assets/freetoken-wechatgroup.png"><b>Community WeChat</b></a> |
+| <a href="https://www.flashml.ai/"><b>Download</b></a> | <a href="https://arxiv.org/abs/2608.16157"><b>Paper</b></a> | <a href="https://join.slack.com/t/flashml/shared_invite/zt-3zpdh5j10-9dwTXrgLiqpVxizhA9KVbA"><b>Developer Slack</b></a> | <a href="https://discord.gg/MsA277cJzZ"><b>Community Discord</b></a> | <a href="https://github.com/FlashML-org/FreeToken/issues/482"><b>Community WeChat</b></a> |
 </p>
 
 
@@ -56,6 +56,7 @@ For More details:
 - [Supported models](https://github.com/FlashML-org/FreeToken/blob/main/docs/models.md)
 - [CLI reference](https://github.com/FlashML-org/FreeToken/blob/main/docs/cli.md)
 - [Repairing old FTW checkpoints](https://github.com/FlashML-org/FreeToken/blob/main/docs/ftw-hotfix.md)
+- [Diagnosing a checkpoint that fails to load](https://github.com/FlashML-org/FreeToken/blob/main/scripts/checkpoint_diagnose/README.md)
 
 ## Citation
 

benchmarks/README.md

@@ -27,6 +27,59 @@ batch size x miss rate.
 python benchmarks/bench_offload_cache_copy.py
 ```
 
+**`bench_scored_ensure.py`** — synthetic (no checkpoint): us per decode `ensure` call at a
+fixed miss count, flashlib `lru_ensure` vs the `--moe-cache-policy` kernels (lru, kd, kdfb,
+rule), from a CUDA graph of two decode steps on a full cache. `--stats` times it with
+`--moe-collect-stats` accumulation on.
+
+```bash
+python benchmarks/bench_scored_ensure.py --slots 1650 --k 10,20
+```
+
+**`bench_moe_copy_overlap.py`** — synthetic (no checkpoint): `FREETOKEN_MOE_COPY_OVERLAP`
+off vs on for Qwen4ExpMoE decode layers at the Qwen3.8-Flash-Next expert geometry, as CUDA
+graphs. Checks the outputs are bitwise identical, times replays ABAB, and profiles the copy's
+start after the ensure and how much of the side-stream shared expert runs inside it.
+
+```bash
+python benchmarks/bench_moe_copy_overlap.py --bs 1 --slots 160
+```
+
+**`bench_moe_prefetch.py`** — synthetic (no checkpoint): `FREETOKEN_MOE_PREFETCH=measure`
+cost. The router-lookahead predictor alone per layer (GEMV `[bs, 2560] x [2560, 512]`, top-K
+select, and the count kernel), then prefetch off vs measure on a real-geometry Qwen4ExpMoE decode
+stack (512 experts, rule eviction, copy overlap on) with forced routing that misses exactly m
+experts per layer, as CUDA graphs. `--on` times whole 48-layer decode steps with prefetch off vs
+on instead: a GEMV per layer stands in for attention (`--filler-us`), each layer prefetches 3 or
+4 candidates of which `--useful` are its misses, and `--delay-us` holds every prefetch copy back
+to price a late one. It reports ms/step, demand misses/step and the late fraction. `--steady
+--trace DIR` runs steady-state decode on a recorded routing trace instead (experts.npy,
+logit_idx.npy, logit_val.npy, req_index.npy): no cache restore, so wrong prefetches and evictions
+carry over, and a noisy copy of each layer's traced logits (`--sigma`) stands in for the
+lookahead. Arms such as `off,measure,on,on:4/4` (budgets before GDN / QSA layers) replay step by
+step in turn, each on its own copy of the cache state; it adds pollution (non-resident demands
+per layer against off's misses) and the late fraction per layer kind. Slots and sigma were
+calibrated to production's measure line (misses/layer ~4.3, precision ~0.56) at 24 layers.
+
+```bash
+python benchmarks/bench_moe_prefetch.py --bs 1,2 --misses 0,4
+python benchmarks/bench_moe_prefetch.py --on --bs 1,2 --misses 4 --delay-us 0,50,150,300
+python benchmarks/bench_moe_prefetch.py --steady --trace DIR --bs 1,2 --steady-slots 740 --sigma 0.55 \
+    --arms off,measure,on,on:empty,on:3/3,on:4/4,on:4/5
+```
+
+**`bench_expert_copy_corun.py`** — synthetic (no checkpoint): the fused expert copy kernel
+alone and beside compute, on registered host banks with the qwen4_exp NVFP4 row sizes. Sweeps
+`fast_index_copy_multi` (threads x blocks/bank) and `fast_index_copy_multi_slim`
+(blocks x threads x unroll) over 1..10-expert plans (GB/s, "hot" back-to-back and "cold"
+spacer-separated copies), then reports the stretch sigma of bf16/fp8 GEMVs, a small matmul and
+the NVFP4 routed decode GEMV while a copy loop runs on another stream. `--json` keeps the raw
+per-trial times.
+
+```bash
+python benchmarks/bench_expert_copy_corun.py --trials 5 --json copy_corun.json
+```
+
 For host RAM vs PCIe bandwidth and the offload/hybrid backend pick, use `ft bench bw`
 instead — it writes the JSON profile the engine reads.