Dump what the router thought of the experts that lost, not only the winners

[mfethe1]

WASTE_DUMP_ROUTE records the top-K ids and their weights. That answers every question about the ranking the router made, and none about a ranking it did not.

That gap turns out to be load-bearing. A residency prior promotes an expert the real router left just outside the top-K. A dynamic-k rule needs to know how far the k+1th sat behind the kth. A pruning criterion needs the scores of the experts it proposes to delete. All three are questions about the losers, and a trace of the chosen cannot say which one, or by how much.

So WASTE_DUMP_SCORES=path appends the whole vector, one line per (token, layer):

pos  L  v0 v1 .. v(E-1)

Evidence

Same container, same prompt, same flags; before is the merge-base built in a sibling worktree.

Before	After

3 generated tokens × 26 MoE layers → 78 lines of 258 fields (pos, layer, 256 experts).

The one detail worth checking in review

It dumps score[e] + bias[e] — the quantity the selection loop ranks on — and not w[j], which takes score[best] without the bias and is applied after the choice is made.

Those differ, and getting it backwards would make every offline re-ranking experiment quietly disagree with the engine while looking correct. It is the sort of thing that produces a confident wrong answer rather than an obvious one, so it seemed worth saying out loud in the comment as well as here.

Cost

Unset: one null check per layer, nothing written. make check is 44 passed / 0 failed / 13 skipped, unchanged from the merge-base on this machine (Windows 11 / Ryzen 7 3700X / MSYS2 UCRT64 / gcc 16.2.0).

Set: one fprintf per (token, layer), same shape as the WASTE_DUMP_ROUTE block it sits beside. 200 tokens × 26 layers × 256 experts came to about 7.8 MB.

Why this shape

The reasoning is the one already written above WASTE_DUMP_ROUTE — "Both questions are 'is the signal there', and both are cheaper to answer from a trace than from a build" — and so are the economics: one fprintf against a rebuild plus a sweep. This is the same instrument, one field wider.

I wanted it for a specific question (does preferring cache-resident experts buy anything the router lookahead has not already collected — arXiv:2412.00099), but the dump is not specific to that, which is why it is offered on its own rather than bundled with the experiment. The experiment may not survive; docs/LEARNED.md §54's "what routing buys is not which experts exist … it is the per-token exclusion" is a strong prior against anything that edits the selection, and I have not yet cleared the continuation bar the README holds num_experts_per_token to. The trace was worth having either way, and would be worth having to anyone testing a dynamic-k or a pruning criterion.

Context

From standing up a native Windows x86 measurement box for #37/#38 work. Companions on the same machine: #42, #43, #44.

🤖 Generated with Claude Code

https://claude.ai/code/session_01B5eVMiauR4Lkt8Dhc67MNb

[marcobambini]

Merged. Exercised here rather than taken on the description, since the value of
a trace is entirely in whether it records the right quantity:

WASTE_DUMP_SCORES=... test_forward MODEL <8 ids> out.bin 3
  286 lines, 258 fields each   (pos, layer, 256 experts)
  logits byte-identical to the same run with the variable unset

The detail you flagged for review is the one that matters, and it is right.
Dumping score[e] + bias[e] — what the selection loop ranks on — rather than
w[j], which takes score[best] without the bias and is applied after the
choice is made. Getting that backwards would make every offline re-ranking
experiment disagree with the engine while looking correct, which is the failure
mode that produces a confident wrong answer instead of an obvious one. Saying
so in the source comment as well as the PR is why I did not have to work it out
from the diff.

The argument for the shape is also the one this project already accepts: the
reasoning above WASTE_DUMP_ROUTE is "both questions are 'is the signal
there', and both are cheaper to answer from a trace than from a build". One
fprintf against a rebuild and a sweep is the same trade, one field wider.

Offering it separately from the experiment it was built for was the right
instinct — it went in on its own merits, and #46 is being judged on its own.