WIP: PD-DFlash Task 9 — route-ahead priority band recovered; BM3 BLOCKED (not justified, do not merge)

[drunkcoding]

WIP / DRAFT — do not merge. Off origin/dev. Reviewable recovery of the Task 9 route-ahead priority-band candidate. BM3 did not produce a ship/no-ship verdict (blocked — see below), so per the plan's iron rule the band is NOT shipped. No false win either way.

What this recovers

PR #161 states the priority-band C++ was "implemented locally, built (sm_120), and validated … then reverted" (Python-only shipped) because its DFlash draft was absent, so BM3 could not run. The draft z-lab/gpt-oss-20b-DFlash is now cached.

The reverted candidate was never committed — verified via git log/pickaxe (-S kRouteAheadPriority), fsck --lost-found, reflog, and all stashes: no add commit, no revert commit, no dangling commit. The only hit was the plan doc's example code block. So the revert was a working-tree discard, and I re-implemented it from the plan spec (Task 9 Steps 4–5), building on the already-shipped EnqueuePrefetchTensors(tensor_ids, priority=1) (Task 8).

The candidate (7 files, +131/-8)

C++ — core/prefetch/task_scheduler.h, archer_prefetch_handle.{h,cpp}, core/python/py_archer_prefetch.cpp:

• Named bands kOnDemandPriority=0, kRouteAheadPriority=1, kBackgroundPrefetchPriority=2 (NUM_PRIORITY stays 20).
• Ordinary EnqueuePrefetch now enqueues at background (2) (was 1); the batched EnqueuePrefetchTensors + prefetch_tensors binding default to the dedicated route-ahead (1) band.

Python — moe_infinity/memory/expert_prefetcher.py:

• Adds ExpertPrefetcher.route_ahead_priority (the exact knob bench_prefetch_priority.py drives). Explicit route-ahead prefetch_experts_list issues at that band; legacy speculative_prefetch issues at background.

Tests — test_prefetch_native_gpu.py, test_speculative_prefetch.py:

• Native GPU smoke asserts all three bands + the knob issue without raising; CPU mock tests assert explicit=route-ahead / legacy=background mapping.

BM3 — BLOCKED (could not reach a verdict)

Two independent issues on dev, both unrelated to this candidate (a 1-line EnqueuePrefetch priority change + a Python knob; the native smoke swept all three bands with no hang):

• (A) Harness deadlock. bench_prefetch_priority.run_priority_ablation._reset_cache() calls the terminal archer_engine.clean_up_resources() between arms. After the first measured generation the offload engine is torn down (has_cleaned_up_resources_=true, StopExec), so the next forward hangs forever in fetch_tensors → _pre_forward_input_hook (moe_infinity/runtime/model_offload.py:1670, from init code commit). Reproduced with a faulthandler stack dump; GEN#1 completes in ~1 s, GEN#2 (post-reset) never returns and no bm3.json is written. It is a native wait (even os._exit couldn't reap it).
• (B) Exposed-fetch is uninstrumented. bm3_decision's condition (1) needs route_ahead.exposed_fetch < default.exposed_fetch, but no exposed_fetch_seconds/get_exposed_fetch_seconds/on_demand_fetch_seconds accessor exists on the engine/prefetcher and route_ahead_stats.as_dict() has no such key, so _exposed_fetch_seconds() returns 0.0 for every arm → exposed_fetch_improved = (0.0 < 0.0) = False. ship_priority_band is structurally unmeasurable on dev (can never be true), regardless of hardware or the now-cached draft.

Because BM3 cannot prove the route-ahead band lowers exposed fetch, and the plan forbids shipping any C++ hop without that proof, the band is not shipped. I did not fabricate a ship=true (no evidence) nor a measured ship=false (the metric is unmeasured).

Validation (evidence, not a ship claim)

• Build: SM120 (MOE_ENABLE_SM120=1) clean, 0 errors; _store.so rebuilt; binding confirmed prefetch_tensors(..., priority=1), ExpertPrefetcher.route_ahead_priority=1.
• Native priority-band smoke (offloaded gpt-oss-20b): tests/python/dflash/test_prefetch_native_gpu.py → 5 passed (3 existing + 2 new: all-three-bands + knob sweep).
• CPU decision/mock tests: test_prefetch_perf_reports.py + test_speculative_prefetch.py → 60 passed.
• gpt-oss offload no-regression: tests/test_gpt_oss_offload_topology.py + tests/python/unit/test_gpt_oss_mxfp4_dispatch.py → 9 passed (matches WIP: PD-DFlash Tasks 9–10 — BM3 priority band + BM4/BM5 overlap/e2e #161).
• ruff (check + format), clang-format, codespell, LSP: clean.

To actually run BM3 later (out of scope here)

Fix _reset_cache() to reset cache state without the terminal clean_up_resources() (Blocker A), and instrument exposed_fetch_seconds (Blocker B, e.g. an on-demand-fetch stall accumulator on the prefetcher or a native accessor). Then rerun python -m benchmarks.dflash.bench_prefetch_priority on offloaded gpt-oss-20b + z-lab/gpt-oss-20b-DFlash.

[drunkcoding]

Superseded by #169 (BM3 settled NO-SHIP).