Recover historical MindSpeed provenance and allocator prerequisites

docs/reference-matrix.md

@@ -89,7 +89,7 @@ Reading rules:
 | **C3 / DLRover FlashCheckpoint** @ `e35b1ad90e2928ca6bacf5d22117f713566c5cb0` (Apache-2.0) | DDP/full engine → common-directory saver → POSIX storage; full/sharded CPU/Gloo replicas and restore consumer; [readset](../src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md#dlrover-extension-inspected-2026-10-22) | located-not-mined — HCCL device branch observed; NPU compatibility not validated | mined — memory readiness/step gates and replica byte/metadata ownership → `checkpoint-recovery-consistency`; restored values unverified | mined — queued save versus shard/done/tracker completion; serializer/synchronization boundaries, no durability proof | mined — actual full/sharded Gloo byte/metadata paths and world-position groups; gather/consumer correctness and throughput unverified | mined — worker versus helper/host lifetime; fallback daemon cannot persist after parent exits |
 | **C4 / NVIDIA Resiliency Extension** @ `49af0b03f14550d36a60dcfebd573848b5fd8435` (Apache-2.0 source headers; Megatron-Core lineage) | local `base_manager.py` complete-iteration selection/save/finalizer and `local_manager.py` dirty/final file lifecycle | mined (contract only) — CUDA D2H synchronized before async request returns; no NPU qualification | mined — snapshot/replica lifetime constraints → `checkpoint-recovery-consistency` | mined — clean/dirty publication, complete-rank generation selection and incomplete-save rejection; no power-loss proof | mined — all-rank availability/finalization and replica failure domains, not transport throughput | out-of-scope |
 | **CR1 / DeepSpeed ZeRO-Infinity** @ `5f266e26d216a705d9c64cac88fb813cefdbac4b` (Apache-2.0) | complete short `offload_config.py`, official ZeRO-3 live offload-field docs; no runtime/checkpoint audit | mined — parameter stage-3 and optimizer stage-1/2/3 boundaries → `training-resource-budget` | mined — CPU optimizer budget, pinned RAM, element-count residency and gated core share | mined — NVMe buffer pools and tiled read/compute/write overlap intent; buffer-size units unresolved in approved excerpt | out-of-scope — CPU/device transfer is not network coverage | mined — CPU resource-budget qualification; no privileged changes |
-| **CR5 / Ascend MindSpeed adaptive memory** — `core_r0.8.0`, page history `53193db8` (full pin and license unresolved) | original GitCode historical `docs/features/adaptive-memory.md` body directly read; implementation unavailable | mined (docs-only case) — spare-NPU-memory versus recompute/swap decision → `training-resource-budget` historical case; changed-layer comparison rejected as controlled gain | mined (docs-only) — CPU swap and allocator/affinity prerequisites | out-of-scope | out-of-scope — NPU/CPU copies not network evidence | mined (historical conditions) — incompatible allocator features and CPU-affinity prerequisites; no current compatibility claim |
+| **CR5 / Ascend MindSpeed adaptive memory** — historical `core_r0.8.0` document @ `53193db8bf5def407c4041a9beb3f4c9e9773daf` (root default BSD-3-Clause terms; identified file exceptions) | 2026-10-15 document inspection retained; [2026-10-30 pinned readset](../src/inferrsi/data/skill_library/methods/training-resource-budget/references/mindspeed-source-readset.md): feature guard, allocator/callback, saved-storage rescue, wrapper and affinity branch | mined (historical source candidate) — spare-memory/recompute/swap decision and effective selection → `training-resource-budget`; changed-layer comparison rejected as controlled gain | mined — eligible shared-storage CPU rescue/return and allocator prerequisites; no whole-model validation | out-of-scope | out-of-scope — NPU/CPU copies not network evidence | mined (historical conditions) — validation versus dispatch exclusions; 910A/910B binding, topology/tools and CPU-share assumptions; no current compatibility claim |
 | InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ `4a350344027960a48384626aa646f1765fda6d4f` (root Apache-2.0; model attribution in NOTICE) | Bounded manager/runner/scheduler/Ouro/host-metadata/Torch-consumer chain; [candidate](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md), [source map](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md), [construction/cases](reviews/issue91-vllm-rlt-finalization.md); historical [intake](reviews/issue91-vllm-rlt-intake.md) retained | source-guidance — synchronous LAST_EXITED copying versus SHARED, written-prefix validity, later deeper attention and normal release/wait/free; candidate, native runtime and efficacy not_run | out-of-scope — host bookkeeping does not establish host-RAM coverage | out-of-scope — no storage mechanism extracted | out-of-scope — local tensor copies; no transport mechanism extracted | out-of-scope — no OS mechanism extracted |
 | vllm-project/afd-plugin @ `df8d6a0c403c3ca1b4ed9cf1687e7204908bebec` (Apache-2.0 selected Python readset; retained MIT CAM subset notice) | Fixed-pin topology/factory/runner/metadata/DeepSeek dense proxy and P2pNccl reads; [full ledger](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md); [historical intake](reviews/issue91-afd-plugin-intake.md) retained | mined (bounded source-only candidate) — role/world/subgroup rank ledger and eager dense-FFN per-stage/per-peer payload ownership → [audit-afd-p2p-exchanges](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md); native/backend/model validation not_run | out-of-scope — no host-memory mechanism extracted | out-of-scope — no storage mechanism extracted | source call chain mined — actual NCCL-backed send/recv consumer identified; underlying completion, graph and multi-host/physical transport unqualified; no Network coverage credit | out-of-scope — no OS mechanism extracted |
 
@@ -271,7 +271,9 @@ Direct wave-7 source readsets and restrictions:
 [recovery C3–C4](../src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md),
 [cross-resource CR1/CR5/CR6](../src/inferrsi/data/skill_library/methods/training-resource-budget/references/sources.md).
 CR6's adapted note retains named attribution and CC BY-SA 4.0 share-alike;
-CR5 remains a historical documentation case with unresolved license/full pin.
+CR5 retains its 2026-10-15 documentation snapshot; the 2026-10-30 bounded source
+follow-up resolves the historical full pin/default license and selected
+allocator/wrapper/affinity paths. Current runtime compatibility remains unqualified.
 
 DCP source/version reconciliation, exact links and tutorial snapshot hash:
 [`dcp-async-save-performance/references/sources.md`](../src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/references/sources.md).

docs/searchspace-matrix.md

@@ -52,7 +52,7 @@ The primary compute and HBM resource. Most current work lives here.
 | **Diagnose / Verify** | Fixed-stage PyTorch/Ascend model report: bind current stage and actually loaded candidate to quality-validator output and comparable baseline | [ascend-performance-analysis](../src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md) | [Stage-evidence rule](../src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md): workflow instructions versus registered marker/keyword checks; completion is not numerical validation | canonical cann/cannbot @ c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110; selected CANN-2.0 scope | Candidate procedure only; host/model/backend not_run, agent efficacy not evaluated; no new measured coverage |
 | **Verify** | Same-window A/B protocol, noise floor, acceptance-state discipline; correctness-first overlap acceptance (fallback works, no KV/state reorder, actual overlap, no outside-module regression) | `ascend-optimization-casebook` (evidence-required section per case), `dense-performance-evidence`, `rsi-evidence`, `cann-attribution-overlap` | MindIE-SD perf-gate protocol: exploration vs acceptance state, three-proof requirement | MindIE-SD perf-gate @ 8481d39b; cann/oam-tools @ 8440935f | 🟢 |
 | **Observe / Configure / Optimize** | DALI CPU/GPU queue overlap and operator buffer sizing under explicit executor-mode gates | `training-input-pipeline` | Separated queues require async+pipelined and non-dynamic mode; output synchronization retained | Reference B1, DALI @ 6fe47c48d3b0 | 🟢 (Python API evidence, not C++/device validation) |
-| **Diagnose / Configure / Optimize** | Training capacity versus throughput, parameter/optimizer tier budgets, recomputation/offload choices; DCP exposed staging pause | `training-resource-budget`, `dcp-async-save-performance` | DeepSpeed stage gates; expert MFU/HFU denominator discipline; historical MindSpeed case only | Reference CR1, CR6, CR5, C1 async-only | 🟢 (MindSpeed implementation/current support unresolved) |
+| **Diagnose / Configure / Optimize** | Training capacity versus throughput, parameter/optimizer tier budgets, recomputation/offload choices; DCP exposed staging pause | `training-resource-budget`, `dcp-async-save-performance` | DeepSpeed stage gates; expert MFU/HFU denominator discipline; historical MindSpeed selection/allocator/rescue source boundaries | Reference CR1, CR6, CR5, C1 async-only | 🟢 (MindSpeed historical source candidate; current support unqualified) |
 | **Observe / Diagnose / Configure / Optimize / Verify** | vLLM V1 token/input/active-request budgets, allocation/preemption gates and async-KV admission; cap changes are conditional hypotheses | `diagnose-vllm-admission-pressure` | Queue length and KV usage do not establish allocation pressure; stats are gated and incomplete; verification proposed only | vLLM @ 7f730b4cea9a, Dot bundle; pinned scheduler/connector source ledger | 🟢 (candidate diagnosis; no runtime or gain validation) |
 | **Diagnose / Verify** | Context partial-ubatch progress, topology/input graph reuse and backend copy/event readiness | [audit-llama-runtime-state](../src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md); [owner map](../src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) | Retained completed ubatches differ from whole-batch rollback; frontend reuse differs from backend captured graphs | llama.cpp @ `707616feb5f379e99ff7442680a55b4a1704c7c9`; runtime/source notes and native test assertions; static inspection 2026-10-21 carried | 🟢 (source-derived candidate; native/backend execution not run) |
 | **Observe / Diagnose / Verify** | KV receive/save completion, delayed-free ownership, in-flight-write fences and failure reporting | `audit-vllm-kv-lifetimes` | Event ledger is proposed, not a telemetry exporter; abstract obligations require concrete connector/caller evidence | vLLM @ 7f730b4cea9a, Dot bundle | 🟢 (contract intelligence; concrete backend unverified) |
@@ -185,7 +185,7 @@ System-level: scheduler, DVFS, process isolation, environment variables, driver
 | **Configure** | Env-preflight: verify device availability, driver health, Python env before measurement | `performance-ceiling-analysis` (env-preflight references TRT-LLM exec-env-check) | TRT-LLM exec-env-check pattern | TRT-LLM @ f33e0e5f | 🟡 |
 | **Optimize** | CPU affinity check-then-pin (accelerator-local NUMA node, cross-round consistency), NUMA placement mode semantics (distribute/isolate/numactl) with page-cache-drop and numa_balancing rules, generation-vs-batch thread/affinity splits, thread-count env discipline, device-busy preflight with graceful-to-force PID escalation, structured ERROR/WARNING env checks, container flags and ulimits; cgroup resource reallocation as capacity spend (quota/weight/headroom reviews gated on observed stalls) | [os-process-placement](../src/inferrsi/data/skill_library/hardware/host/os-process-placement/SKILL.md), `cgroup-resource-controls`; [llama.cpp owner map](../src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) | OMP_NUM_THREADS/PROC_BIND per scenario; `CPU_AFFINITY_CONF`; npu-smi usages preflight; `ASCEND_HOME_PATH` named-var correctness; rollback manifest before any change | TRT-LLM perf-host-optimization @ f33e0e5f; awesome-ascend-skills @ 75ec89c; ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT; static inspection 2026-10-22 carried); kernel cgroup-v2 docs (pack-mediated); historical vllm-ascend Dockerfile/thread-env shipping defaults @ bc8115ed (Reference Matrix); ACLGraph @ 8a8e1e05ecb6 adds no OS-control evidence | 🟢 (isolcpus and scheduler priorities nice/ionice/SCHED_FIFO/RR remain unsourced) |
 | **Verify** | Card health check sequence: npu-smi Health, 4-step smoke verify, proc-mem occupancy; cgroup change verification (matched windows, application metrics, counter deltas, provenance, rollback) | `ascend-device-health`, `cgroup-resource-controls` | Health=OK ≠ card usable; SIGKILL residual state only detectable via smoke run | MindIE-SD @ 8481d39b; kernel cgroup-v2 docs (pack-mediated) | 🟢 |
-| **Diagnose / Configure** | CPU optimizer share, NUMA locality and per-rank garbage-collection stalls; training/helper lifetime in recovery | `training-resource-budget`, `checkpoint-recovery-consistency` | Gate CPUAdam cores to stage-3 super_offload; fallback saver lifetime matters | Reference CR1/CR6, C3; historical CR5 prerequisites only | 🟢 (no scheduling or privileged changes) |
+| **Diagnose / Configure** | CPU optimizer share, NUMA locality and per-rank garbage-collection stalls; training/helper lifetime in recovery | `training-resource-budget`, `checkpoint-recovery-consistency` | Gate CPUAdam cores to stage-3 super_offload; fallback saver lifetime matters; historical MindSpeed 910A/910B binding requires actual device/NUMA/CPU mapping beyond tool presence | Reference CR1/CR6, C3; CR5 pinned affinity source | 🟢 (source guidance; no scheduling or privileged changes) |
 | **Configure / Optimize** | FFCV OS-managed versus process-managed dataset caching | `training-input-pipeline` | Cache ownership is independent of nice/ionice/realtime scheduling | Reference B3 | 🟢 (configuration decision, not runtime result) |
 
 **Second-level targets:** NPU driver, CANN runtime, container/process isolation
@@ -202,7 +202,7 @@ System-level: scheduler, DVFS, process isolation, environment variables, driver
 | Save-path performance qualification (checkpoint) | SSD / Optimize + Verify | Low | DCP asynchronous staging, pinned-buffer reuse and completion gates are now sourced in `dcp-async-save-performance`; no local gains, NPU compatibility or backend durability verified. fastsafetensors-specific performance remains unsourced; no repeat mining authorized |
 | Wave-7 input pipeline qualification | Accelerator / RAM / SSD / Network | Low | Mode-specific DALI behavior, FFCV current toolchain/distributed limits, Mosaic cache-churn and sample correctness remain unmeasured; no hardware run approved |
 | Distributed recovery qualification | RAM / SSD / Network / Verify | Low | C3/C4 contracts extracted; replica transport, whole-state completeness, world-size changes and actual failure recovery remain unvalidated |
-| Cross-resource applicability and historical evidence | Accelerator / RAM / OS | Low | DeepSpeed buffer_size units/dispatch outside inspected config; MindSpeed full pin/license/implementation/current compatibility unresolved; CR6 is expert methodology, not measured gains |
+| Cross-resource applicability and historical evidence | Accelerator / RAM / OS | Low | DeepSpeed buffer_size units/dispatch outside inspected config; MindSpeed historical pin/default license and selected allocator/wrapper/affinity paths resolved, while full search/cache behavior, current compatibility and measured benefit remain open; CR6 is expert methodology, not measured gains |
 | OS/System isolcpus and scheduler priorities (nice / ionice / SCHED_FIFO / RR) | OS / Optimize | Low | cgroup v2 controls are now sourced (`cgroup-resource-controls`); the scheduler-priority class remains without a source and deferred |
 | vLLM-RLT recurrent-depth KV finalization | Accelerator / KV correctness / Verify | Source-guidance candidate; native/efficacy not_run | [One operating candidate](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) traces synchronous host-KVMetadata exit/copy/validity → later depth-specific attention → normal release/wait/free. LAST_EXITED planes, SHARED single plane, bookkeeping and CUDA completion remain distinct; DeviceKVMetadata/async/speculation/capacity/transport excluded. [Intake](reviews/issue91-vllm-rlt-intake.md) retained |
 | AFD P2pNccl completion, graph and transport qualification | Accelerator / Communication / Verify | Candidate audit delivered; native qualification gap remains | [audit-afd-p2p-exchanges](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) covers rank and peer/stage ownership at its pin. The `afd_plugin` runtime descriptor remains unverified; underlying completion, DBO/graph and physical/multi-host transport are unqualified. Historical [intake](reviews/issue91-afd-plugin-intake.md) retained; no new Network coverage |