Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,14 +142,16 @@ worker/connector conformance, cross-pin compatibility or a physical transport.

### Reviewed systems-paper mechanisms

`mined` below means pack-mediated extraction into the linked candidate artifacts.
`mined` below means scoped extraction into the linked candidate artifacts;
the evidence column distinguishes retained packs from direct public-paper reads.
Readsets preserve exact sections, equations, tables, source hashes where supplied,
metric denominators and unreported limitations. Versions are paper identities,
not fabricated implementation SHAs or a version-matched runtime stack.

| Original source / version | Inspected scope and local evidence | Mechanism / consumer | Evidence limits |
|---|---|---|---|
| Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2) | [A readset](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md); §§3–7, A.1–A.2 | mined — operator/queue model calibration and joint-SLO shortlist → [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), ceiling analysis | Empirical prediction, not physical bound; near-capacity CPU overhead and 12.65% maximum error in the specified appendix case; no current runtime calibration |
| Mooncake — FAST '25 proceedings, pp. 155-170 | Direct public-paper read, retained qualification 2026-10-31; [source condition and readset](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md), §2.2, Table 1, Eqs. 1-2 | mined — conditional prefix-KV transfer versus saved-prefill-work comparison → `performance-ceiling-analysis` | Analytical gate only; current inputs and E2E benefit unverified. SSD mechanics excluded. Original transfer-gates extraction history remains positive; missing synthesis novelty UNKNOWN. Raw-paper redistribution permission UNKNOWN. |
| Orca — OSDI 2022 proceedings | [Exact readset](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md); §§3–6, Algorithm 1 | mined — iteration eligibility, selective batching, FCFS/reservation → [assess-serving-preemption](../src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md) | Historical engine/harness and normalized-median denominators; not present vLLM behavior or fairness proof |
| Sarathi-Serve — [OSDI 2024 proceedings, pp. 117–134](https://www.usenix.org/conference/osdi24/presentation/agrawal) | Fresh official-edition grounding, 2026-10-30; full text and artifact appendix; §§4.1–4.3, §5 setup, Figures 9/14 and Table 4 checked for the [case record](../src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json) | mined — decode/partial-prefill/new-request token accounting and chunk-cost tradeoffs → [one optimization-search-policy case](../src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md) | Source-derived candidate; empirical not_run. Combined design has worse P50 TTFT than hybrid-only in both Table 4 traces; no current-engine, universal-budget or fairness claim. Raw source reuse rights UNKNOWN; no archive recovery or novelty over unavailable supplier synthesis claimed. |
| FastServe — [2305.05920v3](https://arxiv.org/abs/2305.05920v3) | [Exact readset](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md) | mined — service priority, promotion and suspended-state host residency → serving-preemption candidate | Aggregate tail improvement is not tenant fairness; same-engine contrast distinct from historical-vLLM comparison |
Expand All @@ -176,6 +178,8 @@ DistServe, PagedAttention, Mooncake) is referenced by the packs but absent from
the supplied local directory. Its six-paper extraction and standalone candidates
are not imported or counted here. Existing DeepSpeed configuration evidence and
Mooncake transport discovery are not substitutes for those paper readsets.
The separate public-source Mooncake condition above does not recover that archive
or import its earlier standalone candidate.

The separate Sarathi-Serve case above uses a freshly pinned official edition;
it does not recover or import that archive or its supplier-authored synthesis.
Expand Down
3 changes: 2 additions & 1 deletion docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ The primary compute and HBM resource. Most current work lives here.
| **Diagnose / Configure / Optimize / Verify** | Runtime ACLGraph dispatch, stable input addresses, capture-resource limits and branch-specific replay/update ordering | `ascend-aclgraph-capture-replay` | Caller owns persistent buffers; DEBUG checks positional tensors only; reduced capture sets trade coverage for resources; [version/responsibility routing](../src/inferrsi/data/skill_library/README.md#vllm-ascend-version-and-responsibility-routing) | vllm-ascend @ 8a8e1e05ecb6, Dot bundle; wrapper/guide/tests inspected by supplier | 🟢 (conditional proposals; tests not run) |
| **Diagnose / Configure / Optimize / Verify** | Device-HBM stream-use lifetime, fragmentation/counter interpretation and gated allocator controls | `torchnpu-device-memory-lifecycle` | Lifetime recording does not insert data dependencies; expandable segments change counters/control legality; GC synchronization/comment conflict retained | TorchNPU @ 67511c53cd4c, Dot bundle; device allocator ledger | 🟢 (source-supported procedure; NPU execution not run) |
| **Diagnose / Verify** | Serving model: operator/context dependencies, calibration/load domain and joint-SLO prediction uncertainty | [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), `performance-ceiling-analysis` | Vidur near-capacity CPU-overhead counterexample; simulator output is not a bound | Vidur v2; Reference Matrix systems-paper row | 🟢 (prediction guidance; calibration not run) |
| **Diagnose / Verify** | Reusable-prefix compute work versus host/network KV loading cost | [performance-ceiling-analysis](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md), [Mooncake condition](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md) | Check bytes, FLOPs and effective rate at one GPU-group scope; favorable Equation 2 still requires exposed-cost/SLO measurements | Mooncake FAST '25 §2.2, Table 1, Eqs. 1-2; separate public-paper source, not recovered foundation archive | 🟢 (analytical source condition; current feasibility and E2E benefit unverified; no SSD credit) |
| **Diagnose / Configure / Optimize / Verify** | Serving service order versus HBM admission and suspended-KV recovery; replica migration versus elastic sequence groups | [assess-serving-preemption](../src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md), [evaluate-elastic-serving-placement](../src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | [Hypothetical pressure/migration cases](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md); cohort/fairness and destination-reservation costs | Orca proceedings, FastServe v3, Llumnix v1, LoongServe v2; implementation pins remain separate | 🟢 (conditional paper-derived plans) |
| **Diagnose / Optimize / Verify** | Training activation lifetime, rematerialization and reserved-address packing | [plan-activation-residency](../src/inferrsi/data/skill_library/methods/plan-activation-residency/SKILL.md) | Dependency repair and capacity feasibility checked separately; no recovery-checkpoint coverage | Checkmate v3, OLLA v2; [C readset](../src/inferrsi/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md) | 🟢 (planning, not solver/runtime validation) |
| **Diagnose / Configure / Optimize / Verify** | Training stage/mesh/sharding feasibility and split F/B/W execution under total-HBM and optimizer-semantics constraints | [choose-parallel-training-plan](../src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md), [assess-split-backward-schedule](../src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/SKILL.md) | Alpa conditional plans; Zero Bubble activation caps versus total memory; rollback versus stale updates | Alpa v3, Zero Bubble v1 | 🟢 (author evidence, no current stack validation) |
Expand Down Expand Up @@ -197,7 +198,7 @@ System-level: scheduler, DVFS, process isolation, environment variables, driver
| Gap | Row | Priority | Next action |
|-----|-----|----------|-------------|
| Further native hardware-source coverage | Accelerator / Communication | Later source expansion | Native NCCL remains unmined. Existing CUTLASS/DeepGEMM/FlashInfer slices do not cover broad dispatch. Investigate UCX when a multi-node workload requires it, and ROCm when an AMD target exists; none is newly covered by this delivery. |
| Earlier six-paper foundation archive unavailable | Uncredited original-archive mechanisms | Input blocked for original-archive import | Supply the approved archive/readsets before importing its ZeRO-Infinity, FlexGen, Sarathi-Serve, DistServe, PagedAttention or Mooncake candidates; do not substitute config/discovery evidence. Separately, the fresh official-edition [DistServe phase-placement reference](../src/inferrsi/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/README.md) is retained under the existing search-policy owner as candidate / not_run source knowledge; it does not recover that archive or qualify runtime, device or service behavior |
| Earlier six-paper foundation archive unavailable | Uncredited original-archive mechanisms; original synthesis and standalone paper candidates | Input blocked for original-archive import | Supply the approved archive/readsets before importing its ZeRO-Infinity, FlexGen, Sarathi-Serve, DistServe, PagedAttention or Mooncake candidates; do not substitute config/discovery evidence. Separately, the fresh official-edition [DistServe phase-placement reference](../src/inferrsi/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/README.md) is retained under the existing search-policy owner as candidate / not_run source knowledge; it does not recover that archive or qualify runtime, device or service behavior. The separately sourced Mooncake analytical condition above does not recover the archive; novelty against the missing synthesis remains UNKNOWN. |
| Reviewed A–F qualification | Scoped Accelerator / Host / Network rows above | Separate future task | Candidates and author evidence do not establish runtime compatibility, measured gains or NPU support; no reproduction authorized |
| Save-path performance qualification (checkpoint) | SSD / Optimize + Verify | Low | DCP asynchronous staging, pinned-buffer reuse and completion gates are now sourced in `dcp-async-save-performance`; no local gains, NPU compatibility or backend durability verified. fastsafetensors-specific performance remains unsourced; no repeat mining authorized |
| Wave-7 input pipeline qualification | Accelerator / RAM / SSD / Network | Low | Mode-specific DALI behavior, FFCV current toolchain/distributed limits, Mosaic cache-churn and sample correctness remain unmeasured; no hardware run approved |
Expand Down
4 changes: 2 additions & 2 deletions docs/skill-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
matrices. **Do not edit by hand**; regenerate after any registry, skill,
matrix or frozen-plan change.

- Input digest: `sha256:43760f8a7e388c4c55bf61fd9680da4ecabe56264eafe5ebe1beacc3ae0b85d8`
- Input digest: `sha256:827a48e08ad75f86c7f87c7b8ef2ec21f79fb8b897dc1f135f1d77723436e947`
- Regenerate: `python tools/build_skill_coverage.py`
- Verify without writing: `python tools/build_skill_coverage.py --check`

Expand Down Expand Up @@ -130,7 +130,7 @@ optimization is fully covered or that any procedure has measured benefit.
| [inference-optimization-casebook](../src/inferrsi/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | **unknown** | — | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-applicability, no-eval |
| [kernel-writing-guide](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | **unknown** | — | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)<br>1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)<br>1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)<br>1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-applicability, no-eval |
| [optimization-search-policy](../src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | **unknown** | — | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-applicability, no-eval |
| [performance-ceiling-analysis](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md) | common | Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical ... | inference serving systems (LLM, diffusion, audio) | — | 1 Accelerator · Diagnose / Verify<br>5 OS · Configure | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (ebugger/Empirical-Roofline-Toolkit @ 0771fa9)<br>1 · Systems sources (hahnyuan/LLM-Viewer @ 1893e4b) | candidate · not_run · 16 readset(s) | no-eval |
| [performance-ceiling-analysis](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md) | common | Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical ... | inference serving systems (LLM, diffusion, audio) | — | 1 Accelerator · Diagnose / Verify<br>5 OS · Configure | 1 · Systems sources (Mooncake — FAST '25 proceedings, pp. 155-170)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (ebugger/Empirical-Roofline-Toolkit @ 0771fa9)<br>1 · Systems sources (hahnyuan/LLM-Viewer @ 1893e4b) | candidate · not_run · 17 readset(s) | no-eval |
| [plan-activation-residency](../src/inferrsi/data/skill_library/methods/plan-activation-residency/SKILL.md) | common | Propose training activation and gradient memory reductions by distinguishing lifetime reordering, address packing and ... | **unknown** | — | 1 Accelerator · Diagnose / Optimize / Verify | 1 · Systems sources (Checkmate — [1910.02653v3](https://arxiv.org/abs/1910.02653v3)) | candidate · not_run · 1 readset(s) | no-applicability, no-eval |
| [repository-evolution-analysis](../src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md) | common | Analyze a repository's commit history to build a capability-evolution model: what the system can do, how it got there ... | any software repository with a recoverable git history | — | — | 2 · Methodology and corpus references (LearningToCommit/LearningToCommit @ 7e5a1d7)<br>2 · Methodology and corpus references (yetone/magpie @ 77800032) | candidate · not_run · 0 readset(s) | no-SearchSpace-layer, no-eval |
| [skill-evolution-playbook](../src/inferrsi/data/skill_library/methods/skill-evolution-playbook/SKILL.md) | common | Practical design choices for evolving agent skills from execution traces: when one trajectory should update a skill ... | **unknown** | — | — | 2 · Methodology and corpus references (Qwen-Applications/Trace2Skill @ 3d0b52a)<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | no-SearchSpace-layer, no-applicability, no-eval |
Expand Down
4 changes: 2 additions & 2 deletions src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ These are knowledge abstracts (`--kind knowledge`), with namespaced package
IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA
procedures use their corresponding backend; shared methods use `common`.

Source snapshot: [95b1e4bac05edd101392a70920fc2295074f0848](https://github.com/InferMatrixLab/InferRSI/tree/95b1e4bac05edd101392a70920fc2295074f0848/src/inferrsi/data/skill_library).
Source snapshot: [ad322aab86d9a148bc4b378a15b2762e41b86908](https://github.com/InferMatrixLab/InferRSI/tree/ad322aab86d9a148bc4b378a15b2762e41b86908/src/inferrsi/data/skill_library).

```bash
inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50
Expand All @@ -29,7 +29,7 @@ contain the current input paths and bytes (Git checkout filters are respected).

```bash
python tools/build_skill_index.py --source-revision <committed-source-revision>
python tools/build_skill_index.py --source-revision 95b1e4bac05edd101392a70920fc2295074f0848 --check
python tools/build_skill_index.py --source-revision ad322aab86d9a148bc4b378a15b2762e41b86908 --check
```

`--check` detects missing, stale and unexpected generated records without writing.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/95b1e4bac05edd101392a70920fc2295074f0848/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "95b1e4bac05edd101392a70920fc2295074f0848",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/ad322aab86d9a148bc4b378a15b2762e41b86908/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "ad322aab86d9a148bc4b378a15b2762e41b86908",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/95b1e4bac05edd101392a70920fc2295074f0848/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"commit": "95b1e4bac05edd101392a70920fc2295074f0848",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/ad322aab86d9a148bc4b378a15b2762e41b86908/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"commit": "ad322aab86d9a148bc4b378a15b2762e41b86908",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/metadata.json",
Expand Down
Loading
Loading