Repository navigation
docs: complete applicability metadata for 17 methods skills - #263
Conversation
Batch 2 of the InferMatrixLab#91 Round 2 plan applies the sidecar contract settled in batch 1 to the `methods/` family. Fifteen of the seventeen registered `methods/` skills gain an `applicability` sidecar and an owner-supplied `recommendations` list. Every value is traceable to a named section of that skill's own `SKILL.md`. No value was carried across skills, and no hardware dimension was inferred from a source's hardware: none of these fifteen bodies establishes a device dependency, so none declares `hardware`. The per-key meanings follow the contract agreed on the issue: scope what the procedure applies to prerequisites what must hold first: access, tooling, authorization input_contract what the caller supplies produced_artifact what it emits, at the body's evidence level failure_modes how the procedure yields a wrong or unusable result refusal_point where the body says it withholds a conclusion `failure_modes` states how the procedure goes wrong in use and does not restate `limitations`, which records what the record does not establish. `tools/build_skill_index.py` emits four generator-owned `limitations` sentences and then appends the sidecar's entries, so the two fields have different owners and different jobs. Key usage across the fifteen: `scope`, `input_contract`, `produced_artifact`, `failure_modes` and `refusal_point` on all fifteen; `prerequisites` on fourteen. `audit-fixed-workload-claim` states no precondition beyond its input contract. Each sidecar is a pure insertion: the diff removes no line from any `metadata.json` and adds only the two top-level fields. `docs/skill-coverage.md` regenerates from the new sources. The `no-applicability` gap falls from 42 rows to 27. Deferred to a follow-up and not included here: `kernel-writing-guide` and `weight-loading-optimization`, the two `methods/` skills whose bodies do establish a device dependency, and the `docs/knowledge-base.md` field-ownership subsection that records these key meanings in the documentation tree. No evaluation, verdict or promotion. `status` stays `candidate` and `empirical_validation` is unchanged for all 77 skills. No matrix row was added or edited; layer decisions are batch 5. Agent-assisted-by: Claude / Opus 5.5 [effort=xhigh] (methods sidecar content) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Skill evaluationAutomatic execution / cumulative PR budget: Advisory: obtain missing evidenceRetained evidence availability: planned / unexecuted. No approval or merge authorization is issued.
PLANNED: execution is not authorized by knowing this hash. Selected comparisons: 1 · Estimated task calls: 2. Applicable only while these revisions and inputs match the current PR. Plan bindings, budget, frozen exam and advisory detailsUnchanged-correct is valid neutral evidence, not incremental benefit. Per-case outcomes (discovery excluded; regressions and gaps first): []Showing at most 20 of 0 pairs; the evidence link contains all outcomes, including omitted regressions/gaps. Plan SHA-256: Model/settings:
Frozen exam hashes: Provider/settings: Uncovered / unsupported paths (97)
|
Dong1017
left a comment
Dong1017
left a comment
There was a problem hiding this comment.
Reviewed the 15 changed method sidecars at head 1a52db0664d343628adcf5d8077aaa4ab65d91dc, against API base / observed main 6e6ee245d9e92ee9365976f2e4e262304999f147, using the #91 field contract.
I read each corresponding unchanged SKILL.md and checked scope, caller inputs, proposed outputs, failure/refusal conditions and advice against it. No additional blocking content finding in that projection. The 15 metadata diffs insert only applicability/recommendations, and the generated diffs do not promote status, empirical_validation or verification. This verifies representation of the existing guidance; it does not independently re-audit every inherited upstream paper claim or validate skill efficacy.
P3 — Correct the PR description's coverage counts. It says no-applicability falls 42→27. The base table records 41, and the current-head table records 26. The reduction is still 15; align the description with the committed evidence. This is a nonblocking reporting correction.
Integration/evidence hold: the API currently reports mergeable=false/dirty. No Ready full-check or installed-package check is present for this head; the 101-pass result in the description remains author-reported here. The latest policy status covers declaration compliance only. Skill Eval planned one comparison but did not execute it; 119 paths remain uncovered. The automated code review also failed to complete. These are not passing task/efficacy evidence.
Next for the existing author: integrate current main while preserving #262's hardware owners, regenerate projections from the combined committed source snapshot and obtain current integration checks. Follow the already-recorded #257 coordination boundary if that source slice lands first. Keep the two deferred sidecars, field-ownership documentation and owner disposition of uncovered evaluation scope explicit; this partial batch does not complete #91.
Source/existing-hosted-evidence inspection only; no local tests, benchmark, model/device run, workflow rerun or paid execution. COMMENTED, not an approval or merge-ready LGTM.
Dong1017
left a comment
Dong1017
left a comment
There was a problem hiding this comment.
Additional blocking provenance finding at head 1a52db0664d343628adcf5d8077aaa4ab65d91dc, API base / main 6e6ee245d9e92ee9365976f2e4e262304999f147, detailed inline.
The earlier review checked the 15 sidecar projections against their SKILL bodies. Subsequent independent source-pin verification found the catalog's referenced snapshot unavailable from the target repository and absent from the PR commit history. That gap must be fixed before merge readiness; the existing conflict and CI/evaluation limits also remain.
Verification used GitHub commit/content lookups, the complete one-commit PR history and generator source; no clean-checkout command, local test, model/device or paid execution was run. COMMENTED is used because the authenticated account is also the PR author; this is not a formal self-REQUEST_CHANGES or approval. Next owner: existing PR author.
| procedures use their corresponding backend; shared methods use `common`. | ||
|
|
||
| Source snapshot: [e736be55bf2927f5cbd0408682baa65f4f6fb7e8](https://github.com/InferMatrixLab/InferRSI/tree/e736be55bf2927f5cbd0408682baa65f4f6fb7e8/src/inferrsi/data/skill_library). | ||
| Source snapshot: [058fc76b656ea5a29bb4646a5e556e32d35bdbc9](https://github.com/InferMatrixLab/InferRSI/tree/058fc76b656ea5a29bb4646a5e556e32d35bdbc9/src/inferrsi/data/skill_library). |
There was a problem hiding this comment.
P2 — Publish the source snapshot referenced by the generated catalog. All 77 generated skill records and this README point to 058fc76b656ea5a29bb4646a5e556e32d35bdbc9, but the target repository's commit lookup returns “No commit found for SHA” (422), and a recorded source URL returns 404. The complete PR commit list contains only 1a52db0664d343628adcf5d8077aaa4ab65d91dc, parent dabd18492b5afa2b57b912e774e75f418645a1de; the claimed source snapshot is not in the submitted history. Public provenance is therefore broken, and the documented --source-revision ... --check cannot be reproduced from a normal clean checkout containing only that history: build_index() first requires git rev-parse --verify <revision>^{commit}.
Make the actual canonical-source snapshot reachable in the submitted history/target repository, then regenerate the catalog with that revision and verify both generator checks from a clean checkout. Preserve #262's sources when resolving the separate integration conflict. This is a current-head provenance/reproducibility blocker, not an efficacy or hardware result.
Dong1017
added
the
area:knowledge
ChatGPT Codex Connector
Add applicability and recommendations for kernel-writing-guide and weight-loading-optimization, completing the selected seventeen methods. Keep hardware dependencies separate from tuning advice and express kernel prerequisites as conditions. Document the field meanings in knowledge-base. Regenerate the skill index and coverage from the committed sources. Preserve verification, status, empirical validation and source provenance; descriptive metadata does not establish execution or efficacy. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (weight-loading metadata and field documentation; inherited attribution) Agent-assisted-by: Claude Code / gpt-6-astra [effort=xhigh] (kernel-writing metadata; inherited attribution) Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (metadata refinement, generated projections and finalization) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Retain the hardware source owners and historical MindSpeed readset from main. Align training resource prerequisites with the updated canonical body. Keep this source snapshot in history so the following projection commit can reference a revision available to ordinary clones. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (source integration and metadata alignment) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Rebuild the catalog and coverage from the committed combined sources. Retain the source commit as an ancestor so catalog links and generator checks remain reproducible after cloning the published branch. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (generated projections and validation) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Dong1017
changed the title
Dong1017
left a comment
Dong1017
left a comment
There was a problem hiding this comment.
Incremental review of a559b12a1dcf1d549e801dd645a949b0f730a75f (tree 07944d4f1a1f7acd8242df8bfb1ba73d33be2581). One current P2 remains: the new kernel-writing sidecar excludes a use case explicitly allowed by its body, detailed inline. Please correct that source projection before merging.
Previous findings:
- Source-snapshot P2: addressed.
ebb4349ac4b24b520502278359b38fa919d59e84is public and the head's direct parent; its indexed source tree exactly matches the current inputs. The generated records reproduce from those bytes. - Count P3: addressed. The committed inventory and description now agree: 41 → 24 for 17 methods; the earlier 15-method stage was 41 → 26.
- The integration preserves #262's hardware sources/matrices and #257's MindSpeed body/readset. The updated training prerequisites match that body; the added field documentation and weight-loading projection have no additional finding in this review.
Scope, independent checks and limits
The earlier review covered 15 sidecars at 1a52db06. Fourteen remain byte-identical, including their bodies. This pass reviewed the two added sidecars, training-resource-budget's changed prerequisites, field semantics and affected catalog/coverage consumers against the accepted 17-method contract.
A temporary data snapshot matched all 2,325 public head blobs. Unchanged trusted-main generators reproduced all 77 catalog records + README, 559 source hashes, and coverage. 51 offline catalog/coverage/index tests passed. All 17 sidecar changes are confined to applicability/recommendations; excluded owners and evidence/maturity fields remain unchanged. This was independently reconstructed data/projection verification, not a claim to have repeated the author's clean-clone command.
API base is ffb77f9a; observed main is 3ece458c. Main's intervening 23-file pilot change does not overlap this PR. Ready 41158189041 is green, but neither CI nor the author receipt clears the semantic finding. Skill Eval remains planned/unexecuted with 96 uncovered paths: no efficacy, device qualification, promotion or completion of #91 is inferred. No model/hardware/paid execution was performed.
COMMENTED because the authenticated account is also the PR author; this records a blocking finding, not a formal self-REQUEST_CHANGES or approval. Next owner: existing PR author.
| "A causal-conv1d or GDN operator is substituted on build success, import success or matching names without resolving dtype, strides, packed boundaries, state rows and cursors", | ||
| "An in-tree vLLM wrapper, a GDN prefill flag or a cache-free classification adapter is read as establishing the executed convolution path or generation-state support" | ||
| ], | ||
| "refusal_point": "Declines to start a kernel until a profiler puts the target on the critical path near its hardware ceiling or names a concrete fusable pair, and sends host-bound cases elsewhere; refuses to benchmark an unverified kernel and withholds a gain whose profiler metric did not move; keeps operator source selection separate from executed dispatch, numerical parity and hardware qualification, so build or import success qualifies nothing; treats AscendC and TileIR content as source-inferred pending CANN and SoC verification with no local NPU run, treats the AKG Affine FileCheck expectations as IR structure rather than executed CUDA correctness, and reads author-reported overlap and FP8 cases as counterexamples to unconditional tuning rules; claims no Ascend support from NVIDIA primitives", |
There was a problem hiding this comment.
P2 — Preserve the body's unsupported-variant route. The same-head body, lines 25–32 permits a custom kernel when the standard implementation does not cover the required dtype, shape or quantization. This refusal instead requires either a profiled critical-path/near-ceiling target or a fusion pair. A caller needing an unsupported variant can therefore be refused even though the body expressly allows that case; there may be no usable standard implementation to profile. The same restriction appears in produced_artifact (line 163) and the first recommendation (line 183), and is copied into discovery records.
The accepted field contract forbids inventing a refusal not stated by the body. Preserve the third alternative in those three descriptions, retaining the verification-before-benchmark and no-unmeasured-gain limits. Then regenerate from a new reachable source snapshot. This needs a source-fidelity correction, not a hardware run or new efficacy gate.
Retain unsupported dtype, shape or quantization variants as a separate reason to write a kernel in the artifact, refusal and first advice fields. Keep verification before benchmarking and the existing evidence limits. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (source fidelity fix) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Pin the canonical skill index to the reachable corrected source commit and refresh the coverage view from the same committed input bytes. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (generated projections and validation) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Dong1017
left a comment
Dong1017
left a comment
There was a problem hiding this comment.
The unsupported-variant source finding is fixed at b4174929, but this version is not ready for merge: the PR description now creates an unintended tracker-closing association.
- Fixed: the produced artifact, refusal condition and first recommendation now preserve all three routes from the unchanged kernel guide, including a required dtype/shape/quantization variant unsupported by the standard implementation. The new canonical source snapshot is public and retained.
- P2 — The PR description would close the living tracker. Its final validation paragraph uses a negated closing-keyword reference. GitHub already returns the open governing tracker in this PR's
closingIssuesReferences; English negation has not prevented that association. Replace that sentence with a plainRefs #91statement and verify that the automatic closing association disappears before merging. This is a current description defect, not the repaired source finding. - Next action stays with the existing author/integrator, @Dong1017. Preserve the Sarathi addition now on main, regenerate shared catalog/coverage from the combined committed sources, and reassess the resulting head and checks. This scoped fix confirmation is not LGTM, formal approval or tracker acceptance.
Evidence and scope
Reviewed head: b4174929dbd4909b8ac840562798aec72ed66e36; API base: ffb77f9a168ba1cb22f7ba261f289d24dbb6698f; assessed main: 55c2afd1721d9b53cff7974f86066b6d46eb63ec.
The source finding was on a559b12a1dcf1d549e801dd645a949b0f730a75f. The incremental canonical diff changes exactly its three indicated values; source snapshot 5b9cf9aa63b893d228d257960f8cc6ef72d12717 is reachable and no library input changes between that snapshot and this head.
Description inspected at updated_at 2026-10-31T04:12:35Z, body SHA-256 a090b246d2ae0287e6a9ee09d83d29f672ce5563f9ffb3031dd7d3b7751f11c3. A read-only GitHub GraphQL query returned head b4174929dbd4909b8ac840562798aec72ed66e36, base main, and one closing target: open tracker 91, with no further page. GitHub's documented keyword behavior explains the default-branch effect. The tracker explicitly retains broader work; scoped software delivery does not authorize its closure.
Source/metadata, discussion, current-head CI metadata and the automatic association were inspected. No local tests, workflow rerun, model, GPU, paid evaluation, description edit or thread resolution was performed. COMMENTED because the posting account is also the PR author account.
Retain the Sarathi source, case, metadata and matrix additions alongside all accepted method applicability and recommendations. Keep the combined source snapshot reachable for the subsequent generated projection commit. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (source integration) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Pin the canonical catalog to the reachable source merge and refresh the coverage view from the same committed input bytes. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (generated projections and validation) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Preserve upstream source and matrix additions with accepted method guidance. Check the actual diffusion query contribution and attribution instead of assuming matching records carry a diffusion scenario filter label. Keep original-record and unverified-target boundaries intact. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (source integration and contract test fix) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Pin the catalog to the reachable combined source commit and regenerate the coverage view from identical committed source bytes. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (generated projections and validation) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Retain the upstream performance-ceiling source, limitations, readset and matrix additions alongside all accepted method applicability and advice. Preserve the research-query assertion repair and the published history. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (source integration) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Pin the catalog to the reachable combined source commit and regenerate the coverage view from the committed source inputs. Agent-assisted-by: Codex / gpt-6-astra [effort=xhigh] (generated projections and validation) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Dong1017
left a comment
Dong1017
left a comment
There was a problem hiding this comment.
Scoped software/source LGTM at 2e6461a053f52de6bf122fee8cf76c1f8b3ef800 (tree cc451ebd51663d02b956d39ae25012bc021d2866), against current main 6810870046b4f493d39b6eaede0f1a4004369235. No remaining substantive blocker found for the accepted 17-method descriptive-metadata scope.
Previous findings are cleared:
- The unsupported-variant P2 is repaired in exactly the three reported values; all three body-supported kernel routes and verification-before-benchmark limits remain.
- The source-snapshot P2 and count P3 remain addressed. Public snapshot
54d4c89dis the head's direct parent, with identical indexed source inputs; the frozen-baseline count is 41 → 24. - The tracker-closing association P2 is addressed: the body uses
Refs #91, and the loaded GitHub Development panel explicitly shows “None yet” for issues that merging may close. This is a current UI readback, not inference from wording alone.
The changed research assertion is justified, not a missing-task regression masked by loosening the test. scenario is a record/category filter: relevant packaged procedures legitimately use skills.discovery. The current result includes the leading diffusion-query record and five task/diffusion-attributed records. The replacement tests actual query contribution and attribution; synthetic fair-union, filtering, unchanged-record and unverified-target checks remain. Production search is unchanged.
Independent checks, integration and limits
- Reconciled the earlier 15-method review, my
a559b12areview, theb4174929follow-up and intervening integration. Compared all 17 current-main sidecars: only applicability/recommendations differ. Other library sources/resources and both matrices match main, preserving #262/#257 and Sarathi/DistServe/Mooncake additions. - Verified 564 canonical source hashes against the public head/source trees. Unchanged trusted-main generators reproduce 77 catalog records + README and coverage exactly from a temporary data snapshot; this is not a claimed clean-clone/native-generator execution.
- 67 focused offline tests passed: 16 research, 26 catalog, 17 coverage and 8 index. No model, device, benchmark or paid execution.
- Ready 41173855047 passes both full suites, both installed-package jobs and model-free KV qualification. The initial declaration failure 41173853413 is superseded by later successes at the same head and workflow, latest 41174255699. Current UI reports all checks passed/no conflicts.
- The accepted batch requires source representation, not efficacy execution. Skill Eval remains planned/unexecuted with 97 uncovered paths; none is counted as a pass. No maturity/evidence promotion, native-device qualification or broader tracker completion is inferred.
This supports the bounded software merge decision while these head/base/check identities remain current. Historical inline threads are outdated but their UI resolution state was not changed. COMMENTED, not formal approval; #91 remains open.
Summary
Complete Batch 2 of #91: the selected 17 methods skills gain descriptive applicability and procedure-specific recommendations under the accepted field contract. All 20 registered methods now have applicability metadata; the other batches remain separate deliveries.
The field meanings are documented in docs/knowledge-base.md. Kernel-writing-guide and weight-loading-optimization retain their body-supported device requirements, separate from tuning advice. Source bodies, provenance, maturity and empirical-validation fields are preserved.
Review corrections
ffb77f9a168ba1cb22f7ba261f289d24dbb6698f, no-applicability falls from 41 to 24 across the 17 selected methods. knowledge(hardware): audit sources and deepen memory, graph and collective guidance #262's hardware owners and Recover historical MindSpeed provenance and allocator prerequisites #257's MindSpeed readset remain intact, including the recovered training-resource-budget conditions.This branch also integrates main
6810870046b4f493d39b6eaede0f1a4004369235, preserving the Sarathi, DistServe and Mooncake sources, both matrices and upstream evidence additions/changes alongside the 17 methods' accepted fields.The research regression assertion now verifies actual task/diffusion query contribution and attribution instead of requiring a scenario-field label. Production search and the source-boundary checks remain unchanged.
Validation and provenance
2e6461a053f52de6bf122fee8cf76c1f8b3ef800.54d4c89d4db82cbfc10d6096fe49bb077136e8b1, followed by a separate generated-projection commit.2e6461a053f52de6bf122fee8cf76c1f8b3ef800; Skill Eval remains planned/unexecuted.These are source and software checks, not efficacy or native-device qualification. Skill Eval execution remains unexecuted; uncovered scope is not a pass. Independent current-head review and merge remain pending. Refs #91. The broader tracker remains open.
Earlier material-author declarations are retained without retrospective verification. The user confirmed Codex / gpt-6-astra / xhigh for the finalization and corrections.