Skip to content

fix(mcp): make --skill-name-filter read the v2 install manifest - #119

Merged
knihit merged 9 commits into
mainfrom
fix/skill-name-filter
Oct 28, 2026
Merged

knihit merged 9 commits into
mainfrom
fix/skill-name-filter

Conversation

@knihit

@knihit knihit commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Description

On Kiro, --skill-name-filter filtered nothing. Every agent's skills server indexed and served all 82 installed skills instead of only the agent's declared ones. This PR makes the filter work. The stacked PR on feat/agent-skill-index builds on it to get agents to load their skills unprompted.

MCP skill server:

  • install_manifest.rs: read_installed_paths now reads the v2 manifest the installer actually writes (strategies[].files[].path, unioned across strategies). It still reads the v1 flat files[].path, because an older install's v1 manifest stays on disk until the next install migrates it. Before this, the reader only knew v1. So it found no installed paths, treated every skill as hand-authored, and the name filter skipped all of them.
  • Fail-open is unchanged: any malformed entry, or a strategy without files, still gives an empty set.
  • A missing manifest stays silent. A manifest that exists but can't be read (permission denied, a directory at the path, other I/O errors), isn't valid JSON, or has no recognized shape logs one skill-lookup-mcp: WARN naming the manifest path and saying --skill-name-filter will not apply under that root. read_installed_paths returns a Result whose read error carries the io::ErrorKind, and only installed_paths_for_root logs.

Tests:

  • Fixtures in index.rs, scanner.rs, initialize_handshake.rs and reload_skills_shakedown.rs used the v1 shape, which is why the bug went unnoticed. They now use v2.
  • New unit tests cover a v2 manifest, two strategies, and a strategy without files.
  • New e2e test in get_skill_shakedown.rs: the server runs with a v2 manifest and --skill-name-filter keep-me. reload_skills reports filtered_out: 1, get_skill drop-me returns not-found, and keep-me and a hand-authored skill are still served.
  • With the reader change reverted, the new e2e test and three existing filter tests fail.
  • Shared contract fixture mcp/lib/skill-lookup-core/testdata/install-manifest.v2.json, so the reader's fixtures can't drift from the writer again. In cli/konductor-rs, serialized_manifest_matches_shared_contract_fixture builds a manifest from the real types, runs the production serialize(), and asserts it equals the file byte for byte. In skill-lookup-core, read_installed_paths_parses_installer_contract_fixture asserts the paths read from the same file. Both reach it through CARGO_MANIFEST_DIR, with no cross-workspace dependency.
  • installed_paths_for_root has a WARN test for each logged case (unrecognized shape, invalid JSON, a directory at the manifest path) and a test that a missing manifest logs nothing. They assert on the manifest message, not on the whole log, since other tests in the crate log through HOME without the HOME lock. HOME is restored by HomeGuard, now shared from test_home_lock, so a panic can't leak the override. The crate's tests passed 8 of 8 runs with --test-threads=16.

Changelog:

  • One bullet under [Unreleased] / Fixed. No VERSION change.

Type of Change

  • Other (describe below): bug fix in the skills MCP server (mcp/)

Testing

  • npm test: 119 of 136 pass. The same 17 fail on main (claude-code-agent-runner, generate-agent-files, run-claude-subset), and this PR touches none of their code. Not rerun after the review fixes, which change no JS.

  • cd cli && make test: 1780 passed, 0 failed. One test that runs synth from a deleted cwd failed once under parallel load on macOS, then passed alone 3/3 and on a full rerun.

  • mcp make test: 327 passed, 0 failed (skill-lookup-core 176).

  • make -C cli lint passes. make synth leaves no tracked changes. make test-skills-sh passed, make claude-plugin-check pass, make test-py 222 passed, kiro-power-check passed

  • Smoke tested: make build && make synth, then install --from . --harness kiro-cli-v2 --no-telemetry into a scratch target and HOME. Running k-developer's skill server with its installed args over stdio, reload_skills gave indexed: 27, filtered_out: 55 (before: indexed: 82, filtered_out: 0). get_skill adr-generator returned "skill not found", and get_skill code-review returned its body.

  • Benchmark: not run on this branch alone. It was run on the stacked feat/agent-skill-index branch, which includes this fix.

    Setup: same tasks T1 to T4 plus a no-skill control, 2 reps each, project-scope install with --no-telemetry, isolated HOME per run, scored from session logs.

    Harness Variant n Loaded a skill Right skill
    Kiro CLI v3 main 12 4 3
    Kiro CLI v3 stacked branch 12 9 9
    Claude Code main 12 5 5
    Claude Code stacked branch, with a Claude line it later dropped 12 6 6

    The stacked PR's prompt block is Kiro-only. The filter's effect on its own is narrower than the table: each Kiro agent's server now offers only that agent's skills. 0 of 28 Kiro runs reported the skills server unavailable, and no get_skill call failed. Caveats: 2 runs per task cell, and main varied by about 1 run in 12 between two benchmark passes.

    After the review fixes, the stacked branch was re-benched on Kiro CLI v3 with two more reps per task: pooled right skill 13 of 22 on its final head, against 15 of 23 before the review fixes (within noise) and 3 of 12 on main. None of those runs reported the skills server unavailable or had a get_skill call fail. See the stacked PR for the table. The fixes on this branch don't change which skills the server serves.

  • No new code files. Existing files keep their SPDX headers. The new JSON fixture has no comment syntax, so it carries no header.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own changes
  • I have commented my code where necessary (doc comments name both manifest shapes)
  • I have made corresponding changes to the documentation (CHANGELOG, and mcp/README.md: the --skill-name-filter row says only manifest-listed skills can be excluded and that a broken manifest turns the filter off with a WARN. .gitattributes pins the byte-compared manifest fixture to LF, so the writer test passes on Windows checkouts with core.autocrlf.)
  • My changes generate no new warnings: make -C mcp lint clippy -D warnings is clean. cargo fmt --check only flags lines unchanged from main, which fail the same way on main with rustfmt 1.9.0.
  • I have added or updated tests that prove my change works

Agents Affected

Kiro CLI only. Claude Code installs have no skills MCP server. Each agent's skills server now serves only that agent's declared skills plus hand-authored ones:

  • konductor
  • konductor-mux-orchestrator
  • konductor-cmux-orchestrator
  • k-developer
  • k-architect
  • k-researcher
  • k-product-manager
  • k-quality-assurance
  • k-tpm
  • k-browser
  • k-media-analyzer

Notes for Reviewers

  • Install is the only producer of --skill-name-filter, and no runtime caller relied on the unfiltered set. Every specialist's prompt and SOPs reference only its declared skills. The orchestrators' undeclared references are all delegated to a specialist or conditional.
  • One soft fallback: k-plan.sop.md lets konductor apply legacy-to-agentic-estimate itself if k-tpm is unreachable. get_skill now returns not-found for it, and the SOP's next step already says to skip the projection, so it degrades gracefully.
  • Six commits. The second restores two existing tests in get_skill_shakedown.rs that a bad save garbled in the first. Every pre-existing line in that file matches main. The rest address review: the shared manifest fixture, the WARN on an unrecognized manifest, the shared HomeGuard, and the WARN on an unreadable or invalid manifest.
  • No conflict with fix(orchestrator): tighten the direct-work rules #114.

Nihit Kasabwala added 2 commits October 27, 2026 18:56
…ilter applies

The installer writes .konductor/manifest as schema v2
({schema_version: 2, strategies: [{..., files: [...]}]}), but the skill
server read only a top-level files array (schema v1). It found no
installed paths, treated every skill as hand-authored, and
--skill-name-filter removed nothing.

read_installed_paths now unions strategies[].files[].path for v2 and
still reads files[].path for a v1 manifest left by an older install.
Any malformed entry still fails open.

Test fixtures in index.rs, scanner.rs, initialize_handshake.rs and
reload_skills_shakedown.rs used the v1 shape, which is why the bug
went unnoticed; they now use v2. New tests cover v2, two strategies,
a strategy without files, and get_skill returning not-found for an
installed skill outside the filter while a hand-authored skill is
still served. With the reader change reverted, the new get_skill test
and three existing core filter tests fail.

No runtime caller relies on the old behavior. The only producer of
--skill-name-filter is the Kiro install path.

Verification (macOS, TMPDIR=/tmp):
- make build: pass. make synth: pass (11 agents, 82 skills).
- make -C cli test: 1779 passed, 0 failed. No flakes, so no
  resolve_cwd_json_envelope / telemetry_hook_scoping reruns needed.
- mcp cargo test --workspace: 322 passed, 0 failed.
- make -C mcp lint: clippy -D warnings clean. cargo fmt --check
  reports diffs only in lines unchanged from origin/main
  (find_skills_shakedown, full_tool_surface, get_skill_shakedown,
  handshake_shakedown); the same files fail fmt --check on main with
  local rustfmt 1.9.0.
- make test-skills-sh: 10 passed, 0 failed.
- make claude-plugin-check: pass.
- make test-py kiro-power-check (venv, pytest 8.3.4, PyYAML 6.0.2):
  222 + 12 passed, kiro-power 12/10/5 run, 0 failed.
- Live probe: install --from . --harness kiro-cli-v2 --no-telemetry
  into a scratch target and HOME, then k-developer's skill server
  args over stdio: reload_skills gave indexed 27, filtered_out 55;
  get_skill adr-generator returned "skill not found"; get_skill
  code-review returned its body.
…s commit

The previous commit's copy of get_skill_shakedown.rs had two existing
tests garbled by a bad editor save (a truncated assert in the
case-insensitive test and a missing fn header for the empty-name
test). It still passed cargo test because the test binary was built
before the save. This rebuilds the file from origin/main plus the new
name-filter test, formatted with rustfmt. Every existing line is
identical to origin/main again (133 added, 0 removed).

Verification (macOS, TMPDIR=/tmp):
- mcp cargo test --workspace: 322 passed, 0 failed (run after this
  change, before staging).
- cargo test --test get_skill_shakedown: 10 passed, 0 failed.
- cargo fmt --check: the new test block is clean; the remaining diffs
  are in lines unchanged from origin/main (same as on main with local
  rustfmt 1.9.0).
- make -C mcp lint's clippy -D warnings: clean. clippy --all-targets
  flags needless_return in telemetry.rs, which is unchanged from
  origin/main.
Nihit Kasabwala added 2 commits October 27, 2026 20:41
The skill lookup reader and the installer live in separate Cargo
workspaces, and each side tested against its own hand-written manifest.
Add mcp/lib/skill-lookup-core/testdata/install-manifest.v2.json as the
one contract fixture: the installer test asserts its serializer output
equals the file byte for byte, and the reader test asserts
read_installed_paths returns the expected paths from it. A writer change
now fails a test on one side instead of silently disabling the filter.

Verified:
- cargo test -p skill-lookup-core: 172 passed
- cargo test --bin konductor cli::install::manifest (TMPDIR=/tmp): 56 passed
- cargo clippy --workspace -D warnings in mcp: clean
A manifest that parses as JSON but matches neither the v2 strategies
shape nor the v1 files shape used to turn the per-agent name filter off
for its skills root without any log line. installed_paths_for_root now
emits one "skill-lookup-mcp: WARN" naming the manifest path and saying
--skill-name-filter will not apply to that root. It still fails open
with an empty set, and a missing manifest stays silent.

read_installed_paths now returns a ManifestReadError (Unreadable,
InvalidJson, UnrecognizedShape) instead of None, so the caller can tell
the cases apart without the reader logging on its own.

Verified:
- make -C cli test (TMPDIR=/tmp): 1780 passed; one deleted-cwd test
  (resolve_cwd_json_envelope) flaked once under load and passed 3/3 in
  isolation and on a full rerun
- make -C mcp test-rust: all passed (skill-lookup-core 174)
- make synth, make test-skills-sh, make claude-plugin-check: passed
- make test-py (pytest 8.3.4, PyYAML 6.0.2 venv): 222 passed
- make kiro-power-check: passed
- make -C cli lint: passed; mcp clippy --workspace -D warnings: clean
@knihit
knihit marked this pull request as ready for review October 28, 2026 02:44
Move HomeGuard from logging.rs's tests into test_home_lock, next to the
lock it holds, so other modules can point HOME at a scratch dir and get
it restored on drop, including when a test panics.
ManifestReadError::Unreadable now carries the io::ErrorKind. A missing
manifest stays silent. Any other read error, invalid JSON, or an
unrecognized shape logs one WARN naming the manifest path, since the
name filter is off for that root. The result is still an empty set.

The WARN tests assert on the manifest message instead of counting every
WARN in the log, because other tests log through HOME without taking
the HOME lock. HOME is restored through HomeGuard. New tests cover a
directory at the manifest path and a manifest that is not JSON.
The malformed-manifest scanner test logged its WARN without holding the
HOME lock, so it could land in another test's scratch log or in the real
~/.konductor/mcp/logs. Take the HomeGuard there, and have the manifest
warning assertions match only their own manifest path.
…manifest fixture to LF

The README's --skill-name-filter row now says only manifest-listed
skills can be excluded and that a broken manifest turns the filter off
with a WARN. The SOP filter row no longer claims the same semantics.
.gitattributes keeps the byte-compared contract fixture LF on every
checkout.
@knihit
knihit merged commit 7428dd9 into main Oct 28, 2026
6 checks passed
@knihit
knihit deleted the fix/skill-name-filter branch October 28, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants