Skip to content

Running pytest tests/ silently deletes the user's installed Claude skill (test_claude_md.py + claude_uninstall project_dir leak) #2168

Description

@AirRocker

Impact

Any contributor who runs the test suite on a machine that has graphify installed for Claude gets their global ~/.claude/skills/graphify/ silently deletedSKILL.md, .graphify_version, and the whole references/ bundle.

Every test passes. Nothing is printed. The deletion is completely invisible from the test output, so the skill just stops working at some later point with no obvious cause.

Triggered by pytest tests/ and equally by pytest tests/test_claude_md.py on its own.

Mechanism

claude_uninstall(project_dir: Path | None = None, *, project: bool = False)graphify/install.py:1757 — calls at line 1770:

_remove_skill_file("claude", project=project, project_dir=project_dir)

project defaults to False. With project=False, _platform_skill_destination resolves the destination via Path.home() and ignores project_dir entirely. So claude_uninstall(tmp_path) does not scope anything to the temp directory — it targets the real global skill and:

  • unlinks SKILL.md
  • unlinks .graphify_version
  • shutil.rmtrees references/
  • rmdirs the parent directories

(which is why the graphify/ directory disappears outright rather than being left empty).

tests/test_claude_md.py has 11 claude_uninstall(tmp_path) call sites and zero Path.home patching. This is inconsistent with the rest of the suite — the _remove_skill_file call sites in test_install_roundtrip.py, test_install_references.py and test_devin.py are all correctly wrapped in patch("graphify.__main__.Path.home", ...).

The tests pass because their assertions only inspect the markdown files under tmp_path; nothing asserts on the skill tree, so its removal is never observed.

Reproduction

  1. Install the skill for Claude so ~/.claude/skills/graphify/ exists (10 files).
  2. Start a filesystem watch on ~/.claude/skills/graphify/.
  3. Run pytest tests/test_claude_md.py -v.

The deletion fires on the first uninstall test to execute, and that test passes.

Watch-log excerpts from a verified run (poller sampling ~/.claude/skills/graphify/ every 100 ms):

Full suite — reported 3607 passed, 3 skipped:

16:18:32.279 START  exists=True  files=10
16:18:38.890 CHANGE exists=False files=0 REMOVED=['.graphify_version', 'SKILL.md',
             'references/add-watch.md', 'references/exports.md',
             'references/extraction-spec.md', 'references/github-and-merge.md',
             'references/hooks.md', 'references/query.md',
             'references/transcribe.md', 'references/update.md']

test_uninstall_cleans_both_standard_and_local in isolation — 1 passed:

16:20:34.926 START  exists=True  files=10
16:20:36.786 CHANGE exists=False files=0 REMOVED=[all 10 files]

test_uninstall_removes_section in isolation — 1 passed:

16:21:44.455 START  exists=True  files=10
16:21:46.105 CHANGE exists=False files=0 REMOVED=[all 10 files]

Both isolated runs confirm it is not one specific test — any of the 11 call sites does it, and in a full run whichever executes first is the one that fires.

Possible fixes — maintainers' call

(a) Sandbox the tests. Patch Path.home in test_claude_md.py the way the rest of the suite already does. Smallest change, fixes the contributor-facing damage.

(b) Make claude_uninstall honour project_dir. Arguably the root cause is the API: a function that accepts a project_dir and then deletes a global path regardless of it is surprising, and the same footgun is reachable from real usage, not just tests.

These aren't mutually exclusive — (a) stops the bleeding, (b) removes the trap. Happy to submit a PR for whichever you prefer; I didn't want to presume the design call.

Note

Unrelated to #2162, and unrelated to the flaky test_labeling.py::test_label_communities_batches_when_over_batch_size thread-scheduling race documented in that PR's description.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions