|
| 1 | +# Manual Testing Guide |
| 2 | + |
| 3 | +Any change that affects a slash command's behavior requires manually testing that command through an AI agent and submitting results with the PR. |
| 4 | + |
| 5 | +## Process |
| 6 | + |
| 7 | +1. **Identify affected commands** — use the [prompt below](#determining-which-tests-to-run) to have your agent analyze your changed files and determine which commands need testing. |
| 8 | +2. **Set up a test project** — scaffold from your local branch (see [Setup](#setup)). |
| 9 | +3. **Run each affected command** — invoke it in your agent, verify it completes successfully, and confirm it produces the expected output (files created, scripts executed, artifacts populated). |
| 10 | +4. **Run prerequisites first** — commands that depend on earlier commands (e.g., `/speckit.tasks` requires `/speckit.plan` which requires `/speckit.specify`) must be run in order. |
| 11 | +5. **Report results** — paste the [reporting template](#reporting-results) into your PR with pass/fail for each command tested. |
| 12 | + |
| 13 | +## Setup |
| 14 | + |
| 15 | +```bash |
| 16 | +# Install the CLI from your local branch |
| 17 | +cd <spec-kit-repo> |
| 18 | +uv venv .venv |
| 19 | +source .venv/bin/activate # On Windows: .venv\Scripts\activate |
| 20 | +uv pip install -e . |
| 21 | + |
| 22 | +# Initialize a test project using your local changes |
| 23 | +specify init /tmp/speckit-test --ai <agent> --offline |
| 24 | +cd /tmp/speckit-test |
| 25 | + |
| 26 | +# Open in your agent |
| 27 | +``` |
| 28 | + |
| 29 | +## Reporting results |
| 30 | + |
| 31 | +Paste this into your PR: |
| 32 | + |
| 33 | +~~~markdown |
| 34 | +## Manual test results |
| 35 | + |
| 36 | +**Agent**: [e.g., GitHub Copilot in VS Code] | **OS/Shell**: [e.g., macOS/zsh] |
| 37 | + |
| 38 | +| Command tested | Notes | |
| 39 | +|----------------|-------| |
| 40 | +| `/speckit.command` | | |
| 41 | +~~~ |
| 42 | + |
| 43 | +## Determining which tests to run |
| 44 | + |
| 45 | +Copy this prompt into your agent. Include the agent's response (selected tests plus a brief explanation of the mapping) in your PR. |
| 46 | + |
| 47 | +~~~text |
| 48 | +Read TESTING.md, then run `git diff --name-only main` to get my changed files. |
| 49 | +For each changed file, determine which slash commands it affects by reading |
| 50 | +the command templates in templates/commands/ to understand what each command |
| 51 | +invokes. Use these mapping rules: |
| 52 | +
|
| 53 | +- templates/commands/X.md → the command it defines |
| 54 | +- scripts/bash/Y.sh or scripts/powershell/Y.ps1 → every command that invokes that script (grep templates/commands/ for the script name). Also check transitive dependencies: if the changed script is sourced by other scripts (e.g., common.sh is sourced by create-new-feature.sh, check-prerequisites.sh, setup-plan.sh, update-agent-context.sh), then every command invoking those downstream scripts is also affected |
| 55 | +- templates/Z-template.md → every command that consumes that template during execution |
| 56 | +- src/specify_cli/*.py → CLI commands (`specify init`, `specify check`, `specify extension *`, `specify preset *`); test the affected CLI command and, for init/scaffolding changes, at minimum test /speckit.specify |
| 57 | +- extensions/X/commands/* → the extension command it defines |
| 58 | +- extensions/X/scripts/* → every extension command that invokes that script |
| 59 | +- extensions/X/extension.yml or config-template.yml → every command in that extension. Also check if the manifest defines hooks (look for `hooks:` entries like `before_specify`, `after_implement`, etc.) — if so, the core commands those hooks attach to are also affected |
| 60 | +- presets/*/* → test preset scaffolding via `specify init` with the preset |
| 61 | +- pyproject.toml → packaging/bundling; test `specify init` and verify bundled assets |
| 62 | +
|
| 63 | +Include prerequisite tests (e.g., T5 requires T3 requires T1). |
| 64 | +
|
| 65 | +Output in this format: |
| 66 | +
|
| 67 | +### Test selection reasoning |
| 68 | +
|
| 69 | +| Changed file | Affects | Test | Why | |
| 70 | +|---|---|---|---| |
| 71 | +| (path) | (command) | T# | (reason) | |
| 72 | +
|
| 73 | +### Required tests |
| 74 | +
|
| 75 | +Number each test sequentially (T1, T2, ...). List prerequisite tests first. |
| 76 | +
|
| 77 | +- T1: /speckit.command — (reason) |
| 78 | +- T2: /speckit.command — (reason) |
| 79 | +~~~ |
0 commit comments