test(docs): CLI.md context/audit --quick section headers checked against real re-runs - #2510
Merged
Merged
Conversation
…nst real re-runs Extends #2212's docs-drift protection for CLI.md with a third tier: for the two commands whose documented output already uses clear ## Section headers (context, audit --quick), extract just those header labels (normalizing away a trailing dynamic count) and compare against a real re-run against a small fixture graph. Deliberately scoped to header labels only, not full output content, which would need real dynamic- content normalization rules (file paths, line numbers, source snippets) — the harder half of #2212's original ask, still deferred. Already caught a real bug during development: the documented context example was missing the Children/Complexity sections added to the CLI since the doc was last updated, and had stale names for two others (Dependencies -> Direct Dependencies (N), Callers -> Callers (N)) — fixed in this same change, mirroring #2212's own precedent. MCP.md's equivalent (tool-call response shape vs. real MCP server output) is a separate, unaddressed piece of scope, filed as #2509. Closes #2369 docs check acknowledged
Contributor
Greptile SummaryThe PR adds a third tier of CLI documentation-drift coverage that reruns
Confidence Score: 5/5The PR appears safe to merge, with no actionable correctness or security issues identified. The updated examples match the section conditions exercised by the fixture, and the new tests consistently compare the intended documented and live CLI output shapes. Important Files Changed
Reviews (1): Last reviewed commit: "test(docs): cli.md context/audit --quick..." | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
#2212 extended docs-drift protection to `docs/examples/CLI.md` with two tiers (static command/flag validity, and representative real execution), but neither verifies that a documented example's output content/shape still matches what the command actually produces — only that it's syntactically valid and doesn't crash.
Fix
Added a Tier 3, deliberately scoped to just the two commands whose documented output already uses clear `## Section` markdown-style headers (`context`, `audit --quick`), per the issue's own suggested starting point:
Real bug caught during development: the documented `context` example was missing two sections that exist in the current CLI (`## Children`, `## Complexity`) and had stale names for two others (`## Dependencies` → `## Direct Dependencies (N)`, `## Callers` → `## Callers (N)`). Fixed in this same PR, mirroring #2212's own precedent of fixing what its own new check finds. `audit --quick`'s documented headers already matched exactly — no doc change needed there.
Deliberately out of scope
Closes #2369
Test plan