Skip to content

fix: make ref-check's suggestions resolve, and cut its false positives - #94

Merged
brentvatne merged 1 commit into
mainfrom
@brent/ref-check-precision
Oct 23, 2026
Merged

brentvatne merged 1 commit into
mainfrom
@brent/ref-check-precision

Conversation

@brentvatne

Copy link
Copy Markdown
Member

Why

Swept against expo/expo's .expo-agents/code-review/ setup, ecr ref-check reported 149 problems. An adversarial pass over them found that following the check's own advice would often fail or mislead:

  • Suggestions that can't resolve. About 28% of the "add @ref X" messages named an X that would itself be a broken ref: a partial path (start/doctor/apple/XcodePrerequisite.ts), a directory glob (packages/*/apple/), a ./internal/* pattern, or a file that was deleted (tools/src/publish-packages/helpers.ts, the one real stale citation).
  • A suffix-glob ref didn't silence its citation. glob:**/start/doctor/apple/XcodePrerequisite.ts resolves, but the prose start/doctor/apple/XcodePrerequisite.ts was still reported.
  • Package-relative names failed by shape. ios/, src/ and __tests__/ mean "each package's directory", which no ref can pin (28 of the 149).
  • Native build files were never checked. .mm, .gradle, .xml, .plist and .podspec weren't citation extensions (26 citations in expo/expo).
  • Top-level directory refs flooded the drift note. The "this PR changes code the reviewer prompts cite" note matches by prefix. With every suggestion followed, it would fire on 282 of expo/expo's last 300 commits.
  • Line-number citations were called unresolvable. The review comment said "cites code that no longer resolves" for a src/a.ts:42 citation of a file that exists.

What

All in src/core/config-refs.ts, with LLP 0012 and the README updated. Grammar and glob dialect are unchanged, so ./ref-check needs nothing.

  • resolvingRef picks a suggestion that resolves:
    • the resolved path or glob;
    • else the one file a partial path is a suffix of;
    • else glob:**/<token> when several files match;
    • a ** tail for directory globs, and no leading ./.
  • When nothing matches, the message says the citation may be stale.
  • Coverage also accepts the suggestion's own forms and reads **/ as zero or more directories, so writing the suggestion down always silences the citation.
  • dirExistsAnywhere treats a non-resolving trailing-slash token as prose when such a directory exists somewhere. iso/ or a deleted oldpkg/ still fails. LLP 0012 states the tradeoff: a stale foo/bar/ whose name survives elsewhere goes quiet.
  • The citation extensions add .m, .mm, .gradle, .kts, .xml, .plist, .podspec, .pbxproj, .xcconfig, .entitlements, .properties, .cmake, .pro and .mdx. .env stays off, because process.env is the far commoner token.
  • A single-segment directory ref must still resolve but no longer feeds citedPaths.
  • reviewSetupRefNotes counts only broken-ref and structural problems.

Adopter impact: the new extensions can make ecr ref-check report citations it ignored before. The review path never shows unannotated citations, so reviews themselves don't change.

Tests

  • 8 new tests in config-refs.test.ts, one per fix. Each fails with its fix reverted.
  • End to end against expo/expo's unannotated setup: 148 problems, versus 149 before. 28 package-relative names dropped out and 27 native-file citations came in.
  • A script wrote every suggested ref into a copy of that setup. All 140 resolve and each silences its citation. The 8 left are citations that match nothing: the deleted helpers.ts, .claude/CLAUDE.md, state.json, and five paths in the uninitialized react-native submodule.
  • Typecheck, bun test (876 pass), lint, format and llp:check pass, and this repo's own setup still passes ecr ref-check.

🤖 Generated with Claude Code

Swept against expo/expo, ref-check reported 149 problems, and following its
"add `@ref X`" advice would have broken or misled in several ways:

- Suggestions that don't resolve. A partial path now gets the one file it is
  a suffix of (or `glob:**/<token>` when several match), a wildcard directory
  gets a `**` tail, and a leading `./` is dropped. A citation that matches
  nothing says so and that it may be stale, instead of suggesting a broken ref.
- A suffix-glob ref didn't silence the citation it covers: coverage now reads
  each `**/` as zero or more directories.
- Package-relative directory names (`ios/`, `src/`, `android/src/`) are prose
  when such a directory exists somewhere in the repo; a name that exists
  nowhere still fails.
- Native build files (`.mm`, `.gradle`, `.xml`, `.plist`, `.podspec`, ...)
  are citations now; they were never checked.
- A ref to a top-level directory still has to resolve but no longer drives
  the "this PR changes code the reviewer prompts cite" note, which matched it
  by prefix and fired on nearly every PR.
- The review comment's "cites code that no longer resolves" note counts only
  refs that fail to resolve, not line-number citations of files that exist.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 AI code review

Decision: Ready for human review

Overall PR risk: Low. This change modifies the ref-integrity checker to improve suggestion quality and add missing file extensions. It affects only ecr ref-check citation detection and the review setup notes; the blast radius is limited to the ref-checker module and is easily reverted. Adopting repos may see new citation reports for previously unchecked file types (.xml, .m, .pro, etc.), and stale directory names that exist elsewhere now pass silently instead of being flagged.

🔎 Reviewer summaries (3)
  • consistency: Checked the new citation extensions, the resolving-suggestion logic, the top-level-dir drift cut, and the review-note filter against sibling code in config-refs.ts and ./ref-check. The change follows the existing patterns. I found no consistency defect.
  • correctness: I reviewed the ref-check precision changes: the native extensions, the resolving suggestions, the package-relative directory skip, the top-level directory drift cut, and the review-note filter. I found no concrete correctness defect.
  • security: I reviewed the ref-check precision changes for security impact. The changes do not touch token forwarding, child processes, prompt sanitization, or comment state markers. I found no security or secret risk.

🔗 Review setup:

  • This PR changes code the reviewer prompts cite (src, src/tests). Check that the guidance quoting it is still correct.

No findings.


This review is advisory — it never blocks a merge and never auto-approves.

@brentvatne
brentvatne merged commit a50c15e into main Oct 23, 2026
3 checks passed
@brentvatne
brentvatne deleted the @brent/ref-check-precision branch October 23, 2026 02:44
brentvatne added a commit to expo/expo that referenced this pull request Oct 23, 2026
The next ecr release (expo/code-review-cli#94) also checks citations of
native build files (`Info.plist`, `build.gradle`, `AndroidManifest.xml`,
`*.podspec`, ...). Pin the 25 the prompts make as `glob:` refs, so
`ecr ref-check` keeps passing on both 0.19.0 and that release.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brentvatne added a commit to expo/expo that referenced this pull request Oct 23, 2026
0.20.0 adds expo/code-review-cli#94 (ref-check precision fixes) on top of 0.19.0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant