Skip to content

fix(native): stop dropping dotfile-named source files, fix undercounted build summary - #2532

Merged
carlos-alm merged 1 commit into
mainfrom
fix/issue-2391
Aug 16, 2026
Merged

fix(native): stop dropping dotfile-named source files, fix undercounted build summary#2532
carlos-alm merged 1 commit into
mainfrom
fix/issue-2391

Conversation

@carlos-alm

Copy link
Copy Markdown
Contributor

Closes #2391

Root cause

collect_files.rs's ignore::WalkBuilder set .hidden(true), which blanket-skips both hidden directories AND hidden files. Dotfile-named source files are common and legitimate:

  • .terraform.lock.hcl — Terraform's dependency lock file, meant to be committed to git
  • .pa11yci.authed.cjs — a pa11y accessibility-testing config

Both were silently dropped by the native collector, which then logged "likely a Rust extractor bug" and paid a second WASM parse pass to backfill them — but the real bug was file discovery, not extraction (the extraction logic itself is correct, as proven by WASM producing identical symbols once it got the chance to see the file).

The JS/TS collector (shouldIgnore in shared/constants.ts) only ever skips hidden directories, never files. collect_files.rs now matches that asymmetry exactly — the existing filter_entry closure right below .hidden(false) already re-implements hidden-directory skipping independently, so directory-level behavior (.git, .next, arbitrary dotdirs) is unchanged.

Secondary defect (same issue report)

The build-completion log line (Native build orchestrator completed: N nodes, M edges, K files) reflected only the files the native orchestrator itself collected, captured before any WASM backfill inserted additional file nodes — so K under-reported relative to codegraph stats whenever a backfill happened for any reason. This is the same shape as #1452's node/edge under-report, fixed the same way: re-derive the count (SELECT COUNT(*) FROM nodes WHERE kind='file', matching how codegraph stats counts files) whenever a post-pass wrote data.

Verification

  • Reproduced both defects using the issue's exact file shapes (a real .terraform.lock.hcl with a provider { ... hashes = [...] } block, and a .cjs file) against the pre-fix binary; confirmed the warning, the backfill, and the undercounted completion line.
  • Rust: 2 new unit tests in collect_files.rs (collect_finds_dotfile_named_source_files, collect_still_skips_hidden_directories) — both verified to fail without the fix (temporarily reverted .hidden(false).hidden(true), confirmed failure, restored). Full cargo test: 1034 passed. cargo clippy --all-targets -- -D warnings and cargo fmt --check clean.
  • TS: 3 new integration tests in tests/integration/issue-2391-hidden-dotfile-native-drop.test.ts covering the .hcl and .cjs dotfile cases plus a hidden-directory-still-excluded regression guard — all verified to fail without the Rust fix (same revert-rebuild-confirm-restore cycle).
  • The secondary (file-count) fix was verified manually rather than via an isolated automated test: reproduced the original bug (both defects present), confirmed the completion log undercounted relative to codegraph stats; then restored only the file-count fix (keeping the primary bug reverted) and confirmed the log became correct; then restored both fixes together. No independent automated regression test was constructible for the secondary defect alone — the only known real-world trigger for "native's own file collector misses a file WASM would find" is the primary bug this same PR fixes, so once it's fixed there's no remaining scenario in this codebase where the two file counts diverge. Documented this explicitly in the test file's top comment for future readers.
  • npm test: 5323 passed. npm run lint clean. codegraph diff-impact --staged -T: bounded to runPostNativePasses and its normal build-pipeline callers.

Not in scope

No Rust changes needed beyond collect_files.rs — the HCL/JS extractors' own logic was already correct; native and WASM already agree on symbol shape once native can see the file. No other WalkBuilder instances exist in the Rust codebase (grepped to confirm).

…ed build summary (#2391)

The native orchestrator's file collector set .hidden(true) on its
ignore::WalkBuilder, blanket-skipping both hidden directories AND hidden
files. Dotfile-named source files are common and legitimate
(.terraform.lock.hcl, .pa11yci.authed.cjs), so every build in a repo
containing one silently dropped it, logged a misleading "likely a Rust
extractor bug" warning, and paid a second WASM parse pass to backfill it —
even though the real bug was file discovery, not extraction. The JS/TS
collector (shouldIgnore in shared/constants.ts) only ever skips hidden
directories, never files; collect_files.rs now matches that, with the
existing filter_entry closure continuing to skip hidden directories.

Also fixes the secondary defect from the same report: the build-completion
log line's file count reflected only the files the native orchestrator
itself collected, captured before any WASM backfill inserted additional file
nodes, so it under-reported relative to `codegraph stats` whenever a backfill
happened. Re-derives it the same way `codegraph stats` counts files
(kind = 'file') whenever a post-pass wrote data, mirroring the existing
node/edge re-count fix for the identical under-report shape (#1452).

docs check acknowledged

Impact: 1 functions changed, 4 affected
@greptile-apps

greptile-apps Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR aligns native file discovery with the JS/WASM collector by retaining dotfile-named source files while continuing to exclude hidden directories. It also reconciles the native completion summary’s file count with persisted file nodes after post-pass writes.

  • Disables the blanket hidden-entry filter and adds focused Rust coverage for dotfiles and hidden directories.
  • Recounts persisted file nodes after native post-pass writes.
  • Adds native integration coverage for hidden HCL and CommonJS source files.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness or security issues identified.

The updated walker preserves hidden-directory exclusion while collecting supported dotfiles, and the reconciled file count remains accurate because no later native pass mutates file nodes.

Important Files Changed

Filename Overview
crates/codegraph-core/src/domain/graph/builder/stages/collect_files.rs Native collection now permits dotfile-named files while its existing entry filter continues to prune real hidden directories; focused unit tests cover both behaviors.
src/domain/graph/builder/stages/native-orchestrator.ts Post-pass reconciliation now derives the displayed file count from persisted file nodes whenever post-passes wrote graph data.
tests/integration/issue-2391-hidden-dotfile-native-drop.test.ts Integration tests verify native parsing of hidden HCL and CommonJS files and preserve hidden-directory exclusion.

Reviews (1): Last reviewed commit: "fix(native): stop dropping dotfile-named..." | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown
Contributor

Codegraph Impact Analysis

1 functions changed4 callers affected across 4 files

  • runPostNativePasses in src/domain/graph/builder/stages/native-orchestrator.ts:2417 (4 transitive callers)

@carlos-alm
carlos-alm merged commit c7c1331 into main Aug 16, 2026
50 of 52 checks passed
@carlos-alm
carlos-alm deleted the fix/issue-2391 branch August 16, 2026 02:16
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(native): Rust extractor drops .terraform.lock.hcl and .cjs, self-reports a likely bug, falls back to WASM

1 participant