tool/pputil: robust LoadIncludes parsing

[fennoai]

Summary

Enhances LoadIncludes in tool/pputil per #884. The previous implementation searched for the literal #include substring anywhere in the file, which mishandled several cases. This rewrites it as a single-pass scanner.

Changes

• Spaces/tabs after #: # include, #\tinclude, etc. are now recognized.
• Directive position: # must be the first non-blank character on a line (matching C/C++ preprocessor rules), so #include appearing mid-expression is no longer matched.
• Comments ignored: #include inside // ... line comments and /* ... */ block comments (including multi-line) is skipped.
• String/char literals ignored: #include inside "..." and '...' literals is skipped, honoring \ escapes.
• Removed the reliance on the mutated package-level include byte slice; parsing no longer mutates shared state.

The scan is a single O(n) pass over the file bytes with no extra allocations beyond the yielded filenames.

Tests

Added tool/pputil/pputil_test.go covering spaced directives, leading whitespace, line/block comments, string/char literals, non-line-start #, missing separators, substrings like #includes/#include_next, unterminated names, EOF without newline, early iterator stop, and LoadIncludes file I/O (success and error).

All tests pass; go vet, gofmt, and go build are clean.

Fixes #884

[codecov]

Codecov Report

❌ Patch coverage is 96.55172% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.69%. Comparing base (ea6aa52) to head (0b1152e).

Files with missing lines	Patch %	Lines
tool/pputil/pputil.go	96.55%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #885      +/-   ##
==========================================
+ Coverage   90.39%   90.69%   +0.29%     
==========================================
  Files          19       19              
  Lines        1791     1827      +36     
==========================================
+ Hits         1619     1657      +38     
+ Misses        172      170       -2     

Flag	Coverage Δ
llgo-tests	90.69% <96.55%> (+0.29%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fennoai]

Review: single-pass #include scanner

The rewrite of the #include scanner into a single-pass byte tokenizer (scanIncludes / skipLiteral / parseInclude) is a solid improvement over the old repeated-bytes.Index approach. It is O(n), idiomatic, well-documented, and the new pputil_test.go covers a wide range of cases (whitespace variants, comments, string/char literals, keyword substrings like #includes/#include_next, early-stop, EOF-without-newline, unterminated header name). Tests pass and gofmt/go vet are clean.

I verified the security concern around manual byte indexing: all accesses are correctly bounds-checked. Unterminated block comments and a trailing backslash in a literal produce an i that overshoots len(b), but that value is only ever consumed by the outer i < len(b) loop guard, so there is no out-of-bounds risk.

A couple of minor points below. None are blocking.

Other notes (no reliable inline location):

• Unterminated block comment robustness (scanIncludes, ~pputil.go:137-143): after an unterminated /* ... EOF, the loop exits with i == len(b) and then i += 2 sets i = len(b)+2. This is safe today (only the outer loop guard reads it), but making it explicit — advancing past */ only when the closing delimiter was actually found, else i = len(b) — would be more robust and self-documenting. Optional, and a test for the unterminated-comment case would be nice.
• Line-continuation splicing not performed: an #include broken across lines with a trailing backslash (backslash-newline splicing, which the C preprocessor performs before tokenizing) is not handled. This is virtually never seen in real headers, so leaving it unsupported is reasonable — a one-line doc note in scanIncludes would set expectations.

[fennoai]

One additional inline note (see the prior review for the full summary).

[xushiwei]

@fennoai fix review comments

[fennoai]

Review comments addressed ✅

Pushed 0b1152e to fennoai/issue-884-1790785273.

[P2] Same-line block comment suppresses next #include
Block comments are now treated as whitespace: the unconditional atLineStart = false after a block comment was removed, so /* c */ #include <x.h> is correctly recognized. Code appearing before a same-line block comment still correctly suppresses the directive.

[P2] Inaccurate doc claim
Reworded the scanIncludes doc — dropped "as required by the C/C++ preprocessor" and now states that comments are treated as whitespace (matching the fixed behavior).

[P3] close shadows the predeclared builtin
Renamed the local variable close → closer in parseInclude.

Tests
Added two scanIncludes cases: #include following a same-line block comment (recognized), and code + same-line block comment + #include (ignored).

Verification

• go test ./tool/pputil/ — pass
• go vet ./tool/pputil/ — clean
• gofmt -l tool/pputil/ — no changes
• go build ./tool/pputil/ — clean