Skip to content

modfetch: select the requested module among multiple go: downloading lines - #167

Merged
xushiwei merged 2 commits into
mainfrom
fennoai/fix-modfetch-multimodule-get
Sep 28, 2026
Merged

xushiwei merged 2 commits into
mainfrom
fennoai/fix-modfetch-multimodule-get

Conversation

@fennoai

@fennoai fennoai Bot commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Problem

modfetch.Get(modPath) intermittently returns the wrong module for a submodule of a multi-module repository.

When go get <mod>@<ver> targets a module that lives in a multi-module repository, go also downloads the parent module and prints one go: downloading ... line per module. The order of these lines is not deterministic. getResult previously took the first line unconditionally.

Reproduction for github.com/llarhub/libcxx/c@v0.1.1 (both github.com/llarhub/libcxx and github.com/llarhub/libcxx/c are separate modules):

$ go get github.com/llarhub/libcxx/c@v0.1.1
go: downloading github.com/llarhub/libcxx v0.1.1      # parent, sometimes printed first
go: downloading github.com/llarhub/libcxx/c v0.1.1    # the requested module

When the parent line comes first, getResult returns github.com/llarhub/libcxx (the /c suffix is dropped). Downstream callers such as llcppg then resolve the wrong module-cache include directory, which causes an intermittent failure (panic: todo: toType int64_t in goplus/llcppg#867).

The order was confirmed non-deterministic across repeated clean-cache runs (parent-first on some runs, child-first on others).

Fix

Scan all go: downloading lines and prefer the one whose module path exactly matches the requested module. Fall back to the first line when there is no exact match, preserving the existing single-module behavior. Get passes the version-stripped requested path down to getResult.

Verification

  • New unit tests (getResult) cover both download orderings, the fallback path, and the not-found case.
  • End-to-end: modfetch.Get("github.com/llarhub/libcxx/c@v0.1.1") now returns github.com/llarhub/libcxx/c in both a fresh cache and a parent-already-cached cache.
  • go build ./... and go test ./modfetch/ pass.

Fixes goplus/llcppg#867

…` lines

When `go get <mod>@<ver>` targets a module that lives in a multi-module
repository, `go` downloads the parent module as well and prints one
`go: downloading ...` line per module. Their order is not deterministic.

`getResult` previously returned the first line unconditionally, so it could
return the parent module path (e.g. `github.com/llarhub/libcxx`) instead of
the requested submodule (`github.com/llarhub/libcxx/c`). Downstream callers
such as llcppg then resolved the wrong module cache directory, which failed
intermittently depending on the download order and cache state.

Scan all `go: downloading` lines and prefer the one whose module path matches
the requested module, falling back to the first line to preserve the existing
single-module behavior.

Fixes goplus/llcppg#867
@codecov

codecov Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.23%. Comparing base (f4300a7) to head (8c7bdf5).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #167      +/-   ##
==========================================
- Coverage   81.34%   81.23%   -0.12%     
==========================================
  Files          10       10              
  Lines         906      906              
==========================================
- Hits          737      736       -1     
- Misses        150      151       +1     
  Partials       19       19              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: fix modfetch multi-module Get

The change is a correct, well-targeted fix. It replaces "parse the first go: downloading line" with "prefer the line whose module path exactly matches the requested one, else fall back to the first line" — the right fix for go get's non-deterministic ordering of downloading lines in multi-module repositories. The exact-match comparison avoids the substring/prefix pitfall, and the fallback preserves the prior single-module behavior. Tests cover both orderings, the fallback, empty reqPath, and the not-found path; they pass locally.

A few minor, non-blocking notes below. Nothing here needs to block merge.

  • Silent fallback on path-format mismatch. getResult selects only on exact equality m.Path == reqPath, where reqPath comes from stripping @version off the caller's modPath. If go ever prints the path in a form that isn't byte-identical to reqPath (e.g. a /vN major-version subdir), the match fails and the code silently falls back to the first line — i.e. the parent-module bug this PR fixes, in a narrower case. The eventual getFromCache(modPath) re-validates against the original path, so blast radius is limited, but worth confirming reqPath is guaranteed identical to what go get prints.
  • Dead next parameter on getMod. After this change the sole caller always passes nil, so the if next != nil branch is now unreachable. Consider dropping the parameter (separate cleanup) since getMod is only called here.

Comment thread modfetch/fetch.go Outdated
}
if found {
mod = first
fmt.Fprintln(os.Stderr, "xgo: downloading", mod.Path, mod.Version)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The success fmt.Fprintln(os.Stderr, "xgo: downloading", ...) is duplicated on both the exact-match path (line 234) and this fallback path. Since both set mod and return, you could set mod in the loop/fallback and emit a single Fprintln after the branch, removing the duplication.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fennoai fix this comment

@fennoai fennoai Bot Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Hoisted the fmt.Fprintln call out of both the exact-match and fallback branches into a single call after the if/fallback block.

Changes:

  • Removed the duplicate fmt.Fprintln from line 234 (exact-match path) and line 240 (fallback path)
  • Both paths now set mod and break/fall-through to a single fmt.Fprintln at line 244

Verified:

  • go test ./modfetch/ passes (all 4 test cases)
  • go build ./... passes

Commit: 8c7bdf5

View job run

Comment thread modfetch/fetch_test.go
Remove duplication by setting mod in both the exact-match and fallback
paths, then emitting a single fmt.Fprintln after the branch.
@xushiwei
xushiwei merged commit 3eb26ae into main Sep 28, 2026
7 of 8 checks passed
@fennoai
fennoai Bot deleted the fennoai/fix-modfetch-multimodule-get branch September 28, 2026 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

unit test TestLLVM_AMDGPUAddrSpace/llvm-22.1.8-support/AMDGPUAddrSpace failed occasionally

1 participant