[NaNIsaac]

Tier: physics (src change)

• New: cdg scan OWNER/REPO --github, a github.com URL, --ref, --path, --auth, --max-file-size in the Python build. Remote scans make GET requests to api.github.com and raw.githubusercontent.com; local scans make none, as before.
• Local scan output is unchanged byte for byte: the new source/ref/commit attributes, the Markdown Source: line and the JSON source object appear only for remote scans.
• Not in this PR: the Rust build (waits for a decision on keeping Python as the reference), MCP --allow-remote, README and site (one issue after implementation).

Stacked on #30 (design). Implements it for the Python build.

How it runs (src/codigest/core/remote.py): the ref is resolved to one commit SHA, the recursive tree listing is filtered with the local scanner's rules (each ancestor folder, then the file; ALWAYS_IGNORE plus the root .gitignore; named paths walked one by one and joined), secret files and files over --max-file-size are dropped before any download, the size question runs on the listed sizes, and only then the selected files are downloaded into a temporary folder. The unchanged local scan reads that folder with a fixed root (ProjectContext(root=...)), and the folder is removed afterwards. Without --auth no token is sent even if GH_TOKEN is set; with it, files come from the blob API so the token goes only to the API host.

Changes outside the new module: scan.py (options, the pre-flight check and rendering split into _confirm_size and _render so the remote path reuses them, a --github hint when a missing local path looks like OWNER/REPO), common.py (fixed root), scanner.py (is_ignored_rel for paths not on disk), prompts.py (optional source, ref, commit attributes).

Tests (tests/test_remote_scan.py, fake GitHub in tests/fake_github.py on two local ports; no network):

• remote output equals the local scan of the same folder, for the whole repository and for --path sets (folders, files, line ranges, an ignored named file, a withheld named file, overlapping paths, a non-ASCII path), including the withheld list;
• ignored, withheld, oversized, .codigest/ and out-of-scope files are never requested; the repository's prompts.toml and config are not applied;
• only GET, only the two hosts; no token without --auth; with --auth the token is on API requests only and never in the output; token refused over plain http to a non-loopback host; a redirect to a third host is refused;
• 404 private / wrong token / 401 / rate limit with reset time / short and long Retry-After / 5xx retried twice / unreachable host / failed download with no partial output / moved repository / unknown ref / renamed default branch / truncated listing with and without --path / symlink, submodule and ../ entries / the size question before any download / the hard ceiling with -y;
• target parsing and usage errors (exit 2, no request made).

Results (Windows 11, Python 3.15.0b4):

• uv run --with pytest pytest -q: 237 passed in 136 s (before the last three fixes below); after them, test_remote_scan.py plus the scan, format, preset and agent-use tests: 107 passed.
• uvx ruff@0.17.1 check and format --check (the CI versions): clean. conventions.py check: 0 findings.
• By hand against github.com (read-only, public repository): cdg scan NaNIsaac/codigest --github --path LICENSE --path src/codigest/core/tokenizer.py:1-5 -s, the URL form with /tree/main/src/codigest/core --format markdown, a private repository without --auth (404 message) and an unknown ref. The opt-in network test from the design is not written yet.

Fixed while testing (before this PR, no issue): a --path naming an ignored file was scanned as the whole folder; overlapping --path values used only the first; the repository name from the API went into URLs unchecked.

Differences from the design, recorded in the doc: one 30 s socket timeout instead of separate connect/read timeouts; a file named with a line range counts with its whole size in the size question; fixtures are written by the tests instead of committed under tests/fixtures/github/.

[NaNIsaac]

Added the opt-in network test from the design (CODIGEST_NETWORK=1): NaNIsaac/codigest at v0.9.0, --path src/codigest/core, against git archive of the same tag scanned locally. Ran once locally: 1 passed (3.6 s). Without the variable it is skipped; the rest of the file: 67 passed, 1 skipped.

[NaNIsaac]

Cross-review (rule 22), correctness and acceptance only, at 688b2fc. Agree / disagree / not reproduced per point.

1. Remote output equals a local scan through the same rules: agree, one disagreement below.

• The pre-download filter is the local scanner itself: _select builds scanner.ProjectScanner(root) on a folder holding only the root .gitignore (remote.py:441-445), so ALWAYS_IGNORE + root .gitignore come from the same _load_gitignore (scanner.py:39-58), and _kept asks is_ignored_rel for each ancestor folder with is_dir=True and then for the file (remote.py:413-422), which is what the local walk does through is_ignored (scanner.py:60-72, 102-104). The local scanner reads no nested .gitignore, so root-only is the same rule. is_secret_file is the same function on the basename (remote.py:462).
• The output itself is rendered by the unchanged local path on the downloaded folder (common.get_context(targets, root=root), scan.py _scan_remote), so the parity tests can only fail when the pre-filter drops too much; downloading too much is caught separately by test_files_that_are_not_shared_are_never_downloaded and ..._outside_the_selection_.... Together they cover both directions.
• --max-file-size is not a local rule (its help says "GitHub:"; the local scan has no such limit), so no parity test exercises it: every parity test runs with the default 1MB, which no fixture file reaches. It is checked only as "not downloaded, listed as skipped". That matches the design, but the PR text lists it among the shared rules; it is a remote-only rule.
• Disagree (bug): with a truncated listing, a --path . next to another --path drops the root files without an error. _list walks each scope by its parts; for scope "" the walk finds no entry named "" and breaks (remote.py:384-391), so only the other scopes' files are listed. Reproduced with my own stand-in server (not tests/fake_github.py), three files README.md, top.py, src/a.py: --path . --path src gives exit 0 and only src/a.py when the listing is truncated, all three when it is not. --path . alone is refused correctly ("too large to list at once"). Suggested fix: treat a "" scope in the truncated branch as the whole repository and refuse it the same way (if "" in scopes: instead of if not any(scopes):), with a test.

2. Read-only: agree, with the limits of what the tests can see.

• GET only: Client.get builds every request with method="GET" (remote.py:240-242), and the fake records any other method as 405; test_only_get_requests_to_the_two_hosts asserts the method set.
• Two hosts: every URL is built from client.api or client.raw (remote.py:312, 331, 377, 382, 394, 486, 489), and redirects to any other host raise (remote.py:208-210, test_a_redirect_to_another_host_is_refused). The "only two hosts" test can only see requests that reached the fake, so it would not notice a request to a third host; the protection is the code above, not that test.
• No git: no git call in the remote path; the only subprocess is gh auth token, and only with --auth (remote.py:173-186).
• Writes: only under tempfile (the cdg-rules- folder for the .gitignore rules, the cdg-remote- folder for downloads; _write refuses paths that resolve outside, remote.py:519-525); the content cache and .codigest/ marker land inside the temporary root. The fixture asserts the temp and working folders are empty afterwards; it does not watch the home folder, which nothing in this path writes to (read, not tested).

3. No token without --auth: agree, reproduced independently. Stand-in server on two loopback ports logging the Authorization header of every request, real CLI in a subprocess (uv run cdg scan octo/probe --github -y -s), GH_TOKEN set:

• without --auth: 6 requests (3 API, 3 raw), none with Authorization;
• with --auth: 6 requests, all to the API host (blobs by hash), all with the token, none to the raw host;
• the token string appears in neither stdout nor stderr in either run.

4. Large repositories, truncation, rate limits: agree except the bug in 1.

• Truncated listing without --path is refused; with --path it walks scope by scope and refuses a scope whose own listing is truncated (remote.py:374-403). Tested, apart from the "" scope above.
• The hard ceiling (5,000 files / 50 MB) is checked on listed sizes before any download, also with -y (scan.py _scan_remote, test_a_hard_ceiling_applies_even_with_yes).
• Rate limits: x-ratelimit-remaining: 0 is never retried and reports the reset time; Retry-After up to 10 s is waited once per attempt, longer is reported; 5xx and unreachable hosts are retried twice with 1 s, 2 s (remote.py:269-279; tests cover each). With --auth, files come from the blob API, so a scan costs one API call per file against the 5,000/h limit; the 5,000-file ceiling keeps one scan inside it, but two large scans in an hour will hit it (not a defect, worth one line in the docs).

Not reproduced: the full 237-test run and the by-hand runs against github.com (I ran only the stand-in probe above).

[NaNIsaac]

Review answered (thanks, phase-gpu):

• Root files dropped on a truncated listing (reproduced): fixed in b0ba6cb. A failing test came first (test_a_truncated_listing_cannot_give_the_root_by_its_name, for --path ., ./ and "" next to --path src: 3 failed before the fix). Now any root scope on a truncated listing stops with "too large to list at once; name folders with --path" and exit 1, as a scan without --path does. No issue opened since the code is not on main; the commit names this review.
• --max-file-size is outside the parity tests: agreed. test_max_file_size_is_a_remote_rule_the_local_scan_does_not_have states it: with --max-file-size 2KB, the over-limit file is skipped, named, never requested, and the output equals the local scan of the folder without that file.

test_remote_scan.py: 71 passed, 1 skipped (the opt-in network test), with GITHUB_ACTIONS=true as in CI. ruff 0.17.1 clean.