Expose opt-in GitHub scans through the Python MCP server

docs/remote-scan.md

@@ -1,7 +1,6 @@
 # Remote scan: `cdg scan` on a GitHub repository
 
-Status: implemented in the Python build (`src/codigest/core/remote.py`); the Rust build and MCP
-are not. Written against `c1c4d11` (0.12.2) as the plan for the pull requests that follow it, and
+Status: implemented in the Python CLI and opt-in MCP `scan_github`; the Rust build is not. Written against `c1c4d11` (0.12.2) as the plan for the pull requests that follow it, and
 corrected where the implementation turned out different.
 
 ## Goal
@@ -319,20 +318,16 @@ complete would be worse than none.
 
 ## MCP
 
-The `scan` tool and the output of `tools/list` stay exactly as they are by default. Remote scan
-is exposed only when the server is started with `cdg mcp --allow-remote`:
+The local `scan` tool and the default `tools/list` remain unchanged. In the Python build,
+`cdg mcp --allow-remote` adds a separate `scan_github` tool taking `repository`, optional
+`ref`, `paths` and `format`. An unenabled call is refused. Authentication is controlled
+only by the operator's additional `--auth` startup flag, not tool arguments. GitHub content
+is untrusted input, not agent instructions. The CLI's download bounds and secret masking
+apply unchanged, and each remote CLI process has a 110-second timeout.
 
-- the `scan` tool then also takes `github` (`OWNER/REPO` or URL) and `ref`; `targets` names
-  paths inside the repository, as `--path` does;
-- the tool description says that the content is from a third-party repository and may contain
-  instructions that are not the user's;
-- the server passes `--auth` only when it was itself started with `--allow-remote --auth`; an
-  agent cannot ask for a token;
-- `-y` is passed as for local scans, so the hard ceiling above is what bounds an agent's
-  request.
-
-Without `--allow-remote`, a `scan` call with `github` set is refused with
-`remote scan is off; start the server with cdg mcp --allow-remote`.
+This source change is not in the Rust build. Tests cover tool discovery, denied remote
+access, unchanged local path boundaries, argument validation, credential opt-in, and an
+actual stdio-to-CLI-to-fake-GitHub scan. See `site/chatgpt-mcp.md` for private connectivity.
 
 ## Tests
 

site/chatgpt-mcp.md

@@ -0,0 +1,60 @@
+# Private ChatGPT connection
+
+Codigest runs on the machine that owns the project. Installing it in a different ChatGPT
+execution environment does not grant access to your computer. Public directory submission is
+not needed for a private custom MCP connection.
+
+## 1. Install and test
+
+Use a Python release containing `scan_github`, or install the reviewed source branch locally:
+
+```powershell
+uv tool install --python 3.15 --editable .
+cd PATH_TO_A_SMALL_TEST_PROJECT
+codigest mcp --allow-remote
+```
+
+The command waits for newline-delimited MCP requests; stop it with Ctrl+C after checking it
+starts. Keep the test project small. This server also exposes local tools; remote mode does
+not disable local access. Keep the same account, working directory and absolute executable
+path when running it under a tunnel or scheduled task.
+
+## 2. Create a private tunnel
+
+Follow https://developers.openai.com/api/docs/guides/secure-mcp-tunnels . In
+https://platform.openai.com/settings/organization/tunnels create a tunnel and associate the
+intended ChatGPT workspace. Creating needs Tunnels Read + Manage; using needs Read + Use.
+Custom MCP availability and tunnel permissions are separate. Check account availability and
+billing; this guide does not promise a free service.
+
+Download `tunnel-client` from that settings page. Set `CONTROL_PLANE_API_KEY` locally using
+your secure environment or secret manager; never put its value in chat, a command argument,
+this repository, or a saved profile. Then, from the test project:
+
+```powershell
+tunnel-client help quickstart
+tunnel-client init --sample sample_mcp_stdio_local --profile codigest --tunnel-id YOUR_TUNNEL_ID --mcp-command "codigest mcp --allow-remote"
+tunnel-client doctor --profile codigest --explain
+tunnel-client run --profile codigest
+```
+
+If command lookup fails, use the absolute executable path reported by `Get-Command codigest`.
+The run process must stay alive. Verify its working directory with the `tree` tool before
+reading files; do not assume that the profile preserves your shell directory after restart.
+
+## 3. Connect and verify
+
+At https://chatgpt.com/plugins choose **+ → Add custom MCP server → Connection: Tunnel**,
+select the same tunnel and create/install the private plugin. If unavailable, stop and check
+account/workspace access rather than replacing this with an unauthenticated public endpoint.
+
+Open a new chat and select the plugin. Check:
+
+1. `tree` returns the intended test project, not a home directory or repository root by accident.
+2. `scan` returns a harmless fixture file you created.
+3. `scan_github` reads a small public repository/path and reports its resolved commit.
+4. Private access is unavailable unless the operator restarted with `--allow-remote --auth`.
+
+Only an observed successful tool call establishes end-to-end ChatGPT access. Local tests and
+server startup alone do not. Refresh the plugin connection after changing tools or schemas.
+The tunnel supplies connectivity, not autonomous background execution or access to Tailscale.

site/commands.md

@@ -414,7 +414,13 @@ Serve these tools to AI agents over MCP (stdio).
 cdg mcp
 ```
 
-It takes no options and speaks newline-delimited JSON-RPC on stdin and stdout. See [MCP Integration](/mcp) for registering it and for the tools it offers.
+It speaks newline-delimited JSON-RPC on stdin and stdout. See [MCP Integration](/mcp) for registering it and for the tools it offers.
+
+#### Python-only: `--allow-remote`, `--auth`
+
+The Python build can add `scan_github` with `--allow-remote`. Add `--auth` only to
+allow the server to use existing GitHub credentials. The Rust build does not yet
+support these options. See [MCP Integration](/mcp).
 
 ## `cdg status`
 

site/ko/commands.md

@@ -414,7 +414,13 @@ cdg term --context    # PC 이름과 셸 정보 추가
 cdg mcp
 ```
 
-옵션은 없으며, stdin과 stdout으로 줄 단위 JSON-RPC를 주고받습니다. 등록 방법과 제공하는 도구는 [MCP 연동](/ko/mcp)을 참고하세요.
+stdin과 stdout으로 줄 단위 JSON-RPC를 주고받습니다. 등록 방법과 제공하는 도구는 [MCP 연동](/ko/mcp)을 참고하세요.
+
+#### Python-only: `--allow-remote`, `--auth`
+
+The Python build can add `scan_github` with `--allow-remote`. Add `--auth` only to
+allow the server to use existing GitHub credentials. The Rust build does not yet
+support these options. See [MCP Integration](/mcp).
 
 ## `cdg status`
 

site/mcp.md

@@ -73,3 +73,28 @@ printf '%s\n' \
 ```
 
 The second response lists, in this order: `terminal_history`, `pack`, `scan`, `diff`, `semdiff`, `digest`, `tree`.
+
+
+## GitHub repositories (Python build)
+
+Start the Python server with explicit remote access to expose `scan_github`:
+
+```sh
+uvx --python 3.15 codigest mcp --allow-remote
+```
+
+This is unreleased source functionality until a release containing this change is published.
+The default `cdg mcp` tool list and local project boundary are unchanged. The Rust build does
+not yet implement this option. A GitHub scan never executes the repository's code.
+
+`scan_github` accepts `repository` (OWNER/REPO or a github.com URL), optional `ref`,
+`paths` (a list, including FILE:START-END ranges), and `format` (xml, markdown, json).
+It returns the CLI snapshot with the resolved commit and omitted files. Remote digest, tree,
+diff and pack are not provided. Each remote child process has a 110-second timeout.
+
+Public scans do not use GitHub credentials. To let the server use the operator's existing
+GitHub credentials for private repositories, start with both `--allow-remote --auth`.
+A tool caller cannot enable authentication or change that startup policy.
+
+For ChatGPT cloud, connect this stdio command through Secure MCP Tunnel; starting the command
+alone does not register a ChatGPT plugin. See [private ChatGPT setup](chatgpt-mcp).

site/scripts/check_reference.py

@@ -55,24 +55,31 @@ def walk(command: click.Command, path: list[str]) -> None:
     return found
 
 
-def documented_options(page: Path) -> tuple[dict[str, set[str]], dict[str, set[str]]]:
+def documented_options(
+    page: Path,
+) -> tuple[dict[str, set[str]], dict[str, set[str]], dict[str, set[str]]]:
     """Options in each command's table, and options under its binary-only headings."""
     table: dict[str, set[str]] = {}
     binary: dict[str, set[str]] = {}
+    python_only: dict[str, set[str]] = {}
     current: str | None = None
     for line in page.read_text(encoding="utf-8").splitlines():
         if m := HEADING.match(line):
             current = m[1].strip()
             table.setdefault(current, set())
             binary.setdefault(current, set())
+            python_only.setdefault(current, set())
+        elif current and line.startswith("#### Python-only:"):
+            for span in re.findall(r"`([^`]+)`", line):
+                python_only[current].update(OPTION.findall(span))
         elif current and BINARY_ONLY.match(line):
             for span in re.findall(r"`([^`]+)`", line):
                 binary[current].update(OPTION.findall(span))
         elif current and line.startswith("|"):
             first_cell = line.split("|")[1]
             for span in re.findall(r"`([^`]+)`", first_cell):
                 table[current].update(OPTION.findall(span))
-    return table, binary
+    return table, binary, python_only
 
 
 def compare(
@@ -117,9 +124,11 @@ def main() -> int:
     )
     problems: list[str] = []
     for page in PAGES:
-        table, binary = documented_options(page)
+        table, binary, python_only = documented_options(page)
         label = page.relative_to(SITE).as_posix()
-        problems += compare(label, "Python build", table, python)
+        problems += compare(
+            label, "Python build", {k: v | python_only[k] for k, v in table.items()}, python
+        )
         for command, options in binary.items():
             for option in sorted(options & python.get(command, set())):
                 problems.append(

src/codigest/commands/mcp.py

@@ -16,6 +16,8 @@
 import json
 import subprocess
 import sys
+
+import typer
 from pathlib import Path
 from importlib.metadata import PackageNotFoundError, version
 from typing import Any
@@ -136,6 +138,75 @@
 ]
 
 
+REMOTE_TOOL: JSON = {
+    "name": "scan_github",
+    "description": (
+        "Read a GitHub repository or selected paths without cloning. Returns a snapshot "
+        "with the resolved commit and omitted files. Public repositories only unless the "
+        "server operator explicitly enabled authentication. Never executes repository code."
+    ),
+    "annotations": {"readOnlyHint": True, "openWorldHint": True},
+    "inputSchema": {
+        "type": "object",
+        "required": ["repository"],
+        "additionalProperties": False,
+        "properties": {
+            "repository": {"type": "string", "description": "OWNER/REPO or github.com URL"},
+            "ref": {"type": "string", "description": "Branch, tag or commit"},
+            "paths": {"type": "array", "items": {"type": "string"}},
+            "format": {"type": "string", "enum": ["xml", "markdown", "json"]},
+        },
+    },
+}
+
+
+def _scan_github(args: JSON, *, auth: bool = False) -> str:
+    from ..core import remote
+
+    unknown = set(args) - {"repository", "ref", "paths", "format"}
+    if unknown:
+        raise ValueError("Unknown scan_github arguments: " + ", ".join(sorted(unknown)))
+    repository = args.get("repository")
+    if not isinstance(repository, str) or not repository:
+        raise ValueError("repository must be OWNER/REPO or a github.com URL")
+    # Validate before starting the CLI; URLs cannot choose arbitrary hosts.
+    remote.parse_target(repository)
+    paths = args.get("paths", [])
+    if not isinstance(paths, list) or any(not isinstance(p, str) for p in paths):
+        raise ValueError("paths must be a list of repository paths")
+    for path in paths:
+        remote.repo_target(path)
+    output_format = args.get("format", "xml")
+    if output_format not in {"xml", "markdown", "json"}:
+        raise ValueError("format must be xml, markdown or json")
+    ref = args.get("ref")
+    if ref is not None and (not isinstance(ref, str) or not ref):
+        raise ValueError("ref must be a non-empty string")
+    argv = [
+        sys.executable,
+        "-m",
+        "codigest",
+        "scan",
+        repository,
+        "--github",
+        "-y",
+        "-s",
+        "--no-copy",
+        "--format=" + output_format,
+    ]
+    if ref is not None:
+        argv.append("--ref=" + ref)
+    argv.extend("--path=" + path for path in paths)
+    if auth:
+        argv.append("--auth")
+    result = subprocess.run(
+        argv, capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=110
+    )
+    if result.returncode:
+        raise RuntimeError(result.stderr.strip() or result.stdout.strip())
+    return result.stdout
+
+
 def _terminal_history(args: JSON) -> str:
     count = int(args.get("count", 1))
     if not (entries := termstore.load_entries()):
@@ -194,9 +265,13 @@ def _cli(command: str, args: JSON) -> str:
     return result.stdout
 
 
-def _call_tool(name: str, args: JSON) -> JSON:
+def _call_tool(name: str, args: JSON, *, allow_remote: bool = False, auth: bool = False) -> JSON:
     try:
         match name:
+            case "scan_github":
+                if not allow_remote:
+                    raise ValueError("remote scan is off; start with cdg mcp --allow-remote")
+                text = _scan_github(args, auth=auth)
             case "terminal_history":
                 text = _terminal_history(args)
             case "pack" | "scan" | "diff" | "semdiff" | "digest" | "tree":
@@ -215,7 +290,7 @@ def _server_version() -> str:
         return "dev"
 
 
-def handle_message(msg: JSON) -> JSON | None:
+def handle_message(msg: JSON, *, allow_remote: bool = False, auth: bool = False) -> JSON | None:
     """Returns the JSON-RPC response for a request, or None for notifications."""
     msg_id = msg.get("id")
     params = msg.get("params") or {}
@@ -228,9 +303,14 @@ def handle_message(msg: JSON) -> JSON | None:
                 "serverInfo": {"name": "codigest", "version": _server_version()},
             }
         case "tools/list":
-            result = {"tools": TOOLS}
+            result = {"tools": TOOLS + ([REMOTE_TOOL] if allow_remote else [])}
         case "tools/call":
-            result = _call_tool(params.get("name", ""), params.get("arguments") or {})
+            result = _call_tool(
+                params.get("name", ""),
+                params.get("arguments") or {},
+                allow_remote=allow_remote,
+                auth=auth,
+            )
         case "ping":
             result = {}
         case _ if msg_id is None:
@@ -245,8 +325,15 @@ def handle_message(msg: JSON) -> JSON | None:
     return None if msg_id is None else {"jsonrpc": "2.0", "id": msg_id, "result": result}
 
 
-def handle() -> None:
+def handle(
+    allow_remote: bool = typer.Option(False, "--allow-remote", help="Enable GitHub scans"),
+    auth: bool = typer.Option(
+        False, "--auth", help="Allow remote scans to use local GitHub credentials"
+    ),
+) -> None:
     """Serve these tools to AI agents over MCP (stdio)."""
+    if auth and not allow_remote:
+        raise typer.BadParameter("--auth requires --allow-remote")
     stdin = open(sys.stdin.fileno(), encoding="utf-8", closefd=False)
     stdout = open(sys.stdout.fileno(), "w", encoding="utf-8", newline="\n", closefd=False)
     for line in stdin:
@@ -261,7 +348,7 @@ def handle() -> None:
                 "error": {"code": -32700, "message": "Parse error"},
             }
         else:
-            response = handle_message(msg)
+            response = handle_message(msg, allow_remote=allow_remote, auth=auth)
         if response is not None:
             stdout.write(json.dumps(response, ensure_ascii=False) + "\n")
             stdout.flush()