Skip to content

fix(bot): scan precision - quoted secrets only, low-risk paths to nit - #18

Merged
samson910022 merged 1 commit into
mainfrom
fix/ai-scan-precision
Sep 9, 2026
Merged

samson910022 merged 1 commit into
mainfrom
fix/ai-scan-precision

Conversation

@samson910022

Copy link
Copy Markdown
Owner

First live ai-scan dry_run produced 3 false positives (bare code assignment, unquoted yml env line, test fixture). Two precision tunings: (1) secret patterns require quoted assignments (bare code and unquoted env lines no longer match; committed .env still caught by forbidden-file check); (2) secret-pattern hits under tests/docs/yml/md downgrade to nit so real findings sort first (fingerprints unchanged, dedupe unaffected). 38 python tests green.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Verdict matrix

  • Roles: gateway_safety, api_compat, docs_config
  • FINAL_VERDICT: APPROVE

Sensitive paths

  • github_bot/src/repo_scan.py

Deterministic pre-checks

  • possible-password: github_bot/src/repo_scan.py,tests/test_ai_bot.py:62
  • private-key-material: github_bot/src/repo_scan.py,tests/test_ai_bot.py:64
  • possible-password: github_bot/src/repo_scan.py,tests/test_ai_bot.py:71

gateway_safety

VERDICT: APPROVE

No BLOCKING findings (no secret leaks, no auth bypasses, no broken wire shapes, no regressions to [provider] invariants).
No SHOULD_FIX findings.

NITS:

  • github_bot/src/repo_scan.py:19: Original unquoted secret patterns are commented rather than deleted; minor audit-trail vs. cleanliness tradeoff.
  • tests/test_ai_bot.py:327: import re + from repo_scan import SECRET_PATTERNS moved inside test method (acceptable but non-idiomatic; other tests import at top).
  • tests/test_ai_bot.py:340: test_prior_dry_run_fps_resolved calls live collect_findings() (full worktree scan); acceptable for regression verification but could be mocked for pure unit-test purity.

All changes preserve recall for real leaks (quoted pasted secrets + separate forbidden-file check for .env etc.), downgrade only noise under tests/ / *.yml/.md/.github/*yml, keep private-key pattern untouched, and pass the stated 38 Python tests. No [provider] safety invariants violated.

api_compat

API Compatibility Review

This PR modifies only the GitHub bot's repository scanner (github_bot/src/repo_scan.py) and its tests (tests/test_ai_bot.py). It does not touch any [provider] runtime code (src/proxy.ts, src/converters/*, src/config/*, route handlers, collectors, retry logic, session management, or documentation).


Checklist evaluation

# Area Impact
1 Route error wire shapes Not touched — no changes to chat/responses/messages/interactions handlers
2 Converter thinking/signature preservation Not touched
3 previous_response_id / session chaining Not touched
4 Reasoning mapping / budget thresholds Not touched
5 Model alias rules Not touched
6 Collector finish==='tool' / flushHeaders Not touched
7 resolveMaxRetries / retry backoff Not touched
8 New endpoints / docs Not applicable

No [provider] code is affected; all eight compatibility invariants are preserved.


Deterministic pre-findings triage

The three pre-findings (possible-password ×2, private-key-material ×1) reference lines inside the scanner's SECRET_PATTERNS list and test assertions. These are regex pattern strings and test fixtures, not actual secrets. The diff evidence is [REDACTED], and the visible diff confirms the lines are pattern definitions and re.search calls against synthetic test strings. No real credential material is present.


Findings

BLOCKING: (none)

SHOULD_FIX:

  1. github_bot/src/repo_scan.py:38 — is_low_risk_path uses str.startswith(tuple) correctly, but the suffix check rel.endswith(LOW_RISK_SUFFIXES) will also match paths like src/config/proxy.yaml or a hypothetical src/secrets.yml. If the intent is strictly fixture/doc noise, consider combining the directory and suffix checks (e.g., only downgrade suffix matches when also under a low-risk directory). Current behavior is defensible but slightly broader than described.

  2. tests/test_ai_bot.py:361–377 — test_prior_dry_run_fps_resolved calls collect_findings() which internally runs scan_worktree against the real working tree. This makes the test environment-dependent (it will pass only when the repo is checked out and the working tree matches expectations). If CI runs from a shallow clone or a different worktree state, the assertions on specific files may become flaky. Consider mocking scan_worktree or using a temp directory fixture.

NITS:

  1. github_bot/src/repo_scan.py:28–29 — The {3,} minimum in the quoted-value capture ([^'\"]{3,}) means a 1- or 2-character quoted secret like password: "ab" won't match. This is likely intentional to reduce noise, but worth a brief inline comment noting the minimum-length choice.

  2. tests/test_ai_bot.py:335 — Test strings use [REDACTED] as placeholder secret values. These are fine for pattern matching, but using a more obviously synthetic value (e.g., sk-test1234abcd) would make the test intent clearer at a glance.


Test gap note

No cross-protocol regression risk — this PR is entirely within the bot/scanner subsystem. The 38 Python tests cover the changed logic. No TypeScript [provider] tests are affected or need updating.


VERDICT: APPROVE

The changes are scoped entirely to the bot scanner's precision tuning and do not touch any [provider] runtime, wire shapes, or API compatibility surfaces. The should-fix items are minor robustness concerns, not blockers.

docs_config

VERDICT: APPROVE

BLOCKING: None

  • No new/changed [provider] env: six-way matrix untouched.

SHOULD_FIX: None

NITS: None

Matrix gap closure — files that must change: none.

Checked:

  • github_bot/src/repo_scan.py:22-35 — quoted-only secret regex + LOW_RISK_DIRS/SUFFIXES + is_low_risk_path is bot-internal, not [provider] config. No sync required in .env.example, config.json.example, Dockerfile, docker-compose.yml, index.ts, docs/configuration.md, AGENTS.md.
  • tests/test_ai_bot.py:330-369 — bot unit tests only. No npm run lint reference, no test:unit rename, no README/CHANGELOG drift.
  • No README.md / README.zh-CN.md / docs/README.md changes needed; no link to git-ignored docs/host-deployments.md added.

FINAL_VERDICT: APPROVE

Session: response_id=1528ca6b-1bef-9426-a795-8dc61486fb5d,resp_req_vrtx_011CetUCLcG41LjCJxJawAtv,chatcmpl-f5bf5235-cf69-4aaf-bd96-f3b5904cbd7a (chain via previous_response_id, 30min TTL). model=[model] provider=[provider].

@samson910022
samson910022 merged commit fab47e2 into main Sep 9, 2026
5 checks passed
@samson910022
samson910022 deleted the fix/ai-scan-precision branch September 10, 2026 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant