Skip to content

Require authenticated Qdrant, least-privilege workflow tokens, scoped dependency triage - #109

Merged
ViperJuice merged 3 commits into
mainfrom
fix/106-qdrant-auth-actions-permissions
Oct 31, 2026
Merged

ViperJuice merged 3 commits into
mainfrom
fix/106-qdrant-auth-actions-permissions

Conversation

@ViperJuice

@ViperJuice ViperJuice commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Refs #106

This PR prepares the source side of #106 as three separate commits. It changes no repository settings and touches no live deployment, credentials or restarts. Merging it does not repair the running deployment. That needs the owner actions listed at the end.

1. Authenticated Qdrant and explicit client migration (a745397)

Before

  • The repository Compose files (docker-compose.qdrant.yml, docker-compose.yml, docker-compose.mcp.yml, docker/compose/development/docker-compose.development.yml) had QDRANT__SERVICE__API_KEY commented out, so Qdrant accepted anonymous calls.
  • They used qdrant/qdrant:latest. docker-compose.qdrant.yml and the development file published 6333 and 6334 on all interfaces.
  • Clients sent QDRANT_API_KEY only when it was set. Otherwise they connected anonymously.

After

  • Compose: QDRANT__SERVICE__API_KEY=${QDRANT_API_KEY:?...}, so Compose refuses to render without a key.
    • An entrypoint guard exits 64 if the key reaches the container empty. docker-compose.mcp.yml relies on this guard alone, because its Qdrant service is an optional profile.
    • The image is pinned to qdrant/qdrant:v1.17.0@sha256:f1c7272c…, the release that wrote the existing data and the digest the smoke harness already uses.
    • Ports are bound to 127.0.0.1, telemetry is off, and the health check is /readyz (exempt from auth).
  • Clients: every server-mode (http/https) client resolves its key through resolve_qdrant_server_api_key(). Without a key it raises QdrantAuthConfigError before sending any request; preflight reports failure_class: qdrant_api_key_missing. QDRANT_REQUIRE_API_KEY=false is an explicit opt-out for disposable test servers only. File-mode and :memory: clients are unchanged.
  • Client inventory (also in docs/operations/qdrant-authentication.md):
    • SemanticIndexer._init_qdrant_client: gateway, plugin semantic path (now logs a warning instead of a silent debug message), indexer registry, index CLI.
    • Semantic preflight check_qdrant, check_qdrant_collection and collection bootstrap.
    • index_management._get_vector_backend_status: the key is resolved outside its broad except, so a missing key is reported.
    • secure_export.
    • Autostart, through preflight and the Compose file.
    • scripts/utilities/debug_semantic_search.py.
    • The v13_qdrant_smoke.py and v13_pmcp_pilot.py harnesses: disposable servers now get a random key per run, passed to docker by name only. The pilot writes it into its private fixture .mcp.json next to its existing per-run guard token.
  • Not changed, and why:
    • Scheduled and other workflows: none of them talk to a Qdrant server.
    • k8s/deployment.yaml already maps the qdrant-api-key secret.
    • File-mode scripts have no network listener.
    • The frozen benchmark record, tests/root_tests/ and archive/ now get 401 from an authenticated server; they do not bypass it.
  • Docs: docs/operations/qdrant-authentication.md has behavior, inventory, migration, readback and rollback. Env templates, README, TROUBLESHOOTING, onboarding and the docker README are updated. CHANGELOG is unchanged because repository policy keeps [Unreleased] empty.

2. Actions least privilege (a65219c)

  • Seven workflows had no top-level permissions: and inherited the repository default, which is currently read/write.
    • Added contents: read to ci-cd-pipeline, container-registry and index-artifact-management.
    • Added permissions: {} to index-management, lockfile-check, mcp-index and maintenance (echo-only or external webhook jobs).
  • Job-level write scopes remain only where a job publishes, signs, attests, uploads SARIF or mutates PRs and releases. Unchanged: release-automation, sign-published-image, and the container push and sign jobs.
  • The two SARIF upload jobs now declare contents: read next to security-events: write. Their checkout needs it once job-level permissions replace the defaults.
  • New tests/test_workflow_permissions.py fails if a workflow lacks top-level permissions, grants a top-level write, or adds a job write scope that is not on the reviewed allowlist.

3. uv.lock advisory triage (717a97d)

uvx pip-audit 2.10.1 against uv export --locked --all-extras found 397 rows in 36 packages, or 280 distinct advisories after alias deduplication. Each match is classified by dependency set and import reachability. Only two base-install matches meet their advisory prerequisites:

Package Change Advisory Reason
starlette 1.0.0 → 1.0.1 GHSA-86qp-5c8j-p5mr AuthenticationMiddleware decides auth exclusions from request.url.path, which Starlette rebuilds from an unvalidated Host header.
certifi 2026.2.25 → 2026.10.18 GHSA-6q48-6c7w-9gw2 All runtime HTTPS clients trust the bundle.

The lockfile diff is 12 lines and touches only these two packages. The revision header stays at 3, and uv lock --locked and uv sync --locked pass. Re-scan: certifi 0 matches; starlette has 4 remaining matches, none applicable.

Deferred and not-applicable items, each with its reason, are in docs/security/dependency-triage-2026-10-31.md. Notable ones:

  • mcp GHSA-hvrp-rf83-w775: deferred. enable_tasks() is used, but the server runs over STDIO with one session per process. The fix also changes how a custom TaskStore lists tasks.
  • cryptography GHSA-537c-gmf6-5ccf: deferred, runtime unknown. The bundled OpenSSL fix needs a major-version bump.
  • pyjwt: single HS256, no PyJWK.
  • aiohttp and pillow: only the synchronous Voyage client runs, and no images are processed.
  • gitpython: only chunker.repo.processor imports it, and that module is never imported (example ID: GHSA-284h-m62q-gf8w).
  • python-multipart: no form parsing.
  • Dev-only and semantic extras: not in the production image.

Test evidence

  • tests/test_qdrant_api_key.py: 43 passed. Covers fail-closed behavior and key propagation for every client construction, and checks that the key never appears in error messages.
  • tests/integration/test_qdrant_auth_container.py -m integration: 5 passed, run against a real disposable container. It starts the repository docker-compose.qdrant.yml in a uniquely named Compose project on loopback host ports with throwaway storage and a random key.
    • Rendering without a key fails.
    • An empty key makes the container exit 64.
    • Anonymous and wrong-key GET /collections, /telemetry, /metrics and an anonymous PUT of a collection all return 401. /readyz returns 200.
    • Default clients fail closed before connecting. An explicit opt-out client gets 401.
    • An authorized SemanticIndexer client creates, upserts, queries and deletes a synthetic collection.
    • Afterwards no test containers, volumes or networks were left behind, and the live mcp_qdrant container was untouched.
  • Workflow tests (test_workflow_permissions, test_localci_workflow_posture, test_workflow_release_policy, test_workflow_action_pins, test_coverage_workflow_posture): 51 passed.
  • Gateway, security and dependency tests after the starlette and certifi bump: 310 passed.
  • Full unit suite (tests/ --ignore=tests/integration --ignore=tests/test_git_integration.py -n auto), local Python 3.14: 4130 passed, 137 skipped, 3 failed.
    • test_release_notes.py::test_changelog_unreleased_empty failed on a CHANGELOG entry I had added. The entry is removed and the test now passes (4 passed).
    • test_v13_pmcp_pilot.py::test_prepare_delivered_wheel_never_builds[True|False] raise python_312_required. They fail the same way on main because the local venv runs Python 3.14 and the test needs 3.12.
  • Gates:
    • make alpha-format-lint: rc=0 (pylint 8.94/10).
    • make alpha-mypy-ratchet: rc=0 (1351 current errors, baseline 1562).
    • make agent-gate: rc=0 at e643513. That commit has the same tree as the head except for the CHANGELOG entry removed afterwards.
    • The first two agent-gate runs failed only in test_baml_contract.py::test_baml_regeneration_and_formatting_are_reproducible, for environment reasons. First, no BAML 0.20.1 toolchain was installed. I then used the release tarball CI downloads, with the same sha256 check 6067729f…. Second, that toolchain refused to run in an agent session without its "agent skill". It passed with BAML_AGENT_SKILL_CHECK=off, which skips only that tooling check.

Residual risk

  • Breaking for anonymous server-mode users. After merge, any client that uses QDRANT_URL without QDRANT_API_KEY stops semantic search (fail closed). Lexical search keeps working and indexed tools return index_unavailable with native_search. docker compose -f docker-compose.qdrant.yml (including make semantic-up and autostart) now needs QDRANT_API_KEY in the environment.
  • Transport: the shipped configs use plain HTTP on loopback or a private Compose network. Cross-host clients need Qdrant TLS or a tunnel. No read-only key or JWT RBAC is configured, because Code-Index-MCP needs write access.
  • Live dev0 deployment: container mcp_qdrant runs from docker-compose.qdrant.yml plus out-of-repo overrides. Its environment contains a QDRANT__SERVICE__API_KEY variable (names inspected only, values not). The override supplies the key through env_file. After this change the base file's environment entry takes precedence, so any recompose must export QDRANT_API_KEY with the same value. Otherwise Compose refuses to render, which fails closed. Live anonymous denial has not been read back.
  • Two deferred dependency items (mcp tasks and the cryptography bundled OpenSSL) need follow-up. Deployed hosts keep starlette 1.0.0 and the old certifi until they re-sync or rebuild.
  • GITHUB_TOKEN hardening protects only workflows on this branch onward. The repository default stays read/write until the owner changes it.

Rollout

  1. Review, then merge (owner).
  2. Owner: flip the Actions default to read-only (below). Then watch the next ci-cd-pipeline, container-registry and manual release runs for permission errors.
  3. Owner: follow docs/operations/qdrant-authentication.md steps 1 to 7 in a maintenance window: inventory, back up storage, provision the key in the secret manager, update every client environment, restart through the repository Compose file, read back, then restart the clients.
  4. Rebuild or re-sync deployed environments to pick up starlette 1.0.1 and certifi 2026.10.18.

Rollback

  • Code: revert the individual commits. The authenticated server configuration can stay, because older clients send QDRANT_API_KEY whenever it is set.
  • Client auth failures (401 or qdrant_api_key_missing): stay failed closed and fix the key injection.
  • Qdrant refuses to start: fix the key injection and run the restart again. The data has not changed.
  • Data problem: restore the step-2 backup and keep the key required.
  • Never roll back by removing the API key, by setting QDRANT_REQUIRE_API_KEY=false on a production client, or by restoring anonymous access.
  • Workflows: revert a65219c, or add a specific job-level scope if a job turns out to need one.
  • Dependencies: revert 717a97d.

Owner approval required (not done here)

  1. Repository Actions default: change Settings → Actions → General → Workflow permissions from read/write to read-only after merge. It is currently write per gh api repos/Consiliency/Code-Index-MCP/actions/permissions/workflow.
  2. Live Qdrant on dev0: provision or confirm the API key in the secret manager, back up storage, restart through the repository Compose file with QDRANT_API_KEY injected, update client environments, and approve the production readback. The readback checks: anonymous GET /collections returns 401, the authorized client works, collections and point counts are preserved.
  3. Any credential creation or rotation for the above.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

ViperJuice and others added 3 commits October 31, 2026 13:42
…lient key

Server-mode Qdrant (http/https) clients now resolve QDRANT_API_KEY through
resolve_qdrant_server_api_key() and raise QdrantAuthConfigError before any
request when it is missing. QDRANT_REQUIRE_API_KEY=false is an explicit
opt-out for disposable test servers only. Covered: SemanticIndexer (and all
its users), semantic preflight probes and collection bootstrap, index
management status, secure export, the debug utility, and the v13 smoke and
pilot harnesses, which now start disposable servers with per-run keys
passed to docker by name only.

Repository Compose files require QDRANT__SERVICE__API_KEY (render fails
without QDRANT_API_KEY; an entrypoint guard exits 64 on an empty key), pin
qdrant/qdrant:v1.17.0 by digest, bind to loopback, disable telemetry and
use the auth-exempt /readyz health check.

Tests: unit fail-closed/propagation coverage per client, and an
integration test that runs the repository docker-compose.qdrant.yml in a
disposable project to prove anonymous denial and authorized access with
synthetic collections. Migration and rollback (never restoring anonymous
access) are in docs/operations/qdrant-authentication.md.

Refs #106

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven workflows had no top-level permissions block and inherited the
repository default (currently read/write). Add top-level
`contents: read` to workflows that check out code, and `permissions: {}`
to echo-only or external-notify workflows. Job-level write scopes remain
only where a job publishes, signs, attests, uploads SARIF or mutates PRs
and releases; the two SARIF upload jobs now also get the `contents: read`
their checkout needs.

tests/test_workflow_permissions.py fails if a workflow lacks top-level
permissions, grants a top-level write, or adds an unreviewed job write.

The repository default stays unchanged here; flipping it to read-only is
a separate owner action.

Refs #106

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… advisories

Scoped pip-audit triage of uv.lock: 397 rows in 36 packages, 280
distinct advisories after alias deduplication. Only two base-install
matches meet their advisory prerequisites:

- starlette 1.0.0 -> 1.0.1 (GHSA-86qp-5c8j-p5mr): AuthenticationMiddleware
  decides auth exclusion from request.url.path, which Starlette
  reconstructs from an unvalidated Host header.
- certifi 2026.2.25 -> 2026.10.18 (GHSA-6q48-6c7w-9gw2): every runtime
  HTTPS client trusts the bundle, which still includes a distrusted root.

Smallest fixed versions; the lock diff touches only these two packages
(revision header kept at 3; uv lock --locked passes). Every deferred and
inapplicable match is recorded with its reason in
docs/security/dependency-triage-2026-10-31.md, including mcp
GHSA-hvrp-rf83-w775 (STDIO single session; fix changes TaskStore listing)
and cryptography GHSA-537c-gmf6-5ccf (major bump, runtime unknown).

Refs #106

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ViperJuice
ViperJuice marked this pull request as ready for review October 31, 2026 18:07
@ViperJuice
ViperJuice merged commit 1a61533 into main Oct 31, 2026
12 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-31T18:08:46.828763Z 717a97d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 717a97dbdd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

server_url = os.getenv("QDRANT_URL", "http://localhost:6333")
if os.environ.get("QDRANT_USE_SERVER", "true").lower() == "true":
# Resolved outside the broad except so a missing required key is reported.
api_key = resolve_qdrant_server_api_key()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle missing Qdrant credentials in the status command

For a lexical-only installation with no file-backed vector index and no Qdrant configuration, QDRANT_USE_SERVER defaults to true, so this now raises QdrantAuthConfigError even though semantic search is optional. The index status command calls _get_vector_backend_status() outside its error handler, causing an uncaught exception and preventing the remaining metadata report from being displayed. Preserve the fail-closed connection behavior, but handle the configuration error in the status command so users can inspect their lexical indexes without configuring an unused Qdrant server.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant