Skip to content

feat: add the project administration and eval resources the CLI calls - #192

Open
brightsparc wants to merge 1 commit into
mainfrom
claude/sdk-admin-resources
Open

brightsparc wants to merge 1 commit into
mainfrom
claude/sdk-admin-resources

Conversation

@brightsparc

Copy link
Copy Markdown
Contributor

Ports the project administration and eval resources from the Rust SDK (introspection-org/introspection-rust-sdk#123) into the Python SDK. None of them were on Python main before this PR.

What's added

Namespace Plane Methods
client.projects CP list(project=, limit=, next=), get(slug_or_id) (the path segment is percent-encoded). Read-only, see below.
client.api_keys CP list(project=, expiring=) (unwraps {"api_keys": [...]}, not paginated), create (returns ApiKeyCreated with the one-time api_key), update, delete(expires_in=) (rotation grace), acknowledge
client.applications CP list/get/create/update/delete, plus .idps (paginated list, create, delete), .secrets (paginated list, create returns the one-time secret, delete(expires_in=)) and .keys (unpaginated list, create with no body, delete)
client.credentials CP list(project=, runtime_id=, runtime=, environment=), get, create, update, delete. The secret is write-only
client.variables CP list, create, update, delete (the API has no read-one route)
client.integrations CP list/get/create/update/delete. Org-wide, so it takes no project. The wire field type is the integration_type= kwarg
client.judges CP list(project=, runtime_group_id=), get, update(enabled=, sample_rate=)
client.oidc CP me(). feature_flags values keep their bool/int/float/str type
client.images / runner.images DP list(kind=, status=, name=, runtime_id=), get, create(name=, source=, provenance=, scan=), delete. source is a kind-tagged union: ContextImageSource or MirrorImageSource. Image.pull_reference is the digest-pinned reference
tasks.commands (client and Runner) DP create(task_id, ExecCommand / UploadCommand / DownloadCommand / tagged dict), list(task_id, status=), get(task_id, command_id, wait_seconds=)
tasks.create DP new kind= and commands= kwargs, plus TaskKind.EVAL, so an eval task can accept commands. This matches the follow-up commit on the Rust PR. Before this, Task failed to parse kind: "eval"

Other details:

  • Most project-scoped CP routes take an optional project= (slug or id). It is sent as ?project= and only when given, because it's needed only when the credential doesn't already name a project.

  • Secrets are redacted. These use pydantic SecretStr, so repr, str and model_dump_json print **********:

    • the API key plaintext
    • an Application client secret
    • an IdP client_secret
    • a credential secret
    • the integration OAuth code

    The request models serialize the plaintext only in the JSON body, through a private WireSecret type. Write methods accept str or SecretStr.

  • MAX_COMMAND_WAIT_SECONDS = 25 is public, from introspection_sdk.runner_resources. The API allows 60s, but the SDK's HTTP timeout is 30s.

  • Environment (development / staging / production) is in introspection_sdk.schemas.lanes.

  • ImagesResource, TaskCommandsResource and the commands / images members are added to the data-plane Protocols. mypy checks that the client and Runner still satisfy them.

  • The sync twins are generated by scripts/gen_sync.py. There's a shared private _crud.py base, which is why duplication went down (see test results).

  • README.md has two new sections, and there's a new example at examples/introspection_examples/api/project_admin.py. The example isn't added to examples.yml because these routes need an owner CP session, and the workflow only has an API key.

Projects are read-only (list + get), on purpose. The user confirmed the SDK must not create, update or delete projects; that stays in the dashboard and API. projects:write and projects:delete are in no API-key, Application/service-account or CLI-login ceiling (see introspection_common/permissions.py and scopes.py in introspection-cloud). Only an owner or admin's browser CP session carries them, so those methods would 403 for every SDK credential. Python main had no project write methods, so nothing was removed. A test asserts Projects has no create, update or delete.

Parity

This mirrors introspection-org/introspection-rust-sdk#123, with one difference: Rust has project create/update/delete and this PR does not.

Out of scope

  • /v1/remotes/connect: a CLI-owned protobuf WebSocket.
  • Endpoint writes.

Tests and checks run (all locally, all passing)

  • uv run ruff format --check . and uv run ruff check .
  • uv run python scripts/gen_sync.py: no diff under _sync/.
  • uv run ty check and uv run mypy.
  • uv run pytest tests/ --cov=introspection_sdk: 823 passed, 10 deselected (the live walks), 92.57% total coverage against the 80% floor.
    • New mock tests in tests/rest/test_project_admin.py and tests/rest/test_images.py (49 tests) pin:
      • paths and the percent-encoded project segment
      • the ?project= / expires_in / filter params
      • body wire names, including that an explicit [] and False are sent
      • the kind-tagged image source and command unions, and that bad tags or a shell-string command are refused before sending
      • the wait cap (including that it stays below the client timeout)
      • secret redaction
      • async variants
  • uv run deptry .: no issues.
  • npx --yes jscpd@5.4.1: passes. Duplication went from 16.22% on main to 16.03%, so .jscpd.json threshold is ratcheted from 16.3 to 16.1.
  • griffe check ... --against v0.29.1 -e scripts/griffe_reexports.py: no breakages.
  • pre-commit run on the changed files: all hooks pass (gitleaks, ruff, ty, gen-sync, json/toml/whitespace).
  • uv run --project examples python scripts/check_examples.py: passes.
  • API contract. scripts/check_api_contract.py gains 56 surfaces covering every new body, read model and list/delete/read query set.
    • Against specs generated from introspection-cloud main source (--spec dp-src.json --cp-spec cp.json): no drift on any new surface. The only change from the baseline is that kind and commands drop out of the existing TaskCreate "missing" list.
    • The DP lane/project_id browser-session selectors are exempt.
    • Other pre-existing drift is untouched: MemberCreate, BrokerTokenRequest, Automation, the project_id exemptions, and others.
    • The scheduled API Contract job compares against the published docs.introspection.dev specs, which lack these routes. It is already red on main for the same reason (members, connections and ShareUpdate are missing there too), so this PR doesn't change its state.

Live tests

tests/live/test_live_admin.py adds ten walks that mirror the Rust walks: oidc_me, api keys, applications, credentials, variables, integrations, judges, projects (read), images and task commands. The Rust tolerances are kept:

  • The applications walk uses a service_account app, and the IdP attach must be refused with 400 or 403.
  • Integration create skips on 409.
  • Image create skips on 422 (evals off).
  • The projects walk only gets the seeded project by slug and checks the list contains it.

The walks use INTROSPECTION_LIVE=1, INTROSPECTION_BASE_API_URL, INTROSPECTION_CP_SESSION (sent alone as Cookie: intro_cp_session=...), INTROSPECTION_PROJECT, INTROSPECTION_DP_URL and INTROSPECTION_MEMBER_TOKEN. They skip rather than fail when any is missing.

They were not run against a deployment (no stack was available). I only checked that they're collected, deselected by default, and skip cleanly with and without INTROSPECTION_LIVE=1.

Relationship to #175. #175 adds the general live suite (tests/live/ with its own conftest.py and _support.py) and isn't on main yet. This PR doesn't touch that branch:


Generated by Claude Code

Ports the namespaces introspection-rust-sdk#123 added. Control Plane:
projects (list/get only), api_keys, applications (idps, secrets, keys),
credentials, variables, integrations, judges and oidc.me. Data Plane:
images on the client and the Runner, tasks.commands with a wait capped
at MAX_COMMAND_WAIT_SECONDS, and kind/commands on tasks.create so an
eval task can accept commands. Secrets the API returns once or takes on
write are pydantic SecretStr.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016njWGNXD7FLvM4zmH3zs7j

This branch was successfully deployed

1 active deployment
build — 4777eba6 Deployed Oct 21, 2026 by brightsparc via Unit Tests #519
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants