Support customer-operated enterprise identity, model access and deployment

[turgaybulut]

Summary

Enterprise installations can run the gateway, dashboard and workers with customer-controlled
identity, secrets and model endpoints. Administrators can provision plans, delegate team
access and register approved deployments while retaining native provider contracts,
one provider attempt, tenant isolation and durable accounting/audit behavior.

For example, an administrator can register an internal model alias, restrict a team's
key to that alias, and inspect the request's policy decisions and usage. If the model
has no known price, reports show an unknown cost rather than a fabricated free request.

Completed work

Operator provisioning and subscription retirement

• Replace subscription checkout, portal and webhook handling with operator-managed plan activation.
• Add organization creation and existing-organization activation through ee/scripts/activate_plan.py.
• Provision the organization, privacy defaults and plan atomically without hosted invitations or email delivery.
• Apply approved tiers to active keys without reactivating revoked keys or resetting usage, reservations or ledger entries.
• Retain historical billing identifiers, periods, cancellation fields and webhook receipts; remove obsolete billing configuration from deployment definitions.
• Narrow the management subscription response to plan, status, source and entitlements, and document existing-customer cutover.

Diagnostic metadata and honest cost reporting

• Capture native provider completion facts without retaining free-form finish messages or stop-sequence text.
• Measure stream time to first supported content with a monotonic clock; keep unavailable measurements null.
• Record repeated-content observations without labelling them as proven retries.
• Hash explicitly supplied system/developer instructions with a secret HMAC scoped to tenant, protocol and deployment.
• Persist deployment classification and diagnostic fields in existing lifecycle/projection JSONB fields, terminal audit evidence and CSV exports.
• Preserve explicit pricing completeness across request, overview, trend, billing, allocation, CSV/PDF and budget-alert views. Unknown prices remain distinct from known zero; alerts identify the known-spend subtotal they use.

Durable policy decisions

• Record versioned verdicts for evaluated provider/model permissions, rate/repeat limits, quotas, privacy processing, spending and other admission failures.
• Bind verdicts to the verified tenant and caller identity; distinguish a workload key from a human actor.
• Persist pre-admission rejection evidence without creating billable usage or making a provider call.
• Preserve strict versus best-effort failure behavior: strict evidence failure prevents forwarding, while best-effort denial retains the original rejection and sanitized error telemetry.
• Reuse committed audit/outbox intent, idempotent delivery and hash-chain verification; expose decisions and caller identity through audit reads.
• Keep requests without an authenticated tenant out of guessed tenant audit records, and distinguish access denial from quota exhaustion.

Customer identity and secrets

• Add OIDC authorization-code login with PKCE, state and nonce checks; validate issuer, signature, audience and token lifetime.
• Bind identities by issuer and subject, require verified-email provisioning and reject unsafe email-based account linking.
• Store encrypted token material in Redis and expose only opaque HttpOnly browser sessions, with Secure cookies in production and Origin checks on mutations.
• Implement bounded session revalidation, expiry and logout, including local session cleanup when the IdP cannot be reached.
• Map configured IdP groups to organization/team roles, synchronize IdP-managed memberships and enforce local account deactivation.
• Keep OIDC configuration independent of Supabase and preserve the hosted authentication mode.
• Add Vault KV v2 storage with tenant/purpose-bound versioned references and token-file rereads for rotation. Provider plaintext remains outside relational records and telemetry.
• Document owner bootstrap, deprovisioning, workload-key revocation, CA trust, proxy behavior and IdP-side LDAP/AD federation.

Teams, scoped keys and quotas

• Add tenant-owned teams and explicit memberships with owner/admin/member/auditor and delegated team-administrator permissions.
• Migrate historical team labels without granting access implicitly or changing historical cost attribution.
• Add immediate key rotation in the existing key row, preserving ownership, expiry, policy and counters while invalidating the old secret.
• Enforce exact model allowlists, team ownership and current membership before provider execution.
• Add shared team daily/monthly request and monthly token quotas alongside existing per-key limits.
• Preserve reservation/settlement/refund accounting, concurrency fencing and consistent lock order; audit management changes through committed intent.

Model registry and native token counting

• Register tenant-scoped aliases, upstream models, native protocols, approved endpoints, credential references, timeouts, classification, owners and declared versions.
• Enforce administrator-controlled origin and CA policy; reject unsafe destinations, redirects and unauthorized or disabled models.
• Route through the existing provider execution path, preserving JSON/SSE, privacy handling, errors and the one-attempt rule without implicit failover.
• Record configuration/deployment audit facts and bound model-health probes by a wall-clock deadline. Declared versions remain operator claims, not serving-platform attestations.
• Keep custom models independent of public catalog admission; reject unpriced inference under monetary provider limits rather than treating it as free.
• Add native Anthropic count_tokens, including beta handling, authentication, rate/privacy policy and strict audit preflight/completion. Counting does not reserve billable quota, settle spending or create an inference lifecycle.
• Update both OpenAPI profiles and executable route/import ownership contracts.

Deployment, operations and recovery

• Add a Helm chart for the gateway, dashboard and four workers, with optional disposable PostgreSQL/Redis and support for customer-managed services.
• Support immutable images, internal registries, pull secrets, persistence, resource settings, ingress/origins, CA mounts and network policy configuration.
• Add worker successful-pass heartbeats/readiness and configurable migration deadlines, revision-specific migration Jobs and schema-ready initialization.
• Document install, bootstrap, upgrades, private backup/restore and schema-compatible application rollback; prohibit production database downgrade.
• Keep restored workers disabled until replay and external side effects have been reviewed.
• Add an isolated real-service chart harness using Keycloak, Vault and two internal model fixtures, with network-denial and runtime asset checks.
• Make cluster reuse reject application workloads across namespaces before cleanup can delete anything, and include its regression in CI.

Signed releases and offline packaging

• Add enterprise release artifacts for supported AMD64/ARM64 platforms: image identity, checksums, SBOM, provenance and explicit-key signatures.
• Keep publication separate from production deployment, pin workflow actions and support authenticated private image pulls.
• Add schema-2 offline bundle creation, inventory/signature verification and image import with archive-derived configuration identity and loaded-image re-export verification.
• Reject tampering, unexpected signers, wrong bundle IDs, extra files, symlinks and invalid archive/platform identities.
• Document independent trust-key provisioning/rotation, internal registry import and offline installation prerequisites.
• Preserve the current offline Ed25519 startup licence and existing licence terms; add no new capacity metering or worker/per-request licence gates.

Repository cleanup and reviewability

• Consolidate architecture guidance and move enterprise-specific operator/metadata guides under ee/.
• Remove obsolete migration trackers and workspace-only progress/rehearsal records from the project; retain reusable guides and runnable regression checks.
• Restore separate feature/fix commits while excluding newly introduced local-record-only commits from PR history.

Validation

The final application tree is identical to the previously verified tree after history restoration.

• Full backend gate: 923 passed, 4 historical-comparison checks skipped; lock, formatting, lint, types and both OpenAPI checks passed.
• Fresh disposable database migration reached the expected linear Alembic head.
• During implementation, isolated production-configured services, private-CA verification, real Keycloak/Vault and internal fixture inference passed.
• Browser login, key management, JSON/SSE inference, audit visibility, revocation and logout passed in the companion dashboard.
• A separate-database restore matched all 32 table fingerprints and passed recovered inference, revocation and audit continuation. Same-image/same-schema Helm upgrade and rollback control paths passed.
• Six-image local test-signed bundle verification/import passed with client networking restricted; wrong signer and tamper checks failed as expected. This did not publish a release or establish approved production signing provenance.

Rollout and limits

Apply the additive team, OIDC and model-registry migrations and deploy the matching dashboard
management contract. Select OIDC explicitly and configure its tenant/group mappings, trusted
origins and Vault access. Reconcile existing billing obligations before retiring the external
subscription integration; removing a webhook does not cancel a customer's subscription.

Customer Entra/PKI interoperability, agreed performance/recovery thresholds, cross-version
upgrade compatibility and independent clean offline installation require deployment acceptance.
Direct LDAP, SCIM, SAML, local-password authentication and new licence-capacity reporting are
not included. Model tests use native protocol fixtures and do not certify every model server
or deliver model weights. No production deployment or approved release publication was performed.

[sentry]

Bug: The _system_prompt_hash function will raise a KeyError if called with prepared.protocol as "count_tokens" because the protocol is missing from a dictionary lookup.
_{Severity: MEDIUM}

Suggested Fix

Add an entry for the "count_tokens" protocol to the dictionary within the _system_prompt_hash function. Setting its value to None, similar to the "chat" case, would prevent the KeyError and make the function more robust against future changes.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: ee/src/shim_enterprise/gateway/pipeline/quota_reservation.py#L92-L97

Potential issue: The `_system_prompt_hash` function uses a dictionary to map
`prepared.protocol` to a field name, but this dictionary is missing an entry for the
`"count_tokens"` protocol. If this function is ever called for a token counting request,
it will raise a `KeyError`. While a guard in `admission.py` currently prevents this code
path from being reached, this is a fragile design. Future refactoring could easily
expose this latent defect, causing an unhandled exception.

_{Did we get this right? 👍 / 👎 to inform future reviews.}