Skip to content

Add cloud-only Polar subscriptions and organization quotas - #41

Open
turgaybulut wants to merge 17 commits into
feat/customer-operated-enterprisefrom
feat/cloud-subscriptions
Open

turgaybulut wants to merge 17 commits into
feat/customer-operated-enterprisefrom
feat/cloud-subscriptions

Conversation

@turgaybulut

@turgaybulut turgaybulut commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Hosted cloud needs self-service subscriptions while customer-operated distributions remain free of commerce code and payment dependencies. Add a separately packaged cloud application using Polar for fixed monthly/yearly plans, with shared organization quotas and the existing enterprise entitlement controls.

Stacked on #40. This diff contains only the cloud subscription work above the enterprise foundation. Companion dashboard: https://github.com/DurthVadr/SHIM_LP/pull/37. Merge the enterprise foundations, then this backend, then the cloud dashboard.

Changes

  • Add owner-scoped durable checkout and customer portal operations, signed webhook intake, and retryable state reconciliation with tenant binding and revision checks.
  • Enforce included request/token quotas across organization keys without resetting usage on plan changes; quota windows follow UTC calendar months for both payment intervals.
  • Isolate the Polar SDK, cloud routes, migrations and worker in ee/cloud; explicitly select community, on-prem and cloud packages, images and API contracts.
  • Document product configuration, operator overrides, deployment and launch verification in ee/cloud/README.md.

Follow-up fixes on top of the initial implementation

  • Scope token-count repeat detection to its own protocol. protocol was passed into the repeat identity but absent from its allowlist, so /v1/messages/count_tokens consumed a /v1/messages repeat allowance.
  • Align both spend-denial audit readers with the stored key. AuditPreflightIntent types usage_summary as int | Decimal, so the stored value must be numeric; the readers still queried the removed string key, which counted spend-limit denials as technical failures in the overview summary and trend.
  • Keep the quota opt-in lock-free in the steady state. It ran from the cloud auth dependency on every authenticated request and always took SELECT ... FOR UPDATE on the organization row, serializing each tenant before its real work began. The locked re-read remains for the drift path.
  • Stop configuration errors echoing credentials. The billing checks moved from a model validator to field validators, so a failure stays on its own field instead of attaching the merged input mapping to ValidationError.errors(), which exposed the raw POLAR_ACCESS_TOKEN even with hide_input_in_errors set.
  • Verify cloud release artifacts in CI. The shim-cloud wheel and sdist are now checked against ee/cloud/LICENSE and ee/cloud/NOTICE, the Elastic-2.0 licence expression, the shim-enterprise requirement, and the absence of community and enterprise source trees.
  • Remove per-call Polar retry/timeout kwargs in favour of one client-level policy, drop two write-only OrganizationPlan fields, reuse the locked row returned by require_team, deduplicate the API-key field validators, and delete two superseded cloud planning documents.

Validation

  • Full suite via uv run --locked --all-packages python -m pytest -q: 972 passed, 4 skipped.
  • Lockfile, Ruff format/check, Ty, all three OpenAPI contract checks, both Alembic checks and git diff --check passed.
  • Migration round trip verified locally: upgrade, downgrade base for cloud then enterprise, re-upgrade, then alembic check on both schemas.
  • Built the community, enterprise and cloud wheels and sdists and ran the artifact verifications, including the new cloud step; the licence boundary check was confirmed to fail when ee/cloud/NOTICE is altered.
  • Community dependency isolation confirmed unchanged: no pyproject.toml or uv.lock diff, and no commerce dependency reaches the community package.

Tests run against real PostgreSQL 16 and Redis 7 locally. No dependency was added or changed.

Before launch

Draft pending real Polar sandbox checkout and merchant/product/webhook configuration. Local tests use isolated services and mocked vendor transport; no live purchase has been performed. No deployment is requested by this PR.

@turgaybulut
turgaybulut added this pull request to stack #42 September 11, 2026 12:32
The admission stage added "protocol" to the repeat-detection payload, but
protocol was absent from the _repeat_material allowlist, so count_tokens
and messages produced an identical prompt identity. A token-count call
silently consumed a /v1/messages repeat allowance.

Add protocol to the allowlist and assert the two share no allowance.
write_spend_denial_preflight wrote usage_summary={"spend_denied": 1},
but both audit readers query usage_summary["denial_reason"]. Nothing
consumed the new key, so spend-limit denials stopped matching the
policy_failed predicate and were counted as technical failures in the
overview summary and trend.

Restore the denial_reason key and pin the writer/reader contract.
Two cloud documents shipped false against the branch that carried them:
the proposal said "not implemented" and claimed release built ee/Dockerfile
with --all-packages, and the implementation file was an agent checklist.
Neither had inbound references and both restated ee/cloud/openapi/cloud.json.

Also collapse the required-gate block to its single maintained copy,
delete a route table the contract file already owns, and sort the
enterprise/cloud route profiles with an assertion so drift fails a test.
configure_organization_quota runs from the cloud auth dependency on every
authenticated request and always took SELECT ... FOR UPDATE on the
organization row, serializing each tenant's requests before their real
work began.

Read the tier first and return without locking when the allowance already
matches; keep the locked re-read for the drift path. Drop two write-only
OrganizationPlan fields while here.
AuditPreflightIntent types usage_summary as int | Decimal, so the writer
must store spend_denied=1; the previous string denial_reason value could
not validate at all. The writer was corrected earlier, but both audit
readers still queried the old string key, so spend-limit denials were
counted as technical failures.

Align the overview and management predicates with the stored key and pin
the writer/reader pairing in both drift directions.
…handlers

Move the outbound timeout and retry-disabled policy to the single Polar
client construction instead of repeating it at five call sites, express
the sellable plan vocabulary once as a Literal, and reuse the locked Team
row and shared field validators in the management routes.
Move the billing checks from a model validator to field validators so a
failure stays on its own field and the merged input mapping never reaches
ValidationError.errors(), which carried the raw POLAR_ACCESS_TOKEN even
with hide_input_in_errors set.

Add the missing CI step verifying the shim-cloud wheel and sdist against
ee/cloud/LICENSE and ee/cloud/NOTICE, the Elastic-2.0 expression, and the
absent community and enterprise source trees.
@turgaybulut
turgaybulut marked this pull request as ready for review September 16, 2026 21:08
Comment on lines 229 to 236
team_id: UUID | None = None
allowed_models: list[str] | None = Field(default=None, max_length=200)

@field_validator("allowed_models")
@classmethod
def validate_models(cls, value: list[str] | None) -> list[str] | None:
if value is not None and any(
not item or item != item.strip() or len(item) > 200 for item in value
):
raise ValueError(
"Model identifiers must be nonblank and at most 200 characters"
)
return list(dict.fromkeys(value)) if value is not None else None

@field_validator("cost_center", "team")
@classmethod
def validate_attribution(cls, value: str | None) -> str | None:
if value is None:
return None
return normalize_attribution(
value,
maximum_length=settings.COST_TAG_MAX_LENGTH,
)
validate_models = field_validator("allowed_models")(_validate_allowed_models)
validate_attribution = field_validator("cost_center", "team")(_validate_attribution)


class ApiKeyView(BaseModel):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The refactored Pydantic validators _validate_allowed_models and _validate_attribution have an incorrect function signature, missing the cls argument, which will cause a TypeError at runtime.
Severity: HIGH

Suggested Fix

Update the _validate_allowed_models and _validate_attribution functions to accept cls as their first argument to match the signature expected by Pydantic's field_validator. For example: def _validate_allowed_models(cls, value: list[str] | None) -> list[str] | None:.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: ee/src/shim_enterprise/api/v1/management.py#L229-L236

Potential issue: The refactored validator functions `_validate_allowed_models` and
`_validate_attribution` are defined to accept only a single `value` argument. However,
they are registered using Pydantic v2's `field_validator`, which invokes validators as
classmethods, passing both `cls` and `value`. When an API key is created or patched via
the `/api-keys` endpoint, this signature mismatch will cause a `TypeError` because the
function receives two arguments instead of the expected one. This will result in a
runtime error, preventing API key creation and modification.

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified as a false positive against this PR's locked Pydantic 2.13.5. field_validator(...) accepts a standalone validator callable with the signature (value); it only auto-wraps a callable as a classmethod when the first parameter is named cls. Both ApiKeyInput and ApiKeyPatch were instantiated, and their normalization, deduplication, and rejection paths were exercised without any TypeError. Adding cls is valid but unnecessary, so no code change is needed.

Reference: https://docs.pydantic.dev/latest/concepts/validators/#field-validators

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant