feat(b2b_learner_records): refuse tokens past the contract end date

README.md

@@ -144,6 +144,20 @@ refuses identically whether the organization is ungranted or doesn't exist.
 There's no round-trip and no grant store; removing the client revokes the
 access. See `docs/b2b-learner-records-provider-authorization.md`.
 
+That tenant does **not** read `X-Userinfo`. It verifies the bearer token
+itself (`tenants/b2b_learner_records/token.py`): RS256 against the realm's
+JWKS, checking issuer, audience, token class and lifetime, and it takes the
+claims it authorizes on from the verified payload. The gateway rebuilds
+`X-Userinfo` only for traffic that goes through the gateway, and the pod is
+reachable without doing so — the pod security group admits the whole pod
+subnet and the CNI runs with network policy disabled on both data clusters.
+Aggregate k-anonymized figures can live with that; records naming individual
+learners can't. `OL_ANALYTICS_API_B2B_LEARNER_RECORDS_ISSUER` and
+`..._AUDIENCE` come from Vault via the Pulumi stack, out of the same entry
+the gateway route reads, so the two can't check different realms. Both
+default to production's, and the app refuses to start if the issuer is left
+at that default in any other deployed environment.
+
 The org-manager round-trip authenticates with this service's **own** OAuth2
 client-credentials token and names the subject user explicitly
 (`?user_global_id=<keycloak sub>`), rather than forwarding the caller's

docs/b2b-learner-records-provider-authorization.md

@@ -41,13 +41,27 @@ client listing the same organization keeps reading the same rows, so ending one
 provider's access means removing that provider's clients, not the
 organization's.
 
-**Open, for pdpinch: does access expire with the contract?** Today it doesn't.
-The per-request steps below check no end date, so a client whose cleanup PR is
-forgotten keeps issuing tokens. Two ways to enforce it:
-
-- Carry the contract end date as a claim, and refuse tokens past it.
-- Check the warehouse instead. `dim_contract` already has `contract_is_active`
-  and `contract_end_date`, and the API serves both on `/courses`.
+**Access ends with the contract, as a backstop.** A client provisioned with
+`contract_end_date` carries it as the `learner_records_contract_end_date`
+claim, and the API refuses the client's tokens once that date has passed. The
+date is inclusive and read as Anywhere on Earth (UTC-12): access runs through
+the end of the end date wherever the partner is, and stops at 12:00 UTC the
+following day. A client provisioned without an end date is open-ended. A claim
+that is present but isn't a `YYYY-MM-DD` string refuses every token, since
+reading it as "no end date" would fail open.
+
+This does not replace removing the client when the contract ends. It is what
+stops a client whose cleanup PR was forgotten, and when it fires the API logs
+`learner_records_contract_ended` at warning level, which means that cleanup is
+overdue. Every authorized request also logs the client's `contract_end_date`
+beside `learner_records_access`, so an alert can raise the renewal
+conversation before a partner's sync breaks.
+
+Checking the warehouse instead (`dim_contract` has `contract_is_active` and
+`contract_end_date`) was the other option. It would put a StarRocks read ahead
+of every authorization decision, and a client covers every contract under
+each of its organizations, so no single `dim_contract` row says when the
+client's access ends.
 
 ## What the API does
 
@@ -61,6 +75,9 @@ In `tenants/b2b_learner_records/auth.py`, per request:
 3. Require the `learner-records:read` scope. Identity fields are always
    populated.
 
+Before any of that, `token.py` verifies the token and refuses one whose
+contract end date has passed (above).
+
 No call to mitxonline or any other service, and no access store. The client
 definition is the only place access is recorded.
 
@@ -107,15 +124,41 @@ definition is the only place access is recorded.
 - Whether APISIX's `openid-connect` plugin passes the hardcoded claim and
   scopes through in `X-Userinfo` on a bearer-only route. The learner-records
   mount needs a bearer-only route, but today's routes use the redirect flow.
-  Check on QA.
-- That APISIX *overwrites* a caller-supplied `X-Userinfo` rather than passing
-  it through. `core/auth/userinfo.py` decodes whatever header arrives without
-  validating a token, and the organization check reads from it. Also confirm
-  the pod can't be reached except through the gateway route: `k8s/` defines no
-  NetworkPolicy. Check both on QA.
+  Check on QA. This no longer gates the tenant (see below), but the claim
+  still has to arrive in the token.
 - The access-token lifespan these clients will get, since it is the
   revocation window.
 
+## The app verifies the token; the gateway is not the trust boundary
+
+Settled 2026-09-18, after Copilot raised it on
+[ol-infrastructure#5939](https://github.com/mitodl/ol-infrastructure/pull/5939).
+
+The question above was whether APISIX overwrites a caller-supplied
+`X-Userinfo`. It does, at the start of its `rewrite` phase. That is beside the
+point, because a caller does not have to go through APISIX. The pod security
+group admits the whole pod subnet, and `aws-eks-nodeagent` runs with
+`--enable-network-policy=false` on both data clusters, so the NetworkPolicies
+that exist are no-ops. Any compromised in-cluster workload can post a forged
+`X-Userinfo` naming `learner-records:read` and any organization UUID, straight
+to the pod.
+
+For k-anonymized aggregates that is a risk worth arguing about. For records
+that name individual learners it is not, so this tenant verifies the bearer
+token itself (`tenants/b2b_learner_records/token.py`): RS256 against the
+realm's JWKS, checking issuer, audience and lifetime, and taking `scope`,
+`learner_records_organizations` and `azp` from the verified payload.
+`X-Userinfo` is not read at all. The gateway route stays as it is; it is now
+defence in depth rather than the only check.
+
+Rejected: locking down the pod security group, and enabling CNI network
+policy cluster-wide. Both are larger changes that protect one tenant by
+changing how every workload on the cluster is reached.
+
+`b2b_dashboard` still authorizes from `X-Userinfo`. Its exposure is aggregate
+and k-anonymized and it has a browser session flow, so it is a separate
+decision, not a follow-up to this one.
+
 ## Follow-ups
 
 Not needed to write the tenant, but each needs an owner before partners

docs/openapi/b2b-learner-records-v1.yaml

@@ -361,11 +361,12 @@ components:
     oauth2ClientCredentials:
       type: oauth2
       description: >-
-        Keycloak client credentials, validated at the gateway. One client is
-        issued per contracted integration and lists the organizations it may
-        read, every contract under each included. Scopes bound to a client are
-        a contractual limit on that credential; they are not the consent
-        mechanism.
+        Keycloak client credentials. The token's signature, issuer, audience
+        and lifetime are verified by the service itself, not only at the
+        gateway. One client is issued per contracted integration and lists the
+        organizations it may read, every contract under each included. Scopes
+        bound to a client are a contractual limit on that credential; they are
+        not the consent mechanism.
       flows:
         clientCredentials:
           tokenUrl: https://sso.ol.mit.edu/realms/olapps/protocol/openid-connect/token
@@ -912,11 +913,17 @@ components:
           example: { code: invalid_parameter, detail: 'limit: Input should be less than or equal to 1000' }
 
     Unauthorized:
-      description: Missing, malformed or expired token.
+      description: >-
+        No token, or one that failed verification: bad signature, unknown
+        signing key, wrong issuer or audience, or outside its validity window.
+        The body does not say which, so it cannot be used to probe. The one
+        exception is a genuine token whose contract has ended (through the end
+        of the contract end date, Anywhere on Earth), which says so and names
+        the date.
       content:
         application/json:
           schema: { $ref: '#/components/schemas/Error' }
-          example: { code: unauthorized, detail: 'Invalid or expired access token' }
+          example: { code: unauthorized, detail: 'Invalid or missing bearer token' }
 
     Forbidden:
       description: >-
@@ -928,7 +935,7 @@ components:
       content:
         application/json:
           schema: { $ref: '#/components/schemas/Error' }
-          example: { code: no_organization_access, detail: 'No access to the requested organization' }
+          example: { code: no_organization_access, detail: 'No grant for the requested organization' }
 
     TooManyRequests:
       description: Per-client rate limit exceeded.

pyproject.toml

@@ -19,6 +19,7 @@ dependencies = [
     "opentelemetry-instrumentation-fastapi>=0.52b0",
     "opentelemetry-instrumentation-httpx>=0.52b0",
     "granian[reload,uvloop]>=2.7",
+    "pyjwt[crypto]>=2.10",
 ]
 
 [tool.uv]
@@ -86,6 +87,7 @@ dev = [
     "cyclopts>=4.22.5",
     "pyyaml>=6.0.3",
     "types-pyyaml>=6.0.12.20260724",
+    "cryptography>=44",
 ]
 
 [build-system]
@@ -127,7 +129,7 @@ ignore = [
 [tool.ruff.lint.per-file-ignores]
 # S105-S107: fixture values named `token` are fake OAuth2 tokens for the
 # MITx Online client tests, not real credentials.
-"tests/*" = ["S101", "ANN", "PLR2004", "S105", "S106", "S107"]
+"tests/*" = ["S101", "ANN", "PLR2004", "PLR0913", "S105", "S106", "S107"]
 # structlog's processor signature is conventionally (logger: Any, method_name: str,
 # event_dict: dict) — matches mitol-django-observability's own typing, ported verbatim.
 "src/ol_analytics_api/core/observability/processors.py" = ["ANN401"]

src/ol_analytics_api/tenants/b2b_learner_records/app.py

@@ -15,19 +15,24 @@
 
 from ol_analytics_api.core.errors import add_shared_error_handlers
 from ol_analytics_api.core.health import register_readiness_check
+from ol_analytics_api.tenants.b2b_learner_records.errors import (
+    ErrorCode,
+    add_error_handlers,
+    error_body,
+)
 from ol_analytics_api.tenants.b2b_learner_records.routers import organizations
 
 TENANT_NAME = "b2b_learner_records"
 
 
 async def _bad_request(_request: Request, exc: RequestValidationError) -> JSONResponse:
-    # The contract's error body is {"detail": "<message>"} with a 400, not
-    # FastAPI's 422 carrying a list of error objects.
+    # The contract's error body is {"code": ..., "detail": ...} with a 400,
+    # not FastAPI's 422 carrying a list of error objects.
     error = exc.errors()[0]
     location = ".".join(str(part) for part in error["loc"][1:])
     return JSONResponse(
         status_code=status.HTTP_400_BAD_REQUEST,
-        content={"detail": f"{location}: {error['msg']}"},
+        content=error_body(ErrorCode.INVALID_PARAMETER, f"{location}: {error['msg']}"),
     )
 
 
@@ -41,6 +46,9 @@ def create_app() -> FastAPI:
     )
     app.include_router(organizations.router)
     add_shared_error_handlers(app)
+    # After the shared handlers: this tenant overrides the 503 so it carries
+    # the code its contract documents.
+    add_error_handlers(app)
     app.add_exception_handler(RequestValidationError, _bad_request)  # type: ignore[arg-type]
     register_readiness_check(TENANT_NAME)
     return app

src/ol_analytics_api/tenants/b2b_learner_records/auth.py

@@ -3,9 +3,14 @@
 There is no user in this flow. MIT issues one Keycloak client-credentials
 client per contracted integration, and that client carries the organization
 UUIDs its contract covers as a hardcoded claim
-(docs/b2b-learner-records-provider-authorization.md). APISIX validates the
-token and forwards its claims in X-Userinfo, so authorization here is a set
-membership test: no call to MITx Online, no grant store.
+(docs/b2b-learner-records-provider-authorization.md). Authorization is
+therefore a set membership test over the token's own claims: no call to MITx
+Online, no grant store.
+
+Unlike b2b_dashboard, this tenant does not read X-Userinfo. It verifies the
+bearer token's signature itself (token.py), because a header rebuilt by
+APISIX only proves anything about traffic that went through APISIX, and
+these records name individual learners.
 """
 
 from __future__ import annotations
@@ -15,19 +20,24 @@
 from typing import Annotated, Any
 
 import structlog
-from fastapi import Depends, HTTPException, Security, status
+from fastapi import Depends, Security, status
 from fastapi.openapi.models import OAuthFlowClientCredentials, OAuthFlows
 from fastapi.security import OAuth2
 
-from ol_analytics_api.core.auth.userinfo import get_userinfo
 from ol_analytics_api.tenants.b2b_learner_records.config import settings
+from ol_analytics_api.tenants.b2b_learner_records.errors import ApiError, ErrorCode
+from ol_analytics_api.tenants.b2b_learner_records.token import (
+    CONTRACT_END_DATE_CLAIM,
+    verified_claims,
+)
 
 ORGANIZATIONS_CLAIM = "learner_records_organizations"
 READ_SCOPE = "learner-records:read"
 
 # Declares the contract's security scheme in this tenant's OpenAPI, so generated
-# clients obtain and send a token. It enforces nothing: auto_error=False, and
-# the checks below read the claims APISIX forwards after validating the token.
+# clients obtain and send a token. It enforces nothing on its own
+# (auto_error=False); the token is verified by the TokenClaims dependency and
+# the checks below run against the verified payload.
 oauth2_client_credentials = OAuth2(
     flows=OAuthFlows(
         clientCredentials=OAuthFlowClientCredentials(
@@ -45,13 +55,13 @@
 
 log = structlog.get_logger(__name__)
 
-UserInfo = Annotated[dict[str, Any], Depends(get_userinfo)]
+TokenClaims = Annotated[dict[str, Any], Depends(verified_claims)]
 
 
-def _granted_organizations(userinfo: dict[str, Any]) -> set[uuid.UUID]:
+def _granted_organizations(claims: dict[str, Any]) -> set[uuid.UUID]:
     # Keycloak emits the claim as a JSON array only when the mapper's claim
     # type is JSON. Any other shape grants nothing rather than being guessed at.
-    claim = userinfo.get(ORGANIZATIONS_CLAIM)
+    claim = claims.get(ORGANIZATIONS_CLAIM)
     if not isinstance(claim, list):
         return set()
     granted = set()
@@ -63,21 +73,29 @@ def _granted_organizations(userinfo: dict[str, Any]) -> set[uuid.UUID]:
 
 def require_organization_grant(
     organization_id: uuid.UUID,
-    userinfo: UserInfo,
+    claims: TokenClaims,
     _token: Annotated[str | None, Security(oauth2_client_credentials, scopes=[READ_SCOPE])],
 ) -> None:
-    scopes = userinfo.get("scope")
+    scopes = claims.get("scope")
     if not isinstance(scopes, str) or READ_SCOPE not in scopes.split():
-        raise HTTPException(
+        raise ApiError(
             status_code=status.HTTP_403_FORBIDDEN,
+            code=ErrorCode.MISSING_SCOPE,
             detail=f"Token lacks the {READ_SCOPE} scope",
         )
-    if organization_id not in _granted_organizations(userinfo):
-        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=NO_GRANT_DETAIL)
+    if organization_id not in _granted_organizations(claims):
+        raise ApiError(
+            status_code=status.HTTP_403_FORBIDDEN,
+            code=ErrorCode.NO_ORGANIZATION_ACCESS,
+            detail=NO_GRANT_DETAIL,
+        )
     # Every granted read discloses identifiable learner records, so record which
     # client read which organization. The access log has the path but not the client.
+    # The end date rides along so an alert can warn MIT ahead of a lapse; token.py
+    # refuses the credential once it passes.
     log.info(
         "learner_records_access",
-        client_id=userinfo.get("azp") or userinfo.get("client_id"),
+        client_id=claims.get("azp") or claims.get("client_id"),
         organization_id=str(organization_id),
+        contract_end_date=claims.get(CONTRACT_END_DATE_CLAIM),
     )