feat(b2b_learner_records): refuse tokens past the contract end date - #73
Open
blarghmatey wants to merge 9 commits into
Open
blarghmatey wants to merge 9 commits into
blarghmatey wants to merge 9 commits into
Conversation
…-Userinfo The tenant authorized from X-Userinfo, which APISIX rebuilds from a token it has already validated. That only says anything about traffic that went through APISIX, and nothing forces a caller to: the pod security group admits the whole pod subnet, and aws-eks-nodeagent runs with --enable-network-policy=false on both data clusters, so the NetworkPolicies that exist are no-ops. A compromised in-cluster workload could post a forged X-Userinfo carrying learner-records:read and any organization UUID and read identifiable learner records. This verifies the token in the app instead. token.py checks RS256 against the olapps realm JWKS, with issuer, audience and lifetime, and require_organization_grant reads scope, learner_records_organizations and azp from the verified payload. X-Userinfo is not read by this tenant at all. Anything that fails verification gets one 401 that does not say which check failed. The key set is cached for a TTL and refetched once on an unseen kid, so a realm key rotation costs a fetch rather than a TTL of refusals; a cooldown keeps forged kids from turning the endpoint into an amplifier pointed at Keycloak. An unreachable JWKS refuses rather than falling back. Config derives the token, JWKS and issuer URLs from one issuer setting, so moving a deployment to another realm is one environment variable. b2b_dashboard still reads X-Userinfo. Its exposure is aggregate and k-anonymized and it has a browser session flow, so that is a separate call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qoz9pfQxUU8VLsRxr1tTnm
The security scheme said the credential is 'validated at the gateway' and the 401 example named a detail string the tenant no longer returns. Both describe the behaviour before the tenant started verifying the bearer JWT itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qoz9pfQxUU8VLsRxr1tTnm
…lass Review findings on the verification added in the previous commit. A JWKS body that parses as JSON but holds no usable key (an empty set, an error document served as 200, a bare list) raised out of the dependency as a 500 rather than the 401 it exists to return, and since the fetch never populated the cache it recurred on every request. _fetch now turns every way the endpoint can go wrong into one JWKSUnavailableError. A fetch failure was neither held off nor backed by the key set already in hand, so once the TTL lapsed with Keycloak unreachable every request took the lock and paid the timeout in turn while a still-valid key set sat unused. Failures now hold off further attempts briefly, and a stale key set is served through an outage: realm keys turn over on the order of months, so refusing every request because a refresh failed is an outage this service inflicts on itself. An ID token minted for ol-analytics-api-client carries the realm's signature, this issuer and this audience, so it cleared verification and was stopped only by carrying no organization claim. The token class is now required and checked. Also: a forced load no longer refetches what another request just fetched; the unknown-kid cooldown starts only once a freshly fetched key set really lacked the kid, so a failed refetch doesn't lock every other kid out for a minute; and the tenant refuses to start when the issuer is left at the production realm in another deployed environment, since every failure mode of that setting is a silent 401. The concurrency test pinned the cache rather than the lock -- a mocked transport never suspends, so the gathered requests serialised themselves and it passed with the lock removed. It now fetches through a callback that awaits. Added cases for the token class, for HS256 signed with the public key out of the JWKS, for the multi-audience shape Keycloak actually emits, and for the unusable-key-set bodies above. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qoz9pfQxUU8VLsRxr1tTnm
…fetch A forced refetch that fails hands back the stale key set rather than raising, so the unknown kid still isn't found and the cooldown started on evidence nobody gathered. The comment there already claimed this didn't happen. The case it breaks is a Keycloak blip during a key rotation: the refetch fails, the cooldown starts, and tokens carrying the new kid keep getting 401s for a minute after Keycloak comes back, with no refetch attempted. Now the cooldown starts only when _fetched_at advanced, which is what says a freshly fetched key set really lacked the kid. A failed fetch is already held off by _retry_after. The regression test fails against the previous code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qoz9pfQxUU8VLsRxr1tTnm
…ys required
The Error schema in docs/openapi/b2b-learner-records-v1.yaml declares every
error response as {code, detail}, with code required and drawn from a fixed
enum, and tells clients to branch on it because the wording of detail may
change. Nothing produced it. FastAPI's HTTPException renders {"detail": ...},
so a client generated from the spec would reject every error this tenant
returned, on all five documented statuses.
errors.py adds ApiError, carrying the code, plus the handler that renders it.
The code can't be derived from the status, which is why it rides on the
exception: 403 is missing_scope when the token's scopes fall short and
no_organization_access when the organization isn't granted, and those stay
distinct to a client while remaining indistinguishable to an attacker, since
both keep the same detail.
400 (invalid_parameter), 401 (unauthorized), both 403s and 503 (unavailable)
now carry their code. Errors the contract doesn't document, a 404 for an
unrouted path or a 405, keep FastAPI's own body rather than being dressed in
a code that claims a contract covering them.
Scoped to this tenant. b2b_dashboard has no published contract to honour, so
adding a field to its error bodies would be a change with no reader. 429 is
enforced at APISIX, not here, so nothing in this service produces that body.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qoz9pfQxUU8VLsRxr1tTnm
… main Two things #70 changed under this branch. _partner_header became _partner_token here while #70 added a test that calls it. Both sides touched different parts of the file, so the rebase merged them cleanly and the result referenced a helper that no longer exists. #70 also added pyyaml, which was the only reason the error-code test asserted against the ErrorCode enum rather than the contract the enum exists to mirror. It now reads the enum out of docs/openapi/b2b-learner-records-v1.yaml, so a code added on one side and not the other fails. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qoz9pfQxUU8VLsRxr1tTnm
Learner-records clients carry learner_records_contract_end_date from their contract, but nothing checked it, so a partner whose contract ended kept reading identifiable records until someone deleted the Keycloak client. The tenant now refuses a verified token once its contract end date has passed, reading the date as inclusive and Anywhere on Earth so no partner is cut off early by its own clock. An absent claim stays open-ended, since existing clients were provisioned without one. A malformed claim refuses, because the alternative fails open. The access log now carries the end date so MIT can alert before a lapse. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GoWu48siDM16978y9JKmhr
Adding a day to date.max raised OverflowError, which the ValueError handler didn't catch, so a client provisioned with the obvious "never" sentinel got a 500 on every request. The check now compares against the end of the end date instead of the start of the next. Also drives the Anywhere-on-Earth boundary through the endpoint rather than the helper, covers the access log's contract_end_date field, and reuses _unauthorized for the contract-ended refusal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GoWu48siDM16978y9JKmhr
blarghmatey
added a commit
to mitodl/ol-infrastructure
that referenced
this pull request
Sep 25, 2026
The comment said nothing checks learner_records_contract_end_date. The learner-records tenant now refuses tokens past it (mitodl/ol-analytics-api#73). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GoWu48siDM16978y9JKmhr
blarghmatey
marked this pull request as ready for review
September 25, 2026 20:21
blarghmatey
added this pull request to stack #74
September 25, 2026 20:22
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The operational contract-ended warning event lacks regression coverage.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Adds contract-end enforcement for learner-record clients using an inclusive Anywhere-on-Earth cutoff.
Changes:
- Rejects expired or malformed contract-date claims.
- Logs contract dates and expiration events.
- Documents behavior and adds boundary tests.
| File | Description |
|---|---|
token.py |
Enforces contract expiration. |
auth.py |
Adds contract dates to access logs. |
test_learner_records_token.py |
Tests expiration and date validation. |
b2b-learner-records-v1.yaml |
Documents expired-token responses. |
b2b-learner-records-provider-authorization.md |
Documents the authorization policy. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The expired-token test checked only the response, so demoting or dropping the learner_records_contract_ended warning would pass while silently disabling the alert that keys on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GoWu48siDM16978y9JKmhr
blarghmatey
added a commit
that referenced
this pull request
Sep 30, 2026
The partner's lookup and unwrap commands passed the wrapping token as a curl argument, where another local process could read it and unwrap first. They now read it without echo and pipe it on stdin. Planned removal said "on the end date", which cuts a partner off before the inclusive, Anywhere-on-Earth cutoff #73 enforces; it now deploys after 12:00 UTC the following day. Claude-Session: https://claude.ai/code/session_01S7nfyd7Pky17JRQS3fUy1C
blarghmatey
added a commit
to mitodl/ol-infrastructure
that referenced
this pull request
Oct 1, 2026
* feat(keycloak): issue per-contract learner-records clients The learner-records tenant in ol-analytics-api authorizes a caller by the organization UUIDs in its token, and access is settled when a contract is signed (docs/b2b-learner-records-provider-authorization.md). Nothing issued those tokens yet. This adds the learner-records:read client scope to the olapps realm, with an audience mapper that addresses its tokens to ol-analytics-api-client, and one client-credentials client per entry in the keycloak stack's olapps-learner-records-clients config. Each client carries its organizations as a JSON-typed hardcoded claim, since a String-typed claim grants nothing in the API. Credentials land in Vault under sso/learner-records/<name> for handoff. ol-analytics-api gets a bearer-only APISIX rule for /api/v1/learner-records on every host. It verifies tokens against the JWKS (introspection reports client-credentials tokens inactive) and requires the scope and the audience at the gateway, so a token minted for another olapps client is refused before the app sees it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rhFxFFvpxfnHNAYDyfttX * refactor(keycloak): fail when learner-records clients lack the API client Learner-records entries were skipped with no error when the olapps-ol-analytics-api-client-secret config was absent, since the clients sit inside that guard. They now raise at deploy time. Also drops a JSON round-trip for the Vault document and notes that removing the service_account scope (and its client_id claim) is intended. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rhFxFFvpxfnHNAYDyfttX * feat(ol_analytics_api): give the learner-records tenant its realm and audience The learner-records tenant now verifies the partner's bearer JWT itself instead of trusting the X-Userinfo this route forwards, because the pod is reachable without going through APISIX: the pod security group admits the whole pod subnet, and aws-eks-nodeagent runs with --enable-network-policy=false on both data clusters, so the NetworkPolicies that exist are no-ops. Without an issuer it falls back to production SSO, which is wrong everywhere but production. This templates OL_ANALYTICS_API_B2B_LEARNER_RECORDS_ISSUER and _AUDIENCE out of secret-operations/sso/ol-analytics-api, the same entry the route's openid-connect plugin reads for its discovery URL and client id, so the gateway and the app cannot end up checking different realms or a different audience. The app derives its JWKS and token URLs from the issuer. The app side is mitodl/ol-analytics-api's fix(b2b_learner_records): verify the bearer JWT instead of trusting X-Userinfo. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qoz9pfQxUU8VLsRxr1tTnm * doc(keycloak): point the contract-end claim at the API's enforcement The comment said nothing checks learner_records_contract_end_date. The learner-records tenant now refuses tokens past it (mitodl/ol-analytics-api#73). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GoWu48siDM16978y9JKmhr * style(keycloak): reflow the contract-end claim comment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GoWu48siDM16978y9JKmhr * feat(keycloak): rotate a learner-records client secret with secret_version Without a rotation knob the only way to reissue a partner's secret was replacing the client, which deletes and recreates it and every mapper on it. Bumping secret_version feeds client_secret_regenerate_when_changed, so Keycloak issues a new secret in place and the Vault document picks it up on the same apply. Claude-Session: https://claude.ai/code/session_01S7nfyd7Pky17JRQS3fUy1C * doc(keycloak): say when a rotated learner-records secret can be handed over Claude-Session: https://claude.ai/code/session_01S7nfyd7Pky17JRQS3fUy1C * fix(apisix): run the integration tests on the APISIX the chart ships APISIX_CHART_VERSION is 2.17.0, whose appVersion is 3.18.0, but the integration suite still defaulted to 3.17.0, so it tested a gateway production no longer runs. Claude-Session: https://claude.ai/code/session_01S7nfyd7Pky17JRQS3fUy1C --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What are the relevant tickets?
N/A. Stacked on #69, which adds
token.py.Description (What does it do?)
Learner-records clients carry
learner_records_contract_end_date(hardcoded from the contract in mitodl/ol-infrastructure#5939), but nothing checked it. A partner whose contract ended kept reading identifiable records until someone deleted its Keycloak client. This refuses the partner's tokens with a 401 once that date has passed.Implementation details
verified_claims, after the signature and token-class checks. It returns 401unauthorized, not 403, because the credential itself is no longer valid.contract_end_dateis set), and treating absence as expired would revoke all of them on deploy.YYYY-MM-DD(what the mapper writes withdate.isoformat()) refuses.date.fromisoformatalso accepts20270630and ISO week dates, so the parse round-trips to catch those.9999-12-31is a valid end date, not an overflow.detailnaming the date, unlike the uniform verification refusals. Only a validly signed token reaches that check, so the message tells a forger nothing, and it tells a partner why its sync broke.learner_records_contract_endedat warning level when it fires. That means a client outlived its contract and nobody removed it.learner_records_accessnow carriescontract_end_date, so a Loki alert can warn MIT ahead of a lapse. The alert rule itself isn't in this PR.docs/b2b-learner-records-provider-authorization.mdreplaces the open "does access expire with the contract?" question with this behavior. @pdpinch, that question was addressed to you, so please confirm the end-of-day reading before this merges. Partners should be told about it too.How can this be tested?
ruff check,ruff format --check,mypy src,bin/generate-openapi-spec --check, and the fullpytestsuite (340 tests) pass locally. New tests cover an expired claim (refused), a future one (accepted), an absent one (accepted), nine malformed values (refused),9999-12-31, the access-log field, and the UTC-12 boundary through the endpoint at 2027-07-01 11:59:59Z vs. 12:00:00Z. Nothing was run against a live Keycloak token.🤖 Generated with Claude Code
https://claude.ai/code/session_01GoWu48siDM16978y9JKmhr