From 98b118b48b8312e92399a54c20994fa7f3e04a0d Mon Sep 17 00:00:00 2001 From: Adam Wright Date: Sat, 19 Sep 2026 06:03:51 +0000 Subject: [PATCH 1/2] Settle the analysis-summary contract, and fix what `sub` actually means Agreed with the website session. Nothing in the contract is open now. Human presence rides as `human`, `human_iat` and `human_sub` on the caller token, minted only when their Turnstile-backed identity cookie validated. Freshness is 30 minutes, inclusive at exactly 1800.000s, enforced at both ends so neither side is a single point of failure. The analysis token is passed straight through and never validated by the caller: this service already distinguishes 404, 410 and the undocumented 500 a malformed token returns, and a second validator disagrees with the first eventually. A refusal now carries a `reason` -- `no_caller`, `no_human` or `stale_human` -- because "failed" is not something an interface can say to a person, and `stale_human` is the only refusal a reader can act on. Advisory: an unrecognised value must read as a plain refusal. The claim is `human_sub` rather than `sub` because `sub` was already taken. And then the more useful correction, which came back from them: this repository's description of `sub` was wrong. Their `callerSubject()` prefers the identity cookie's subject whenever the reader has passed a challenge, falling back to a per-visit value only when they have not -- so `sub` has been browser-scoped for every gated reader since the gate shipped, and the answer endpoint's backstop limiter has been counting people across visits rather than within one. That is the stronger throttle and it is kept deliberately. What was wrong was the docstring in caller_token.py and the comment in rate_limit.py, both of which named the fallback case as though it were the only case. Someone reasoning about a burst on that endpoint would have had the wrong model, and the analysis-summary limiter would have looked inexplicably similar to one they thought was scoped differently. Also recorded: while that preference holds, `sub` and `human_sub` carry the same value whenever both are present. They stay separate because they mean different things and would diverge the moment it changed, and because agreement between them is not a signal anything should test. Documentation and comments only; no behaviour change. Co-Authored-By: Claude Opus 5 --- .../contracts/summary_endpoint.md | 54 ++++++++++++++++--- .../research.md | 11 +++- src/util/caller_token.py | 13 ++++- src/util/rate_limit.py | 12 +++-- 4 files changed, 75 insertions(+), 15 deletions(-) diff --git a/specs/011-summarise-analysis-results/contracts/summary_endpoint.md b/specs/011-summarise-analysis-results/contracts/summary_endpoint.md index c0b241e..7e3e68f 100644 --- a/specs/011-summarise-analysis-results/contracts/summary_endpoint.md +++ b/specs/011-summarise-analysis-results/contracts/summary_endpoint.md @@ -1,7 +1,8 @@ # Contract: the analysis summary endpoint -Draft. Not implemented, and not yet agreed with the website repo. The parts -marked **open** need their agreement before anything is built against them. +Not implemented. **Agreed with the website repo on 2026-09-19** -- the request +shape, the refusal shape and how human presence is asserted are settled, and +they have the presence claims built on a branch. Nothing here is open. ## Request @@ -14,15 +15,54 @@ Content-Type: application/json "disclosure": "aggregate" } ``` +**The analysis token is passed straight through, never validated by the +caller.** This service already distinguishes 404, 410 and the undocumented 500 +a malformed token returns (research D2). A second validator on the website +would be a second thing that believes it knows what a good token looks like, +and two validators disagree eventually. + `disclosure` is `aggregate` or `identifiers`, and is **required** — there is no default, because a default is not a choice. `aggregate` never transmits the user's identifiers, filenames, sample names or expression column labels. -**Open**: how the caller demonstrates a person is present. Spec 010's D1 settled -that `caller_token` asserts service identity and says nothing about humanity, so -it cannot carry this on its own. The likely shape is an additional claim minted -after the Turnstile check the website already performs on the chat, but that is -theirs to agree. +### Human presence + +`caller_token` carries three additional claims, minted only when the website's +Turnstile-backed identity cookie validated on that request: + +| claim | meaning | +|---|---| +| `human` | `true`. **Absent otherwise, never `false`** -- a missing claim and a failed check are indistinguishable here | +| `human_iat` | epoch seconds, when the challenge was solved | +| `human_sub` | the cookie's random 16-byte subject, for per-person rate limiting | + +**Freshness is 30 minutes and the bound is inclusive**: a challenge solved +exactly 1800.000s ago is accepted, 1800.001s is not. Both sides enforce it -- +the website refuses to mint past it, this service refuses to accept past it -- +so neither is a single point of failure, and if it is ever changed it is +changed in both places in one change. + +**`human_sub`, not `sub`.** `sub` already means something here: an opaque +*per-visit* id, which `identity_of` uses to key the answer endpoint's backstop +limiter. The cookie subject is also 16 bytes but is per-*browser* and lives as +long as the cookie. Putting it in `sub` would silently change that limiter from +"this visit" to "this browser", on an endpoint neither repo is otherwise +touching. A separate claim also makes it explicit that a persistent +pseudonymous identifier is being held -- hashed, in memory only. + +**And the model of `sub` above was itself wrong.** The website corrected it on +2026-09-19: their `callerSubject()` prefers the identity cookie's subject +whenever the reader has passed a challenge, falling back to the per-visit value +only when they have not. So `sub` has *already* been browser-scoped for every +gated reader, and the answer endpoint's backstop limiter has been counting +across visits since the gate shipped. That is the stronger throttle and it is +kept; the source comments describing it as per-visit are fixed. + +A consequence to write down: while `callerSubject()` prefers the verified +identity, `sub` and `human_sub` carry the **same value** whenever both are +present. They are still separate claims, because they mean different things and +would diverge the moment that preference changed -- and because agreement +between them is not a signal anything should test. ## Response: Server-Sent Events diff --git a/specs/011-summarise-analysis-results/research.md b/specs/011-summarise-analysis-results/research.md index 341eb92..685bb04 100644 --- a/specs/011-summarise-analysis-results/research.md +++ b/specs/011-summarise-analysis-results/research.md @@ -139,9 +139,16 @@ on the request: |---|---| | `human` | `true`, set only when a valid, unexpired identity cookie was presented. **Absent otherwise, never `false`**, so a missing claim and a failed check are indistinguishable to us | | `human_iat` | when the challenge was solved, epoch seconds. Derived from the cookie's expiry minus their identity TTL; no cookie format change | -| `subject` | the cookie's random 16-byte identifier, for per-identity rate limiting. Carries nothing about the person | +| `human_sub` | the cookie's random 16-byte identifier, for per-identity rate limiting. Carries nothing about the person | -**Freshness is 30 minutes, enforced at both ends.** We refuse a `human_iat` +**Named `human_sub`, not `sub`.** `sub` is already an opaque *per-visit* id +that `identity_of` uses for the answer endpoint's backstop limiter. The cookie +subject is per-*browser* and lives as long as the cookie, so reusing the claim +would change that limiter's meaning on an endpoint neither repo is touching -- +a behaviour change arriving through a rename. Caught before either side built +to it. + +**Freshness is 30 minutes, inclusive (1800.000s accepted, 1800.001s not), enforced at both ends.** We refuse a `human_iat` older than that, and they refuse to mint the claim past it, so neither side is a single point of failure. Long enough that reading a result, choosing a disclosure tier and requesting a summary is never re-challenged; short enough diff --git a/src/util/caller_token.py b/src/util/caller_token.py index 3e4c949..5f42a42 100644 --- a/src/util/caller_token.py +++ b/src/util/caller_token.py @@ -9,8 +9,17 @@ What it does assert is **caller identity**: this request came from the Reactome website's server, for one visit. They mint server-side per request, EdDSA, with -`iss`, `aud`, `iat`, `exp` at +120s, and `sub` -- an opaque per-visit id that is -128 random bits, not derived from anything about the reader. Abuse control is +`iss`, `aud`, `iat`, `exp` at +120s, and `sub` -- 128 random bits, not derived +from anything about the reader. + +**`sub` is browser-scoped for a gated reader, not per-visit**, and this said +otherwise until 2026-09-19. Their `callerSubject()` prefers the Turnstile +identity cookie's subject whenever the reader has passed a challenge, and falls +back to a per-visit cookie only when they have not. So the backstop limit below +has been keyed on an identifier lasting as long as that cookie -- the stronger +throttle, and the behaviour in production since the gate shipped. It is kept +deliberately; what was wrong was this description of it, which named the +fallback as though it were the only case. Abuse control is theirs: the panel is opt-in behind a click, so a crawled search never reaches a model, and their proxy rate limits by address. diff --git a/src/util/rate_limit.py b/src/util/rate_limit.py index 6d0d303..0b0b321 100644 --- a/src/util/rate_limit.py +++ b/src/util/rate_limit.py @@ -30,10 +30,14 @@ def _positive_int(name: str, default: int) -> int: def identity_of(claims: dict[str, object], token: str) -> str: """Who to count against. - The token's claims are D1 and not yet settled with the website, so `sub` may - never arrive. `sub` then `jti` are used when present, so this starts keying on - a real person the moment D1 lands; until then a hash of the token itself is - the best available proxy -- one issuance, short lived, one person. + `sub` then `jti` when present, else a hash of the token itself. + + D1 is settled now: `sub` does arrive, and for a reader who has passed the + Turnstile challenge it is the identity cookie's subject -- browser-scoped + for the life of that cookie, not per-visit. So this counts a person across + visits rather than within one, which is the stronger backstop and is what + has been running since the gate shipped. Worth knowing before reasoning + about what a burst here means. Hashed, never raw: this lands in a dict that lives as long as the process, and a bearer token is a credential. From 2b0bf6a4b16100dfad81a3f87e146ccef47e3af4 Mon Sep 17 00:00:00 2001 From: Adam Wright Date: Sat, 19 Sep 2026 06:15:12 +0000 Subject: [PATCH 2/2] Adversarial review: we agreed a millisecond edge on a whole-second claim The bound was written as "1800.000 accepted, 1800.001 refused", and both sides were about to pin a test at exactly that edge. It is not a distinction the claim can carry. `human_iat` is epoch seconds, derived as the cookie's expiry minus a constant TTL, so it arrives already rounded to a second -- a sub-second edge is a boundary neither side can be on, tested against a clock finer than the value being tested. Restated as `now - human_iat <= 1800` in whole seconds, with the consequence written down: effective precision is one second, and a challenge solved 1800.4s ago presents as 1800 and is accepted. The point they made about boundaries -- that one nobody tests is one nobody agreed on -- holds just as well for a boundary tested at a precision the data does not have. Also added while looking: the summary must not end with a prose list of its citations. They arrive as `citation` events, and the duplicate is exactly what cost the website a pattern it could not write correctly on the answer route. This endpoint's prompt is its own, so the fix is not to ask for the list rather than to strip it afterwards. Co-Authored-By: Claude Opus 5 --- .../contracts/summary_endpoint.md | 20 +++++++++++++++++-- .../research.md | 4 ++-- 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/specs/011-summarise-analysis-results/contracts/summary_endpoint.md b/specs/011-summarise-analysis-results/contracts/summary_endpoint.md index 7e3e68f..9b196de 100644 --- a/specs/011-summarise-analysis-results/contracts/summary_endpoint.md +++ b/specs/011-summarise-analysis-results/contracts/summary_endpoint.md @@ -36,8 +36,18 @@ Turnstile-backed identity cookie validated on that request: | `human_iat` | epoch seconds, when the challenge was solved | | `human_sub` | the cookie's random 16-byte subject, for per-person rate limiting | -**Freshness is 30 minutes and the bound is inclusive**: a challenge solved -exactly 1800.000s ago is accepted, 1800.001s is not. Both sides enforce it -- +**Freshness is 30 minutes and the bound is inclusive**: `now - human_iat <= 1800`, +so a challenge solved exactly 1800s ago is accepted and 1801s is not. + +Stated in whole seconds on purpose. The bound was first agreed as "1800.000 +accepted, 1800.001 refused", which is not a distinction this claim can carry: +`human_iat` is epoch **seconds**, and it is derived as the cookie's expiry +minus a constant TTL, so it arrives already rounded to a second. A sub-second +edge would be a boundary neither side can actually be on, tested against a +clock finer than the value. Effective precision is one second, and a challenge +solved 1800.4s ago presents as 1800 and is accepted. + +Both sides enforce it -- the website refuses to mint past it, this service refuses to accept past it -- so neither is a single point of failure, and if it is ever changed it is changed in both places in one change. @@ -87,6 +97,12 @@ data: {"state": "summarised", "seconds": 6.2} `failed`. Anything but `summarised` means render no summary. Always HTTP 200 — never an error code, so the analysis page cannot be broken by this service. +**No prose source list.** Citations arrive as `citation` events, as on +`/api/answer`, and the summary text must not end with a list of them -- +`SourcesSectionStripper` exists because that duplicate cost the website a +pattern it could not write correctly. This endpoint's prompt is its own, so +the right fix here is not to ask for one in the first place. + `cached` on `start` says whether this text was generated now or reused. It exists because the interface must not imply determinism it does not have: a reader who regenerates may get different wording, and `cached: false` is when that happens. diff --git a/specs/011-summarise-analysis-results/research.md b/specs/011-summarise-analysis-results/research.md index 685bb04..ee14b81 100644 --- a/specs/011-summarise-analysis-results/research.md +++ b/specs/011-summarise-analysis-results/research.md @@ -138,7 +138,7 @@ on the request: | claim | meaning | |---|---| | `human` | `true`, set only when a valid, unexpired identity cookie was presented. **Absent otherwise, never `false`**, so a missing claim and a failed check are indistinguishable to us | -| `human_iat` | when the challenge was solved, epoch seconds. Derived from the cookie's expiry minus their identity TTL; no cookie format change | +| `human_iat` | when the challenge was solved, epoch **seconds**. Derived from the cookie's expiry minus their identity TTL, so it arrives rounded to a second -- the bound is whole-second, and a sub-second edge is not a state this claim can represent | | `human_sub` | the cookie's random 16-byte identifier, for per-identity rate limiting. Carries nothing about the person | **Named `human_sub`, not `sub`.** `sub` is already an opaque *per-visit* id @@ -148,7 +148,7 @@ would change that limiter's meaning on an endpoint neither repo is touching -- a behaviour change arriving through a rename. Caught before either side built to it. -**Freshness is 30 minutes, inclusive (1800.000s accepted, 1800.001s not), enforced at both ends.** We refuse a `human_iat` +**Freshness is 30 minutes, inclusive in whole seconds (`now - human_iat <= 1800`), enforced at both ends.** We refuse a `human_iat` older than that, and they refuse to mint the claim past it, so neither side is a single point of failure. Long enough that reading a result, choosing a disclosure tier and requesting a summary is never re-challenged; short enough