Skip to content

feat(gateway): make List pagination token-only, drop web cursor secret - #899

Merged
behinddwalls merged 1 commit into
mainfrom
gateway-list-token-pagination
Oct 20, 2026
Merged

behinddwalls merged 1 commit into
mainfrom
gateway-list-token-pagination

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Why?

The gateway's List token already carries the queue and window, yet callers had to resend both bounds on every page. So web/submitqueue wrapped the window plus the token in a second, HMAC-signed cursor, and every deployment needed a signing secret that protected nothing the gateway does not already validate. Stovepipe's List already does this right.

What?

  • ListRequest bounds become optional (oneof, same field numbers 2/3, so the wire format is unchanged). The first page still requires both; with a page_token, the window comes from the token and any bound supplied must match it.
  • ListResponse echoes the resolved window (received_at_or_after_ms, received_before_ms), so clients can show it without decoding the token.
  • Tokens are versioned base64url JSON, checked field by field, including that the last receipt lies inside the window. Unversioned tokens still decode for one release (TODO(list-token)).
  • The Go client sends bounds on the first page only.
  • web/submitqueue links the raw gateway token as ?page= and calls list without bounds when one is present; a token the gateway rejects redirects to the live page. CursorCodec, extension/cursor (HMAC) and the cursors option are removed, along with SUBMITQUEUE_WEB_CURSOR_SECRET from the reference host, Makefile, compose and e2e harness.
  • Breaking for web hosts: GatewayReader.list takes the oneof bound shape and must return the window. A Connect client satisfies it as-is; hand-written adapters need updating, and callers must drop cursors.

Test Plan

✅ ./tool/bazel test //submitqueue/gateway/controller/... //service/submitqueue/gateway/... //submitqueue/client/... — first page needs both bounds, token-only continuation, matching/mismatched bounds, tampered/wrong-version/out-of-range/legacy tokens, proto presence round-trip
✅ ./tool/bazel test //web:check — raw token in links, token-only list on ?page=, rejected-token redirect, no CursorCodec
✅ ./tool/bazel build //... (integration and e2e suites compile); make test passes except //tool/compose:compose_test, a macOS /private/var tmp-path mismatch unrelated to this change
⚠️ Gateway integration suite (//test/integration/submitqueue/gateway:go_default_test) not run locally — needs Docker; CI covers it

🤖 Generated with Claude Code

## Summary

### Why?

The gateway's `List` token already carries the queue and window, yet callers had to resend both bounds on every page. So `web/submitqueue` wrapped the window plus the token in a second, HMAC-signed cursor, and every deployment needed a signing secret that protected nothing the gateway does not already validate. Stovepipe's `List` already does this right.

### What?

- `ListRequest` bounds become optional (`oneof`, same field numbers 2/3, so the wire format is unchanged). The first page still requires both; with a `page_token`, the window comes from the token and any bound supplied must match it.
- `ListResponse` echoes the resolved window (`received_at_or_after_ms`, `received_before_ms`), so clients can show it without decoding the token.
- Tokens are versioned base64url JSON, checked field by field, including that the last receipt lies inside the window. Unversioned tokens still decode for one release (`TODO(list-token)`).
- The Go client sends bounds on the first page only.
- `web/submitqueue` links the raw gateway token as `?page=` and calls `list` without bounds when one is present; a token the gateway rejects redirects to the live page. `CursorCodec`, `extension/cursor` (HMAC) and the `cursors` option are removed, along with `SUBMITQUEUE_WEB_CURSOR_SECRET` from the reference host, Makefile, compose and e2e harness.
- Breaking for web hosts: `GatewayReader.list` takes the `oneof` bound shape and must return the window. A Connect client satisfies it as-is; hand-written adapters need updating, and callers must drop `cursors`.

## Test Plan

✅ `./tool/bazel test //submitqueue/gateway/controller/... //service/submitqueue/gateway/... //submitqueue/client/...` — first page needs both bounds, token-only continuation, matching/mismatched bounds, tampered/wrong-version/out-of-range/legacy tokens, proto presence round-trip
✅ `./tool/bazel test //web:check` — raw token in links, token-only `list` on `?page=`, rejected-token redirect, no `CursorCodec`
✅ `./tool/bazel build //...` (integration and e2e suites compile); `make test` passes except `//tool/compose:compose_test`, a macOS `/private/var` tmp-path mismatch unrelated to this change
⚠️ Gateway integration suite (`//test/integration/submitqueue/gateway:go_default_test`) not run locally — needs Docker; CI covers it
@behinddwalls
behinddwalls marked this pull request as ready for review October 20, 2026 21:38
@behinddwalls
behinddwalls requested review from a team and sbalabanov as code owners October 20, 2026 21:38
@behinddwalls
behinddwalls added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit 8294e24 Oct 20, 2026
38 checks passed
@behinddwalls
behinddwalls deleted the gateway-list-token-pagination branch October 20, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants