[Bug]: Web browser sessions race refresh across requests and replicas #1148

Problem

A browser session can lose or intermittently fail authenticated Web requests when its access token expires and multiple requests reach the BFF at once. Each request constructs an independent WebRuntimeClient with the same refresh-token cookie. The Astra refresh token is single-use, so only one instance can rotate it; the others receive 401. With multiple Web replicas, process-local coordination cannot prevent this race.

There is a second lifecycle boundary: Web Server Components can invoke authenticated runtime requests, but Next.js does not allow them to mutate cookies during rendering. If an access token is refreshed there, failing to persist the rotated pair leaves the browser holding a consumed refresh token. Delayed responses can also overwrite a newer token cookie.

Reproduction and evidence

1. Start two independent clients with the same expired access token and valid refresh token.
2. Issue requests concurrently against a service that consumes the refresh token on the first successful rotation.
3. Both clients attempt refresh; one request succeeds and the other receives 401.

An isolated SDK/HTTP probe against the implementation reviewed in #1133 reproduced two refresh attempts with one success and one failure. This is a mock-based reproduction, not a claim about measured production frequency. The same instance boundary is visible in getRuntimeClient. The sidebar and authentication actions also contain separate refresh paths:

• web/app/api/sidebar/route.ts
• web/lib/auth/actions.ts

The canonical server refresh transaction locks the user and atomically compare-revokes the old token before inserting its successor. It stores token hashes, not recoverable successor credentials. A lock alone cannot supply the winning pair to other requests. #1133 fixes coordination within one SDK instance, but deliberately does not claim to fix this browser-wide / multi-replica problem. Related: #1132.

Desired behavior

One browser session should remain usable when many requests, tabs, or Web replicas encounter token expiry together. Each request should either use the current authorized session or receive a clear failure; a losing refresh attempt must not randomly produce an offline sidebar, failed chat, or an unrecoverable stale cookie. Server-rendered pages should not need to mutate cookies in a read-only rendering context.

Design constraints

• Keep authentication, revocation, and rotation owned by the canonical server authentication service. Do not add a parallel auth state machine or a process-local token cache presented as a multi-server fix.
• Preserve single-use refresh-token replay protection. Do not return a successor merely because a caller presents a consumed refresh token.
• Handle concurrent refresh, logout, account replacement, provider disconnect, token expiry, database or provider outage, and delayed/lost responses without resurrecting a revoked session or switching accounts.
• Avoid holding a database lock across network/provider calls, unbounded scans, and expensive per-request work. Measure the common authenticated request path and use selective indexed lookups.
• If a shared browser-session owner is necessary, consider a stable opaque HttpOnly handle and existing encrypted-credential infrastructure, with token rotation and owner update committed atomically. Bind the handle to the selected runtime; establish expiry, cleanup, key management, deployment ordering, and a safe schema-evolution plan. This is a design candidate, not a requirement to introduce a particular table.
• Account for Astra's strict core schema-version contract: a core version bump currently rejects existing databases. The schema framework supports independently versioned components, which may allow an additive, explicitly initialized and validated owner table without changing the core version. Do not assume CREATE TABLE IF NOT EXISTS alone provides a safe rollout. The existing auth_tokens table can hold ciphertext, but its unpaginated admin listing and lack of browser-session expiry indexes make direct reuse a potential scaling and lifecycle problem.

Acceptance and validation

• Two independent Web/server instances share one browser session; simultaneous expiry yields successful authorized requests without competing destructive rotations. Verify a delayed request and more than two contenders.
• Exercise real Next Server Component rendering with an expired access token and confirm no cookie write is attempted during render.
• Verify both refresh/logout orderings, new login versus delayed old responses, other-session isolation, lost response after commit, rollback before commit, invalid/expired handles, credential-key mismatch, and backend/provider failures.
• Confirm direct replay of a consumed backend refresh token remains rejected. Test every relevant login, logout, refresh, sidebar, runtime-config, REST, chat stream, and Work stream path.
• Check indexed queries and lock lifetimes under concurrency; retain deterministic offline coverage and an isolated real-database multi-instance test.