Skip to content

perf(mobile): thread list resumes from the saved cache, and large gaps reload over gzip HTTP - #32287

Open
juliusmarminge wants to merge 1 commit into
mainfrom
t3/shell-cold-resume
Open

juliusmarminge wants to merge 1 commit into
mainfrom
t3/shell-cold-resume

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

On a cold start every client downloaded the full thread list, even when its saved copy was a few events behind. When a client did resume from a cursor beyond the server's replay cap (1,000 events, 8 MB), the server answered with a full snapshot over the socket. iOS does not compress that, so it arrived at 1.37 MB instead of the 88 KB the gzip HTTP route costs.

Fix

  • Resume from the saved cache. The shell cache now records the snapshot its rows came from as origin: server version, database migration, omission flags, and snapshotAt. A client whose cache matches its subscription and is under 12 hours old subscribes with afterSequence instead of loading a fresh snapshot. Replay never renews snapshotAt, so drift stays bounded.
  • snapshot-required. A resuming client sends httpSnapshotFallback: true. If the server cannot replay that cursor (over the gap cap or pruned), it answers with one snapshot-required item and doesn't send the snapshot. The client then loads the gzip HTTP snapshot and subscribes again from it.
    • A cursor from a snapshot that was just loaded never sends the flag, so a refusal cannot loop.
    • Old servers ignore the flag, and old clients never send it.
  • Items buffered from a superseded session are dropped, and a resume that catches up bumps generation.

Scope and approval

Maintainer-authored. It replaces https://git.cafe/pingdotgg/t3code/pulls/30732 at the maintainers' direction, after measuring replay against a fresh snapshot. Layer 1 of 2; https://git.cafe/pingdotgg/t3code/pulls/32288 builds on it.

Measurements

Measured on a real database with 2,279 threads:

path wire server CPU client CPU (JIT / interpreter)
HTTP snapshot (gzip) 88.5 KB ~210 ms 11.6 / 40 ms
replay ≤ 1,000 events (deflate) 7 KB ~125 ms 0.6 / 3.2 ms
over-cap snapshot over the iOS socket (before) 1.37 MB ~160 ms —

Replay beats a snapshot on every metric up to the cap. Past roughly 1,200 dense events it costs about as much as a snapshot, which is why the cap stays at 1,000.

Real history decides how often the cap is hit: replay covers 96% of returns after 5 minutes away, 50% after 30 minutes, 25% after 2 hours, and 4% after 8 hours. Large gaps were the common case for heavy users, and those now reload over gzip HTTP.

Verification

  • Tests:
    • shell-sync.test.ts gains a resume table and a snapshot-required case, covering cold and warm. Reverting the loop guard fails 6 cases.
    • ws.test.ts gains a pruned-boundary row.
    • The cache-store test covers the origin round-trip.
  • Web, against a copy of a 2.1 GB real database:
    • Reloading made no shell snapshot request.
    • On a warm reconnect against pruned history, the trace showed a refused subscribe (13 ms), then one HTTP snapshot (112 ms), then a resubscribe without the flag.

Web client after a warm reconnect over pruned history: thread list intact

Credit

This replaces https://git.cafe/pingdotgg/t3code/pulls/30732 by @Bil0000 and builds on https://git.cafe/pingdotgg/t3code/pulls/30317 by @emmems. Both found the cold-start cost and the uncompressed socket fallback.

Built by Claude Fable 5.1 and Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

…s reload over gzip HTTP

On a cold start every client downloaded the full thread list even when its
saved copy was a few events behind. The shell cache now records the server
snapshot its rows descend from (`origin`: server version, database
migration, omissions, and when the snapshot was taken), and a client whose
cache matches the current subscription and is under 12 hours old resumes
from the saved cursor instead. Replay never renews `snapshotAt`, so drift
from skipped older-settled cuts and rebuilt projections stays bounded.

When the gap is over the server's replay cap, the server used to answer the
resume with a full snapshot over the socket, which iOS does not compress.
A client that sends `httpSnapshotFallback` with `afterSequence` now gets a
single `snapshot-required` item instead; it drops the origin, loads the
gzip HTTP snapshot, and subscribes again from its cursor. Old servers ignore
the flag and old clients never send it.

Items buffered from a superseded session are dropped rather than applied to
the next session's rows, and a resume that catches up bumps `generation`
so thread families and search stop waiting for a snapshot.

Thanks to @Bil0000 for #30732 and @emmems for #30317, which found the
cold-start cost and the uncompressed socket fallback and shaped this fix.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Nov 7, 2026
@macroscopeapp

macroscopeapp Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 460a65b

Macroscope's review found this PR approvable — Thread lists resume from recent, compatible cached snapshots and reload unreplayable gaps through HTTP. Provenance checks, a 12-hour age limit and authoritative snapshot fallback keep the synchronization change bounded.

You can add or adjust custom eligibility rules. Learn more.

@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 21.5 KiB 21.5 KiB 0 B (0.0%) 24.4 KiB ✅
Codex Live turn messages 1 1 0 (0.0%) 8 ✅
Claude Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 21.9 KiB 21.9 KiB 0 B (0.0%) 24.4 KiB ✅
Claude Live turn messages 1 1 0 (0.0%) 8 ✅

Baseline: 23bdb9e · PR result: 460a65b · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Shell cache records now include origin metadata. The client uses this metadata to decide whether to resume a cached snapshot. When a resume requires a snapshot, the server can signal the client to load one over HTTP and resubscribe.

Changes: Shell resume and snapshot fallback

Layer / File(s) Summary
Resume contracts and cache metadata
packages/contracts/src/orchestrationV2.ts, packages/client-runtime/src/platform/orchestrationCache.ts, packages/client-runtime/src/platform/persistence.ts, apps/mobile/src/connection/environment-cache-store.ts, apps/mobile/src/connection/environment-cache-store.test.ts, apps/web/src/connection/storage.ts
The shell stream contract adds snapshot-required and the optional httpSnapshotFallback input. Cache types and mobile and web stores support optional origin metadata. The mobile test checks that the metadata survives a save and load.
Server snapshot fallback
apps/server/src/ws.ts, apps/server/src/ws.test.ts
When a shell resume requests HTTP snapshot fallback, the server emits snapshot-required instead of loading and streaming a shell snapshot. Tests cover both fallback and the existing snapshot behavior.
Client cache validation and reload
packages/client-runtime/src/state/shell.ts, packages/client-runtime/src/state/shellReducer.ts, packages/client-runtime/src/state/shell-sync.test.ts
The client resumes cached rows only when origin identity matches and the snapshot is less than 12 hours old. It handles snapshot-required by reloading over HTTP, updates cache origin metadata, and filters stream items by active subscription. Tests cover eligible and ineligible cached origins and cold and warm fallback paths.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant ShellSync
  participant subscribeOrchestrationV2Shell
  participant HTTPSnapshotLoader
  participant EnvironmentCacheStore
  ShellSync->>subscribeOrchestrationV2Shell: Subscribe with cursor and HTTP fallback
  subscribeOrchestrationV2Shell-->>ShellSync: Emit snapshot-required
  ShellSync->>HTTPSnapshotLoader: Load authoritative snapshot
  HTTPSnapshotLoader-->>ShellSync: Return snapshot
  ShellSync->>EnvironmentCacheStore: Save snapshot with origin
  ShellSync->>subscribeOrchestrationV2Shell: Resubscribe from snapshot sequence
Loading

Possibly related PRs

Suggested reviewers: bil0000, saphid, t3dotgg

Merge Risk: Minimal · up to 460a6

The change lets cold starts resume from the saved cache and falls back to the gzip HTTP snapshot when replay is not possible. No concrete merge-blocking defect was established.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the two main changes: the thread list resumes from the saved cache on cold start, and large replay gaps reload over gzip HTTP. The raw summary confirms both: cache origin with afterSeq…
Description check Passed The description covers the problem, the fix, scope and approval, measurements, and verification, which matches the template's sections. It explains the cold-start cost, the cache origin and snapshot-r…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the two main changes: the thread list resumes from the saved cache on cold start, and large replay gaps reload over gzip HTTP. The raw summary confirms both: cache origin with afterSequence resume, and the snapshot-required fallback that loads an HTTP snapshot. The title follows the conventional commit format and is specific to the changeset.

Full details: Description check

Explanation

The description covers the problem, the fix, scope and approval, measurements, and verification, which matches the template's sections. It explains the cold-start cost, the cache origin and snapshot-required mechanism, and the loop guard. It links the superseded PRs and gives concrete test and manual verification results. The template asks for a conventional commit title and a maintainer approval link. The author states the work is maintainer-authored and gives a reason for scope, but the Scope section does not link a triaged issue or a maintainer approval comment, so it is only partly complete. The Problem section is also folded into the opening paragraph rather than headed separately. The Verification section includes a screenshot and a trace, so the UI-change requirement is met. Overall the description is mostly complete, so the check passes.

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 23bdb9e and 460a65b.

Files selected for processing: 11

  • apps/mobile/src/connection/environment-cache-store.test.ts
  • apps/mobile/src/connection/environment-cache-store.ts
  • apps/server/src/ws.test.ts
  • apps/server/src/ws.ts
  • apps/web/src/connection/storage.ts
  • packages/client-runtime/src/platform/orchestrationCache.ts
  • packages/client-runtime/src/platform/persistence.ts
  • packages/client-runtime/src/state/shell-sync.test.ts
  • packages/client-runtime/src/state/shell.ts
  • packages/client-runtime/src/state/shellReducer.ts
  • packages/contracts/src/orchestrationV2.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor
Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant