Skip to content

fix: preserve completed read snapshots during WAL merges - #396

Merged
beinan merged 1 commit into
mainfrom
codex/consistent-list-snapshot
Oct 19, 2026
Merged

beinan merged 1 commit into
mainfrom
codex/consistent-list-snapshot

Conversation

@beinan

@beinan beinan commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Continuous WAL merges can make a completed listing retry all its decoding merely because a newer base version was committed. After three such attempts, the request fails despite having read a complete snapshot. Separately, missing WAL data can surface as Wrapped error: Not found, which the case-sensitive missing-file check fails to recognize, causing HTTP 500 instead of refreshing the read view.

Validate the base version around WAL manifest capture, before payload reads. This still rejects a base/WAL pair split by a merge and retries missing generations, but accepts a successfully read captured snapshot even if another commit follows it. Recognize wrapped missing-file messages regardless of capitalization. Pinned reads, pending-generation caps, and the bounded retry count are retained; missing generations never produce partial results.

Evidence and validation:

  • Isolated Azure concurrent LIST/merge testing reproduced errors on both the current production reader and the prior candidate: three HTTP 500s in 27 reads on 7255…, two HTTP 404s in 32 reads on fe08…. A repeat on fe08… captured a missing _mem_wal/.../data/*.lance file wrapped as Not found. These tests do not establish that every production slow read has this cause.
  • A deterministic regression failed before the change with dataset changed during read; retry the request: each successful snapshot read was followed by a later write/merge before returning.
  • Added regressions for retaining a completed snapshot and retrying the observed wrapped missing-generation error. Local checks passed: 17 list-related tests (one unrelated etcd-dependent test skipped), the bounded-retry/pinned-read contract, and generic batch-read contracts.
  • Fixed-image Azure concurrent-read qualification remains required before production rollout. The earlier merge-then-readback and handoff tests do not cover this concurrency window.

@beinan
beinan merged commit d6ca44b into main Oct 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant