Skip to content

Execute WAL catch-up inside resident masters - #326

Merged
beinan merged 1 commit into
mainfrom
codex/master-local-wal-merge
Oct 7, 2026
Merged

beinan merged 1 commit into
mainfrom
codex/master-local-wal-merge

Conversation

@beinan

@beinan beinan commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

WAL catch-up currently needs remote staging workers or a separate one-table executor. Add an explicit resident-master mode that reads and encodes WAL locally through the existing durable MergeWal queue and fenced publisher. Additional ordinary master replicas can share the queue without creating per-table Kubernetes Jobs.

ROLLOUT_APPEND_LOCAL applies only to opted-in owned append-only rollouts. Staging shares one process-wide Arrow budget and cache session; table concurrency uses the existing separate merge pool. Startup rejects missing shard identities, inadequate budgets, a shared general/merge pool, and conflicting Kubernetes catch-up configuration. Low-count tail scheduling also works without worker endpoints. Defaults retain the existing remote path.

Existing old-writer guards, recovery fences and retry policy remain in force. Switching this flag does not adopt protected external publishers. Memory reservations are not a process RSS cap; oversized indivisible generations and other master work still require headroom.

Validation: all 174 master tests pass against local etcd, including two replicas sharing the queue without workers and a saturated resident merge budget while general indexing completes. All GitHub checks pass, including Rust workspace tests, coverage, Clippy and Python tests/builds. Production has not enabled resident local staging; the pinned release will be qualified on isolated staging before publisher handoff.

@beinan
beinan marked this pull request as ready for review October 7, 2026 18:29
@beinan
beinan merged commit cabc6cf into main Oct 7, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant