Skip to content

feat(wal): persist publisher bindings and ownership epochs - #405

Merged
beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/wal-publisher-authority
Oct 20, 2026
Merged

beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/wal-publisher-authority

Conversation

@beinan

@beinan beinan commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

A restarted publisher needs to recover its table binding and distinguish its ownership epoch from an earlier owner's. Add immutable destination/ingestion-contract bindings and expected-epoch acquisition through the catalog group's durable Raft commit path. Exact retries of the current grant return the original grant index after consensus; a local applied view cannot authorize an isolated owner.

Application checkpoints preserve the bounded catalog alongside accepted data and producer tails, without retaining old control payloads in data packs. This changes the optional experimental application identity and image format to v2; v1 directories are rejected. Enforcing these epochs at Lance publication, binding data admission to the catalog, source retirement and production integration remain separate work.

Refs #378.

Testing

  • 55 focused Rust tests passed, including follower sync-failure fencing and real TCP processes exercising the quorum boundary, leader loss, exact grant retries, isolated ownership, snapshot transfer and full restart.
  • Mixed catalog/data checkpoint tests reject reordered or duplicate tables, duplicate destinations and grants beyond the cut, including with recomputed outer checksums.
  • All-feature WAL Clippy and changed-source spelling checks passed.
  • Complete repository CI helper passed: workspace Rust formatting, Clippy and tests; Python tests (229 passed, 2 skipped, 1 expected failure), Ruff and Pyright. Server build and workspace packaging passed; source and server checksums remained unchanged.

@beinan
beinan merged commit b69ca80 into lance-format:main Oct 20, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant