Skip to content

feat(wal): persist local Raft checkpoint boundaries - #402

Merged
beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/wal-checkpoint-storage
Oct 20, 2026
Merged

beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/wal-checkpoint-storage

Conversation

@beinan

@beinan beinan commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

GroupLog currently retains its entire committed prefix. Add a local checkpoint boundary that persists a bounded application descriptor, retained term ranges and logical prefix compaction in one synced engine batch. Reopen preserves HardState, the boundary term and the committed/uncommitted suffix; uncertain writes fence every group sharing the engine.

The caller must validate and durably store the application snapshot before installing its descriptor. This storage API does not supply payload packs, remote snapshot installation, physical reclamation or Lance publication. The replication Node does not invoke it yet.

Refs #378.

Testing

  • 33 focused Rust tests passed, including real child-process SIGKILL, invalid/corrupt checkpoint recovery and injected sync failure; optional-feature Clippy passed.
  • Workspace formatting, Clippy and 605 Rust tests passed. Python: 229 passed, 2 optional skips, 1 existing xfail, with the public HTTP server built and CI mode enabled; Ruff and Pyright passed.
  • Spelling and workspace packaging checks passed. The initial local helper stopped because the new environment lacked test extras; after installing the declared extras, its remaining Python checks passed on unchanged source.

These tests establish retained-storage process recovery, not power-loss durability or throughput.

@beinan
beinan merged commit 0b496c5 into lance-format:main Oct 20, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant