Skip to content

fix: measure WAL component disk footprint from files - #397

Merged
beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/wal-disk-accounting
Oct 19, 2026
Merged

beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/wal-disk-accounting

Conversation

@beinan

@beinan beinan commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

context-wal-lab disk reported raft-engine's segment-size estimate as disk_bytes. Batches can overshoot the target segment size, so that estimate can understate the actual WAL footprint.

Measure the closed engine's regular files outside write/reopen timing, report apparent and allocated sizes, and retain the engine estimate separately. disk_bytes now sums each file's larger apparent/allocated size. These fields measure directory footprint, not physical bytes written or write amplification.

Refs #378.

Testing

  • Real raft-engine regression with synced batches larger than the target segment size; added to CI.
  • Release lab tests, feature-enabled Clippy, and CLI disk/reopen/full-payload smoke passed.
  • Workspace formatting, Clippy and all-target tests passed (603 tests; 154 ignored by default).
  • Python: 220 passed, 11 skipped, one expected failure; Ruff and Pyright passed. Installed the declared test/dev extras to complete the helper's Python checks.

@beinan
beinan merged commit d623ad0 into lance-format:main Oct 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant