Skip to content

feat: define immutable WAL ingestion contracts - #384

Merged
beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/wal-ingestion-contract
Oct 19, 2026
Merged

beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/wal-ingestion-contract

Conversation

@beinan

@beinan beinan commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Ingress and materializers need the same immutable routing keys, schema identity, write policy and record limits before accepting a table generation. Add an independent contract feature with serializable metadata, explicit validation errors and a domain-separated schema fingerprint. It requires neither Arrow nor a replication engine. Callers still enforce durable binding, validate actual payloads and provide authentication and ID uniqueness.

Fixed synthetic binary/hash fixtures preserve the metadata representation. CI exercises the optional feature. Production ingestion is unchanged.

Refs #378.

Testing

  • Contract tests cover a fixed bincode representation, unknown policy rejection, independently checked schema hashing, UTF-8 byte bounds and invalid decoded limits.
  • Optional-feature tests and release Clippy passed.
  • Full repository check script passed: workspace formatting, Clippy and all-target Rust tests; Python pytest, Ruff formatting/lint and Pyright.

@beinan
beinan merged commit 31025ea into lance-format:main Oct 19, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant