Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/rust-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ jobs:
run: |
cargo clippy -p lance-context-wal --features raft-log --all-targets -- -D warnings
cargo test -p lance-context-wal --features raft-log --lib --test raft_log --test raft_log_cache_identity
- name: Client journal durability and process recovery
run: |
cargo clippy -p lance-context-wal --features client-journal --all-targets -- -D warnings
cargo test -p lance-context-wal --features client-journal --lib client_journal
- name: Replicated WAL quorum and process recovery
run: |
cargo clippy -p lance-context-wal --features replication --all-targets -- -D warnings
Expand Down
1 change: 1 addition & 0 deletions crates/lance-context-wal/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ publish = false
description = "Independent Rust WAL experiments owned by lance-context"

[features]
client-journal = ["dep:anyhow", "dep:raft-engine", "dep:bytes", "dep:serde", "dep:serde_json", "dep:tokio", "tokio/sync"]
replication = ["raft-log", "record", "dep:bincode", "dep:slog"]
raft-log = ["placement", "dep:raft", "dep:raft-engine", "dep:raft-proto", "dep:protobuf"]
vectored-io = ["dep:bytes", "dep:tokio"]
Expand Down
11 changes: 11 additions & 0 deletions crates/lance-context-wal/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,17 @@ Implementation and validation are tracked in
[#378](https://github.com/lance-format/lance-context/issues/378). Related PRs should
include `Refs #378`; individual steps should leave the tracking issue open.

The optional `client-journal` feature stores recoverable client state in one
local engine and shares durable barriers across writers. It returns latest
values for a fixed set of keys, with immutable scope/settings binding, bounded
admission, reclamation and a shared failure fence. Callers still own producer
ordering, exact retry identity and recovery validation; a local completion does
not establish a remote quorum ACK. See [the API and recovery contract](../../docs/design/client-journal.md).

```bash
cargo test -p lance-context-wal --features client-journal --lib client_journal
```

The independent `contract` feature defines ingestion routing keys, the schema
fingerprint, immutable-ID policy and record byte/row limits. Validation checks
the representation after decoding; callers must durably bind and compare the
Expand Down
Loading
Loading