Skip to content

feat(rollout): expose flush-on-add in SDKs - #346

Open
KaiqiJinWow wants to merge 2 commits into
lance-format:mainfrom
KaiqiJinWow:feat/rollout-client-flush
Open

KaiqiJinWow wants to merge 2 commits into
lance-format:mainfrom
KaiqiJinWow:feat/rollout-client-flush

Conversation

@KaiqiJinWow

Copy link
Copy Markdown
Contributor

Follow-up to #191.

Problem

The server already supports POST /rollouts/{name}/records?flush=true, but the Rust and Python clients cannot request it. The standalone remote flush() call is a no-op, so SDK callers that need read-your-write visibility must poll for the background sweeper.

await store.add(record)
rows = await store.list()  # the new row may not be visible yet

Change

  • Add backward-compatible Rust add_with_flush and add_rollouts_with_flush methods.
  • Add keyword-only flush=False to sync and async Python add and add_one.
  • For remote stores, send ?flush=true on the same append request.
  • For local stores, append and then seal the local MemWAL memtable.
  • Keep the existing default path unchanged.
  • Document the latency and partial-success semantics.
await store.add(record, flush=True)
rows = await store.list()  # the new row is readable

A flush can also publish earlier or concurrent pending appends. If the flush step fails, the append may already be durable, so retryable callers should supply and reuse stable record ids. Remote flush-on-add requires server v0.6.5 or newer.

Testing

  • cargo check --locked -p lance-context-client -p lance-context -p lance-context-python --all-features
  • cargo clippy --locked -p lance-context-client -p lance-context -p lance-context-python --all-features -- -D warnings
  • cargo test --locked -p lance-context --all-features
  • cargo test --locked -p lance-context-server flush_true_gives_read_your_write_and_default_does_not
  • Ruff format and lint
  • Local public-API verification with default and flushing appends
  • Real-server JSON and multipart verification with both background sweepers disabled

@KaiqiJinWow

Copy link
Copy Markdown
Contributor Author

Hi @beinan , could you take a look when you have a chance? CI is green. Thanks!

@KaiqiJinWow
KaiqiJinWow force-pushed the feat/rollout-client-flush branch from 9f32b9e to f4a9edc Compare October 22, 2026 23:22

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant