You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Delete dataset URIs through object storage and propagate deletion failures #250
The rollout delete handler removes the registry entry and calls tokio::fs::remove_dir_all with the dataset URI. For a file://, s3://, or gs:// URI this is not a valid local filesystem deletion. Errors are only logged and the endpoint still returns HTTP 204.
Local reproduction using file:// (no cloud resources involved):
Create a rollout store, write a row, and merge it.
Delete it: the route returns 204.
Create another store with the same name.
The supposedly deleted row reappears.
Expected: successful deletion removes the dataset; failed deletion is reported and remains retryable.
Actual: the registry hides a dataset whose data still exists, and same-name recreation reopens it.
Relevant code: crates/lance-context-server/src/routes/rollouts.rs::delete_rollout_store. Other store-kind delete handlers use the same local-filesystem pattern and should be checked.
Please use the configured object-store backend, close resident writers before removal, preserve a retry path on failure, and add URI-backed deletion/recreation tests. Cloud behavior follows from the code path but was not tested against production storage.
Found while reviewing main at e292518; fixes will be based on refreshed main (currently 8fe27b2).
Review of PR #251 at 948b970 reproduced a remaining lifecycle regression: retain a rollout store Arc, evict its LRU entry, DELETE the store, then append through the retained handle. The append incorrectly succeeds (Ok(2)). Datagen and generic stores use the same deletion path.
The follow-up tracks live handles with weak references independently of LRU residency. A per-name lock spans create/open/delete so reopening an evicted live store reuses its handle and a slow open cannot cross a local delete/recreate boundary. DELETE closes and fences the surviving handle before removing data. Tests cover all three store kinds, recreation, and reclamation of unused tracking entries.
Activity
beinan commentedon Sep 10, 2026
Review of PR #251 at
948b970reproduced a remaining lifecycle regression: retain a rollout store Arc, evict its LRU entry, DELETE the store, then append through the retained handle. The append incorrectly succeeds (Ok(2)). Datagen and generic stores use the same deletion path.The follow-up tracks live handles with weak references independently of LRU residency. A per-name lock spans create/open/delete so reopening an evicted live store reuses its handle and a slow open cannot cross a local delete/recreate boundary. DELETE closes and fences the surviving handle before removing data. Tests cover all three store kinds, recreation, and reclamation of unused tracking entries.