Repository navigation
Expose per-record delete/forget, external-id lookup, and relationships via the REST server (API parity) #75
Description
Motivation
The REST server added in #66 exposes only a subset of what the core library and
the Python Context API already support. Applications that adopt the server as
their integration surface cannot perform several first-class record operations
and must fall back to the in-process API or open the Lance dataset directly.
Current server routes
GET /api/v1/health
POST /api/v1/contexts # create
GET /api/v1/contexts # list
GET /api/v1/contexts/{name}
DELETE /api/v1/contexts/{name} # delete whole context
POST /api/v1/contexts/{name}/records # add batch
GET /api/v1/contexts/{name}/records # list
GET /api/v1/contexts/{name}/records/{id} # get by internal id
POST /api/v1/contexts/{name}/search
POST /api/v1/contexts/{name}/retrieve
GET /api/v1/contexts/{name}/version
POST /api/v1/contexts/{name}/checkout
POST /api/v1/contexts/{name}/compact
GET /api/v1/contexts/{name}/compact/stats
Capabilities present in the library but missing from the server
The core ContextStore / Python Context provide these, with no equivalent
server route:
- Per-record delete / forget:
delete_by_id,delete_by_external_id, and
Pythonforget(...)logically delete a record by writing a tombstone. The
server can delete an entire context, but not a single record. - Get by external id:
get_by_external_id(...)resolves the caller-supplied
stable identifier documented in the Python API and README. The server's
records/{id}route resolves internal ids only. - Relationship traversal:
list_related(...)/ Pythonrelated(...)from
the relationships feature is not exposed through REST.
The same gap exists at the lance-context-api trait level, which currently
declares only add / get / list / search / retrieve / version / checkout / compact / compaction_stats.
Proposal
Bring the server, remote client, and API trait to parity with the core library
for these existing record-level operations:
DELETE /api/v1/contexts/{name}/records/{id}: logically delete a record by
internal id.DELETE /api/v1/contexts/{name}/records?external_id=...: logically delete a
record by caller-supplied external id.GET /api/v1/contexts/{name}/records/by-external-id?external_id=...: fetch a
record by external id.GET /api/v1/contexts/{name}/records/related?target_id=...&relation=...&limit=...:
list records with relationships pointing at a target, honoring lifecycle
visibility options.
Query parameters are preferable for external_id and relationship targets
because these caller-supplied identifiers often look like source URIs or paths
and may contain /, #, or ?.
Scope for a first PR
- Add the routes above with request/response DTOs consistent with existing
records/search handlers. - Extend the
lance-context-apitrait and remote client so callers can use the
same operations through either local or remote stores. - Add focused tests for delete by id, delete by external id, external-id lookup,
and related-record traversal.
Non-goals
- No new storage capabilities; this only exposes behavior that already exists in
the library. - No hard-delete or physical-erasure semantics.
delete/forgetremain
logical tombstone operations. - No auth, multi-tenancy, schema registry, or database-style information schema
changes. - Hybrid retrieval (Design hybrid retrieval API for text, vector, filters, and fusion #72) and broader architecture (Change the management of data tables and add some interface exposure for agent #37) are out of scope.
Backward compatibility
Purely additive; existing routes and payloads are unchanged.
Activity
feat: expose record parity APIs over REST (lance-format#79)