Skip to content

feat: support S3-backed context stores #14

Closed

Description

@beinan

Background

Context URIs are currently assumed to be local filesystem paths. Cloud-hosted agents need to persist shared context into remote object storage.

Proposal

  • Teach to detect URIs and initialize a Lance dataset backed by S3 via object store configuration.
  • Surface optional AWS credentials/configuration through environment variables and/or explicit parameters.
  • Update the Python path so S3 URIs are passed straight through and validated early.

Design Notes

  • Leverage Lance's existing integration (, ) rather than reimplementing S3 clients.
  • Allow per-run temporary credentials by honoring standard AWS env vars and the Lance object store configuration file.
  • Keep the URI format aligned with Lance's convention () so downstream tooling interoperates cleanly.

Acceptance Criteria

  • A context created with an S3 URI can append, checkout, and search records without regressions.
  • Python integration tests cover smoke usage against moto or a stubbed S3 endpoint.
  • Documentation explains required AWS permissions and credential discovery order.

Activity

  1. self-assigned this
    on Jan 19, 2026
  2. beinan commented on Jan 20, 2026

    @beinan
    CollaboratorAuthor

    implemented in #20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions