Skip to content

Align cluster commands with nested S3 storage access - #59

Merged
rmorehig merged 9 commits into
mainfrom
rafa/cluster-s3-storage-api
Oct 19, 2026
Merged

rmorehig merged 9 commits into
mainfrom
rafa/cluster-s3-storage-api

Conversation

@rmorehig

@rmorehig rmorehig commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Cluster and database commands now use the cluster-scoped s3_storage_access and nested s3_storage contract from Platform #1482.

cluster create accepts optional --s3-storage-access-external-id and --s3-storage-access-resource-tag. The API generates omitted identity values; the CLI forwards only supplied overrides. Custom cluster storage requires an explicit External ID matching the role trust policy. List/status JSON reads the nested access object, including null for legacy clusters. Database commands reuse the saved identity and accept no override. Managed cluster storage and inherited database storage retain distinct labels.

Flags now mirror API field paths on both cluster and database creation:

Flag API field
--s3-storage-data-bucket s3_storage.data.bucket
--s3-storage-data-path s3_storage.data.path
--s3-storage-backups-bucket s3_storage.backups.bucket
--s3-storage-backups-path s3_storage.backups.path
--s3-storage-role-arn s3_storage.role_arn
--s3-storage-access-external-id (cluster only) s3_storage_access.external_id
--s3-storage-access-resource-tag (cluster only) s3_storage_access.resource_tag

Examples

Managed storage, with API-generated identity:

rtree --org acme cluster create --name production --replicas 2 --min-size 2:8

Custom cluster storage:

rtree --org acme cluster create \
  --name production --replicas 2 --min-size 2:8 \
  --s3-storage-access-external-id rawtree-5b71c457-7d83-474b-9db9-6dc48a1ec042 \
  --s3-storage-access-resource-tag rawtree-dc1fb78e-341d-48f8-a183-9ecea12eb41d \
  --s3-storage-data-bucket acme-data --s3-storage-backups-bucket acme-backups \
  --s3-storage-role-arn arn:aws:iam::123456789012:role/RawTreeS3

The creation body contains these storage fields alongside name, replicas and sizing:

{
  "s3_storage_access": {
    "external_id": "rawtree-5b71c457-7d83-474b-9db9-6dc48a1ec042",
    "resource_tag": "rawtree-dc1fb78e-341d-48f8-a183-9ecea12eb41d"
  },
  "s3_storage": {
    "data": { "bucket": "acme-data", "path": "" },
    "backups": { "bucket": "acme-backups", "path": "" },
    "role_arn": "arn:aws:iam::123456789012:role/RawTreeS3"
  }
}

Database creation sends only s3_storage:

rtree --org acme --cluster production database create analytics \
  --s3-storage-data-bucket analytics-data --s3-storage-backups-bucket analytics-backups \
  --s3-storage-role-arn arn:aws:iam::123456789012:role/RawTreeS3

Compatibility and validation

The prior unscoped --external-id/--resource-tag, legacy database identity flags, --s3-external-id, and former --s3-data-*, --s3-backup-*, --s3-backups-*, --s3-role-arn, --s3-storage-external-id, and --s3-storage-resource-tag flags are rejected. Region discovery/routing from current main is retained. This is coordinated with Platform #1482 and MCP #33; no release/version bump is included.

  • cargo test --locked: 230 tests passed; the live Platform suite is normally ignored.
  • The live cluster-identity test passed separately against the rebuilt candidate Platform API: create without identity flags, read back nested identity, validate UUIDs/stability, delete temporary cluster.
  • Formatting and diff checks passed.
  • Earlier full-branch autoreview through P3 with thermo-nuclear criteria passed: autoreview --mode commit --commit <snapshot-parented-to-main> --max-priority P3.
  • Destination-shape update: all 230 CLI tests and formatting passed. autoreview --mode local --max-priority P3 raised one finding about empty paths; rejected because the backend defaults omitted paths to "" and accepts explicit "" as the bucket root. No actionable findings remain.
  • Flag-path update: all 230 tests, both command help outputs, formatting and diff checks passed. autoreview --mode local --max-priority P3 with thermo-nuclear criteria completed clean with no findings.
  • Fresh Devin review requested for the flag update. CI is pending. The live CI job targets Platform main, so the new contract depends on coordinated rollout.

Note

Medium Risk
Breaking CLI flag changes and cluster/database provisioning payloads require coordinated Platform #1482 rollout; misconfigured customer S3 or IAM identity can block creates until the API rejects them.

Overview
This PR updates cluster and database creation to match Platform #1482’s nested s3_storage and cluster-scoped s3_storage_access API. New shared flags (--s3-storage-data-bucket, paths, backups bucket, --s3-storage-role-arn) build the create payload when any S3 option is set; cluster create also accepts optional --s3-storage-access-external-id and --s3-storage-access-resource-tag, forwarding only supplied values while the API generates missing identity fields.

List/status/create output now surfaces storage mode (RawTree-managed vs customer-owned S3), resource tags, and external IDs; database list adds a storage column. Legacy --s3-* and old identity flags are rejected; database commands cannot override cluster identity. Implementation is split into s3_storage and cluster_identity modules, CLI tests move to src/cli/tests.rs, and new mock/live contract tests cover payloads and validation.

Reviewed by Cursor Bugbot for commit 29099a2. Bugbot is set up for automated code reviews on this repo. Configure here.

@rmorehig
rmorehig marked this pull request as ready for review September 7, 2026 18:38
@rmorehig rmorehig changed the title Add S3 storage options to cluster and database CLI Add cluster and database S3 storage options; bump CLI to 0.6.12 Sep 24, 2026
@rmorehig rmorehig changed the title Add cluster and database S3 storage options; bump CLI to 0.6.12 Align cluster identity and S3 commands with the new Platform API Oct 17, 2026

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@rmorehig rmorehig changed the title Align cluster identity and S3 commands with the new Platform API Align cluster commands with nested S3 storage access Oct 19, 2026
devin-ai-integration[bot]

This comment was marked as resolved.

@rmorehig
rmorehig merged commit 0fe64ec into main Oct 19, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant