Skip to content

feat(files): presigned part uploads to S3 object storage - #66176

Open
karlitschek wants to merge 1 commit into
masterfrom
feat/s3-presigned-part-uploads
Open

karlitschek wants to merge 1 commit into
masterfrom
feat/s3-presigned-part-uploads

Conversation

@karlitschek

@karlitschek karlitschek commented Nov 3, 2026 •

Copy link
Copy Markdown
Member
  • Resolves: #

Summary

Large imports and migrations into S3 primary storage currently push every byte through PHP. This lets a server-side caller have the content uploaded straight to the bucket instead, without the storage credentials ever leaving the server.

  • OCP\Files\ObjectStore\IObjectStorePresignedUpload (implemented by the S3 object store): start a multipart upload, hand out one presigned request per part, each bound to the part's SHA-256, then complete or abort it.
  • OCP\Files\ObjectStore\IObjectStoreStorage: exposes the object store and the object id behind a file, through storage wrappers.
  • Setups that would need a client-side secret (SSE-C) or can't sign checksums (V2 signatures) report that presigned uploads aren't available, so callers fall back to the normal path.

Tests: unit tests, plus a real round trip against S3 (MinIO) in S3Test.

This draft description was written with AI assistance (Claude Code) at the author's request; the author reviews it before the PR leaves draft.

TODO

  • Author's description and sign-off

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

This change was written with Claude Code (claude-opus-5-5) and is to be reviewed by the author before it leaves draft; every commit carries an Assisted-by trailer. Commits still need the author's sign-off (DCO).

🤖 Generated with Claude Code

@karlitschek
karlitschek force-pushed the feat/s3-presigned-part-uploads branch from 133832f to 54b26f9 Compare November 3, 2026 12:03
@karlitschek
karlitschek marked this pull request as ready for review November 3, 2026 12:05
@karlitschek
karlitschek requested review from a team as code owners November 3, 2026 12:05
@karlitschek
karlitschek requested review from GhassenKefi, come-nc, icewind1991, salmart-dev and sorbaugh and removed request for a team November 3, 2026 12:05
@karlitschek
karlitschek force-pushed the feat/s3-presigned-part-uploads branch 3 times, most recently from 64ad364 to 5aeb60d Compare November 3, 2026 19:45
A client (for example an importer) can now upload a file's content
straight into the S3 bucket behind Nextcloud's object storage, keeping
PHP out of the data path: the server starts a multipart upload, hands out
one presigned request per part, and lists and completes the upload as
before. Every part is signed with its SHA-256, so S3 refuses bytes that
don't match it, and the bucket's credentials never leave the server.
Configurations that would need a secret on the client (SSE-C keys) or
can't sign a checksum (V2 signatures) say so.

IObjectStoreStorage names the object store and the object of a file for
callers outside the object store code, through storage wrappers.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Frank Karlitschek <karlitschek@users.noreply.github.com>
@karlitschek
karlitschek force-pushed the feat/s3-presigned-part-uploads branch from 5aeb60d to cc4eaf5 Compare November 3, 2026 22:20

@come-nc come-nc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not know enough of s3 to review this.
My only input is that compatibility with encryption app should be double-checked.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants