Skip to content

CLDSRV-909: Reject CopyObject when source exceeds 5 GiB#6177

Open
tcarmet wants to merge 3 commits into
development/9.4from
bugfix/CLDSRV-909-enforce-copyobject-source-size-limit
Open

CLDSRV-909: Reject CopyObject when source exceeds 5 GiB#6177
tcarmet wants to merge 3 commits into
development/9.4from
bugfix/CLDSRV-909-enforce-copyobject-source-size-limit

Conversation

@tcarmet
Copy link
Copy Markdown
Contributor

@tcarmet tcarmet commented May 28, 2026

Align cloudserver's CopyObject with the AWS S3 contract by rejecting requests whose source object exceeds 5 GiB. AWS S3 returns HTTP 400 InvalidRequest ("The specified copy source is larger than the maximum allowable size for a copy source: 5368709120") and expects clients to use the multipart copy flow instead; cloudserver currently accepts these requests, leaving a gap with the published contract.

Targeting development/9.4 because changing a previously-accepted response is a behavior break for clients that relied on cloudserver's permissive behavior, happy to retarget development/9.3 if we prefer wider rollout.

Last commit contains a prettier fmt command and is kept separate from the actual code change for review. We should not focus on it as most of it is existing change.

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented May 28, 2026

Hello tcarmet,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented May 28, 2026

Incorrect fix version

The Fix Version/s in issue CLDSRV-909 contains:

  • None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

  • 9.4.0

Please check the Fix Version/s of CLDSRV-909, or the target
branch of this pull request.

@claude
Copy link
Copy Markdown

claude Bot commented May 28, 2026

LGTM

Review by Claude Code

@codecov
Copy link
Copy Markdown

codecov Bot commented May 28, 2026

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
9097 1 9096 0
View the full list of 1 ❄️ flaky test(s)
should create a bunch of objects and their versions::put and head object with versioning With v4 signature on versioning suspended then enabled bucket w/ null version should create a bunch of objects and their versions

Flake rate in main: 13.81% (Passed 668 times, Failed 107 times)

Stack Traces | 600s run time
Timeout of 600000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (.../test/versioning/objectHead.js)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@tcarmet tcarmet force-pushed the bugfix/CLDSRV-909-enforce-copyobject-source-size-limit branch from 0094687 to be2daf0 Compare May 28, 2026 00:53
@claude
Copy link
Copy Markdown

claude Bot commented May 28, 2026

LGTM

Review by Claude Code

Comment thread lib/api/objectCopy.js Outdated
Comment thread tests/unit/api/objectCopy.js
@claude
Copy link
Copy Markdown

claude Bot commented Jun 1, 2026

  • config.bypassMaxPutObjectSize leaks if the bypass test fails — reset it in the after block (tests/unit/api/objectCopy.js:660)

Review by Claude Code

Comment thread tests/unit/api/objectCopy.js Outdated
@claude
Copy link
Copy Markdown

claude Bot commented Jun 1, 2026

  • config.bypassMaxPutObjectSize cleanup in the bypass test is inside the callback — if the assertion throws, the flag leaks. Move the reset into the after() block.
    • See inline comment on tests/unit/api/objectCopy.js:790

Review by Claude Code

Comment thread lib/api/objectCopy.js Fixed
Comment thread lib/api/objectCopy.js Fixed
Comment thread lib/api/objectCopy.js Fixed
Comment thread lib/api/objectCopy.js Fixed
Comment thread lib/api/objectCopy.js Fixed
Comment thread lib/api/objectCopy.js Fixed
Comment thread lib/api/objectCopy.js Fixed
@tcarmet tcarmet force-pushed the bugfix/CLDSRV-909-enforce-copyobject-source-size-limit branch from 35eb632 to 213648e Compare June 1, 2026 22:45
Comment thread tests/unit/api/objectCopy.js
@claude
Copy link
Copy Markdown

claude Bot commented Jun 1, 2026

  • config.bypassMaxPutObjectSize is not restored in the after() block of the 'objectCopy source size limit' test suite — if the test callback doesn't run (timeout, sync throw), the flag leaks to subsequent suites.
    - Add config.bypassMaxPutObjectSize = false; to the after() block.

    Review by Claude Code

@tcarmet tcarmet force-pushed the bugfix/CLDSRV-909-enforce-copyobject-source-size-limit branch from 213648e to f34ae66 Compare June 1, 2026 22:56
@claude
Copy link
Copy Markdown

claude Bot commented Jun 1, 2026

LGTM

Review by Claude Code

@tcarmet tcarmet marked this pull request as ready for review June 4, 2026 21:43
@tcarmet tcarmet requested a review from a team June 4, 2026 21:43
Comment thread lib/api/objectCopy.js Outdated
'The specified copy source is larger than the maximum ' +
`allowable size for a copy source: ${constants.maximumAllowedUploadSize}`);
// eslint-disable-next-line no-param-reassign
request.sourceServerAccessLog && (request.sourceServerAccessLog.error = err);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a strange construct, I would rather do a plain if statement

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is, but what I'm doing here is following the same pattern found across the file/project. I could go for a if and leave a bit of inconsistency or change it as whole. What I went with now is just consistency.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, generally speaking I tend to prefer having an inconsistent but incrementally correct style than a consistently subpar style, but that's my take, do as you think is best!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with that, I think I needed an extra opinion to make a call so thanks, will have a look into it.

@tcarmet tcarmet force-pushed the bugfix/CLDSRV-909-enforce-copyobject-source-size-limit branch from f34ae66 to 636984c Compare June 5, 2026 19:35
@claude
Copy link
Copy Markdown

claude Bot commented Jun 5, 2026

LGTM

Review by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants