Backport/tentacle 20.2.4 cobaltcore storage/keystone project reader

PendingReleaseNotes

@@ -1,3 +1,109 @@
+>=22.0.0
+
+* RBD: The Python ``Image.list_lockers()`` method now returns a dictionary
+  containing ``tag``, ``exclusive``, and ``lockers`` fields even when no clients
+  hold the lock, instead of an empty list. Callers should check the
+  ``lockers`` field rather than the truth value of the returned object
+  to determine whether any clients hold the lock.
+
+* RGW: The default value of ``rgw_rest_conn_connect_to_resolved_ips`` changed
+  from ``false`` to ``true``.
+* RGW: The default value of ``rgw_rest_conn_ip_fail_timeout_secs`` changed
+  from ``2`` to ``30`` seconds.
+
+* RGW: Fixed a bug in same-account evaluation of bucket policy statements that
+  allow the account principal. Users and roles in that account now require a
+  corresponding allow from identity policy to match AWS behavior. The evaluation
+  of bucket policy against user or role principals is unchanged and still does
+  not require identity policy.
+* RGW: The STS AssumeRole and AssumeRoleWithWebIdentity actions now consult
+  identity policy according to the AWS same-account and cross-account policy
+  evaluation rules. They had previously only consulted the role's trust policy.
+  This means that account users will now require an Allow from identity policy
+  to assume roles in other accounts.
+
+* ``src/script/ceph-backport.sh`` now runs on macOS in addition to Linux: it
+  re-execs under GNU bash >= 4 when the system bash is too old, and honors a
+  ``$GETOPT`` environment override so GNU getopt can be selected
+  (``brew install bash gnu-getopt``).
+* CephFS: Added a multi-tiered scoring matrix to the MDS Monitor to dynamically
+  prioritize standby promotion during failover events. The `mds_join_fs` configuration
+  variable is now checked in combination with infrastructure topology rules to enforce
+  node anti-affinity constraints. An isolated, generic standby daemon (Score 5) will
+  now be chosen over a preferred standby daemon targeting that file system if the
+  preferred standby resides on the same physical host as the failed active rank (Score 3).
+  This safely mitigates localized hardware failure boundaries during rapid automatic
+  recovery cycles. This behavior can be disabled per file system using the command
+  `ceph fs set <fs_name> standby_enable_host_anti_affinity false` (default: true)
+  to restore legacy strict 'mds_join_fs' prioritization.
+
+* CephFS: New MDS configuration options for non-blocking dirfrag prefetch have
+  been added. ``mds_dir_prefetch_backend``, when enabled with
+  ``mds_dir_prefetch=false``, serves dentry requests via a fast targeted keyed
+  OSD fetch while lazily launching a full dirfrag prefetch in the background.
+  The background prefetch is triggered only after
+  ``mds_dir_prefetch_backend_hit_threshold`` (default 16) accesses to the same
+  dirfrag, avoiding wasted full-omap reads for directories that are only hit
+  a few times in random workloads.  ``mds_dir_prefetch_backend_max`` (default 1)
+  limits the number of concurrent background fetches.  This reduces latency in
+  large directories while retaining cache warmth for hot directories.
+
+* CephFS: The ``client_force_lazyio`` configuration option is now correctly marked
+  as not supporting runtime updates. Previously, the configuration schema indicated
+  this option could be changed at runtime, but changes had no effect on opened file
+  handlers in ceph-fuse or libcephfs clients because there is no logic to propagate
+  changes to open file handles.
+
+* RADOS: PG autoscaler allows overlapping roots. Each root receives a PG target based
+  on its OSDs, with OSDs shared across multiple roots contributing proportionally
+  less to each root's allocation.
+
+* CephFS: CephFS snapshots metadata is now mutable. It is now possible to add,
+  update and remove existing key-value pairs that are part of a snapshot
+  metadata via libcephfs API ceph_do_snap_md_op().
+
+* CephFS: MDS dmclock support for subvolume QoS. With this feature, MDS assigns QoS
+  to throttle client metadata requests (e.g., create, mkdir, lookup, and so on) to
+  subvolumes, where each subvolume QoS is shared among multiple client sessions at
+  that time.
+* MON: Each monitor now reports four new perf counters covering its data
+  directory: ``data_disk_total_bytes``, ``data_disk_avail_bytes``,
+  ``data_disk_avail_percent``, and ``db_total_bytes`` (estimated on-disk size
+  of the key/value store). They surface automatically as ``ceph_mon_*``
+  Prometheus metrics via the ``prometheus`` mgr module and ``ceph-exporter``,
+  enabling trending of mon DB growth and filesystem free space.
+
+* librados/neorados: The C++ APIs for executing Ceph Class (CLS) methods
+  have undergone a breaking change to enforce compile-time type safety, replacing
+  legacy string-based parameters with strongly-typed ClsMethod structs. C++
+  developers must update their method calls to use these new structs, which ensure
+  read/write semantics are correctly applied.
+
+* RGW: S3 ListObjects and ListObjectVersions now support the
+  ``x-amz-optional-object-attributes: RestoreStatus`` request header to include
+  restore status in listing responses. Restore status is stored in the bucket
+  index, so only objects written or restored after this upgrade will populate
+  the field. Existing objects are unaffected.
+* RGW: New S3 Control APIs to apply PublicAccessBlock configuration to User Accounts.
+* RGW: PutBucketEncryption and GetBucketEncryption now support the
+  ``BlockedEncryptionTypes`` element. Blocking ``SSE-C`` on a bucket rejects new
+  SSE-C uploads with ``403 AccessDenied`` while leaving existing SSE-C objects
+  readable. Newly created buckets are unchanged by default. Enable the new
+  ``rgw_s3_block_sse_c_by_default`` option to block SSE-C on them instead,
+  matching the AWS behavior for new buckets.
+* RGW: New config option ``rgw_keystone_accepted_project_reader_roles``.
+  A Keystone user whose only access-granting role is one of the configured project-reader
+  roles is capped to read-only access within its project, mirroring the
+  ``project_reader_roles`` behavior of Swift's keystoneauth middleware. A bucket
+  policy naming the user can still grant additional access. The reader role
+  admits the user on its own; it need not also be in
+  ``rgw_keystone_accepted_roles``.
+* RGW: New config option ``rgw_keystone_implicit_deny_roles``.
+  A Keystone user whose only access-granting role is one of the configured
+  implicit-deny roles is authenticated but granted no implicit permissions.
+  Access must be granted explicitly through a bucket policy or IAM policy, and
+  applies only to the buckets and objects the policy names. The role admits the
+  user on its own.
 * ceph-volume: Raw BlueStore OSD preparation now pre-formats NVMe devices and
   skips the slower BlueStore discard phase,reducing mkfs time on
   very large namespaces.

doc/radosgw/bucketpolicy.rst

@@ -122,6 +122,11 @@ For all requests, condition keys we support are:
 - aws:UserAgent
 - aws:username
 
+Request that authenticate with Keystone also include:
+
+- keystone:role
+- keystone:userid
+
 We support certain s3 condition keys for bucket and object requests.
 
 *Support for the following bucket-related operations was added in the Mimic

doc/radosgw/keystone.rst

@@ -54,6 +54,88 @@ only use implicit tenants, and the other protocol will
 never use implicit tenants.  Some older versions of ceph
 only supported implicit tenants with swift.
 
+Project-scoped reader role
+--------------------------
+
+By default any Keystone user accepted by the gateway is granted full control
+over its project's object storage. ``rgw keystone accepted project reader
+roles`` designates one or more Keystone roles as *project-scoped readers*
+(the Swift ``project_reader_roles`` equivalent)::
+
+   rgw keystone accepted roles = member
+   rgw keystone accepted project reader roles = objectstore_viewer
+
+When the reader role is the only access-granting role a user holds, the user
+is capped to read-only within its Keystone project: it may read objects and
+inspect bucket configuration (ACLs, policy, versioning, location, ...), but
+not write. Every user of a project maps to the same RGW account (the project
+itself), so a reader lists and reads every bucket the project owns through the
+normal owner-scoped paths, and cannot reach another project (a different
+owner). This scoping is by ownership and holds regardless of ``rgw keystone
+implicit tenants``, which only controls the bucket-name namespace: with it
+off, a reader reaching another project's bucket gets ``403`` (the bucket is
+in the shared namespace, visible but not owned); with it on, it gets ``404``
+(the bucket is in a separate namespace and not visible). Either way the
+reader stays confined to its own project.
+
+Note the following:
+
+- **A project reader role admits the user on its own.** It need not also
+  appear in ``rgw keystone accepted roles`` (mirroring Swift's
+  ``project_reader_roles``): a user whose only granting role is a project
+  reader role is admitted and capped to read-only.
+
+- **The cap is a floor, not a hard ceiling.** A bucket policy that names the
+  user (via a ``keystone:userid`` / ``keystone:role`` condition or a principal
+  ARN) can still grant specific actions such as writes on specific buckets;
+  an explicit policy ``Deny`` can likewise remove access. Only bucket creation
+  and Swift account-metadata writes are refused up front regardless of policy.
+
+- **A role that grants more than read overrides the cap.** If the same user
+  also holds ``admin`` or any other role in ``rgw keystone accepted roles``
+  (for example ``member``), that role wins and the user receives full control.
+  To keep a user read-only, assign it *only* the reader role. Roles that are
+  in no accepted list at all are ignored and do not affect the cap.
+
+Implicit-deny roles
+-------------------
+
+``rgw keystone implicit deny roles`` designates Keystone roles
+that authenticate a user without granting any implicit permissions::
+
+   rgw keystone accepted roles = member
+   rgw keystone implicit deny roles = objectstore_authenticated
+
+A user whose only access-granting role is such a role is denied every
+operation by default. Access must be granted explicitly through a bucket
+policy or IAM policy (for example with a ``keystone:userid`` or
+``keystone:role`` condition), and an explicit ``Allow`` then applies only to
+the buckets and objects the policy names. Unlike a project reader, a user
+with only an implicit-deny role cannot list the buckets of its project;
+bucket creation and Swift account-metadata writes are always refused.
+
+The project-reader notes above apply here as well: the role admits the user
+on its own (no co-listing in ``rgw keystone accepted roles`` needed), an
+explicit policy ``Deny`` always wins, and any role granting more (``admin``,
+a project reader role, or another accepted role such as ``member``) takes
+precedence over the implicit-deny cap.
+
+.. note:: These tiers cap what a request may do at authorization time. They
+   apply to the default Keystone mapping, where each project maps to a plain
+   tenant-scoped gateway user reaching the gateway over S3 or Swift. They do
+   **not** apply in the following cases:
+
+   - **The librgw frontend (NFS-Ganesha).** It authorizes every request with
+     full control, so a gateway reached over that path is not capped. Assign
+     these roles only where clients use S3 or Swift.
+   - **Keystone users associated with an** :doc:`RGW account <account>`.
+     Access for an account is governed entirely by IAM policy (evaluated
+     before the permission mask), so the tiers place no cap there. Use them
+     with the default (non-account) mapping.
+   - **The** ``radosgw-admin`` **command and the admin REST API.** These are
+     backend administration paths that do not authenticate a Keystone user,
+     so they are not subject to these roles at all.
+
 Ocata (and later)
 -----------------
 
@@ -159,6 +241,19 @@ S3 API (with AWS-like access and secret keys), if the ``rgw s3 auth
 use keystone`` option is set. For details, see
 :doc:`s3/authentication`.
 
+Requests authenticated via Keystone (Swift tokens or S3 with Keystone-managed
+credentials) expose the Keystone idenitity in the IAM policy environment
+as the condition keys: ``keystone:role`` (role names) and
+``keystone:userid`` (user UUID). These conditions can be used in bucket
+policies and idenitity policies with ``StringEquals``, ``StringNotEquals`` etc.
+
+- **keystone:role** - Allow or deny by *role* (e.g only users with role
+  ``reader`` get read access). Use for RBAC.
+- **keystone:userid** - Restrict to specific *user*. Use when policy
+  depends on a specific user, not just their role.
+
+See :doc:`bucketpolicy` for list of supported condition keys.
+
 Service token support
 ---------------------
 

doc/radosgw/s3/authentication.rst

@@ -216,6 +216,26 @@ access the same resources when things like Swift user ACLs are in
 play. This is one of the many reasons that you should use S3 bucket
 policies rather than S3 ACLs when possible.
 
+The SNS topic actions are mapped the same way, so that the Keystone
+project-reader role (see :ref:`radosgw-keystone`) can inspect topics but
+not modify them:
+
++---------------------------------------+---------------+
+| Operation                             | Permission    |
++=======================================+===============+
+| ``sns:GetTopicAttributes``            | ``READ``      |
++---------------------------------------+---------------+
+| ``sns:ListTopics``                    | ``READ``      |
++---------------------------------------+---------------+
+| ``sns:CreateTopic``                   | ``WRITE``     |
++---------------------------------------+---------------+
+| ``sns:SetTopicAttributes``            | ``WRITE``     |
++---------------------------------------+---------------+
+| ``sns:DeleteTopic``                   | ``WRITE``     |
++---------------------------------------+---------------+
+| ``sns:Publish``                       | ``WRITE``     |
++---------------------------------------+---------------+
+
 
 .. _Authenticating Requests (AWS Signature Version 4): https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html
 .. _Authenticating requests (AWS signature version 2): https://docs.aws.amazon.com/AmazonS3/latest/userguide/auth-request-sig-v2.html

release-management/backports/01a16411-8df8-7e1f-baaf-8aa40a09ecdd.md

@@ -0,0 +1,80 @@
+---
+id: 01a16411-8df8-7e1f-baaf-8aa40a09ecdd
+title: 'IAM / Bucket Policy: Keystone role and user ID support'
+component: rgw
+category: feature
+provenance:
+  type: upstream_pr
+  upstream_prs:
+  - https://github.com/ceph/ceph/pull/67752
+  upstream_pr_state: merged-main
+  upstream_commits:
+  - b06b7a82ed1a5c57dce8b9ce2fa0931fe0f55ff7
+  previous_backport: 01a0cdb3-3f84-74ab-9403-c7638a039697
+justification:
+  type: customer
+  refs:
+  - cobaltcore-dev/cloud-storage#566
+  - cobaltcore-dev/cloud-storage#565
+  - https://github.com/cobaltcore-dev/cloud-storage/issues/380
+  - https://github.com/cobaltcore-dev/cloud-storage/issues/372
+risk:
+  blast: cosmetic
+  conflict: trivial
+  coverage: strong
+---
+
+## Justification (public)
+
+Allows Keystone role names and user IDs to be injected as `keystone:role` and
+`keystone:userid` condition keys in S3 bucket policies, enabling role-based and
+per-user access control via standard IAM policy conditions.
+
+## Justification (internal)
+
+Improves keystone authorization features.
+
+## Risk notes
+
+**blast: cosmetic** — purely additive. Two new condition keys are populated for
+Keystone-authenticated requests. No existing condition key, policy or auth path
+changes meaning, and non-Keystone requests are untouched.
+
+**conflict: trivial**
+
+**coverage: strong** — 
+47/47 unit tests pass in `unittest_rgw_iam_policy`.
+14/14 live checks pass against vstart RGW and a real Keystone, with two
+multi-role users in one project: `keystone:role` matching (single value,
+multi-value, shared role, absent role), case handling (`StringEquals` is
+case-sensitive, `StringEqualsIgnoreCase` is not), `keystone:userid` matching per
+user, `StringLike` wildcards, and three `StringNotEquals` cases that confirm the
+AND semantics with multiple roles.
+7/7 live grant checks pass with non-owner users from a second project: `Allow`
+on `keystone:role` or `keystone:userid` grants exactly the listed read, list,
+write and delete actions, users without the role or ID get nothing, and `Deny`
+overrides `Allow`.
+
+## Behavior change
+
+Requests authenticated via Keystone now expose two additional IAM policy
+condition keys: `keystone:role` (one entry per role, lowercased) and
+`keystone:userid` (the Keystone user UUID). Requests not authenticated via
+Keystone are unchanged, and no existing policy changes behaviour.
+
+### Two points for policy authors
+
+Role names in policies are case-sensitive. Roles are lowercased on the way into
+the policy environment, so how the role is spelled in Keystone does not matter,
+but the value typed in the policy is compared exactly. A policy containing
+`"TestRole"` never matches and gives no warning. Write role names in lower case,
+or use `StringEqualsIgnoreCase`.
+
+Rules that distinguish between users of the same project must be written as
+`Deny`. All users in an OpenStack project share a single RGW account, so each is
+an owner of buckets created there, and owners are not restricted by `Allow`
+conditions. `Deny` statements do apply to owners and discriminate correctly.
+
+## Upgrade notes
+
+None.

release-management/backports/01a18d83-9554-7c5f-8407-f6a30b7f3c9f.md

@@ -0,0 +1,53 @@
+---
+id: 01a18d83-9554-7c5f-8407-f6a30b7f3c9f
+title: Keystone Project RO/Viewer role + Implicit Deny
+component: rgw
+category: feature
+provenance:
+  type: upstream_pr
+  upstream_prs:
+  - https://github.com/ceph/ceph/pull/70372
+  upstream_pr_state: merged-main
+  upstream_commits:
+  - 87252a5deac6450dafc5d2e279aa0671509d1c1a
+  - ba0c6ade977334f9b5cd01d5dc56354c9ad43507
+  - dcc1de7d4d28bf825b1479cf48018c4ab847bf6a
+  previous_backport: 01a14417-d161-7221-87ad-c32f7e8170ec
+justification:
+  type: customer
+  refs:
+  - cobaltcore-dev/cloud-storage#566
+risk:
+  blast: availability
+  conflict: substantive
+  coverage: strong
+---
+
+## Justification (public)
+
+Adds two Keystone roles: a project reader (`rgw_keystone_accepted_project_reader_roles`), read-only by default, and an implicit-deny role (`rgw_keystone_implicit_deny_roles`), no access by default. Both can be granted more through bucket policy using `keystone:userid` or `keystone:role`.
+
+## Justification (internal)
+
+Customer roadmap feature, mirroring Swift's `project_reader_roles`.
+
+## Risk notes
+
+**blast: availability** — off by default, but it changes how permissions are computed for every Keystone user. A mistake could remove access; no data is at risk.
+
+**conflict: substantive** — tentacle lacks some newer upstream code, so STS, SNS topics and the new unit test needed small manual adaptations. Merged code matches upstream otherwise.
+
+**coverage: strong** — unit tests and live tests against RGW and Keystone pass. They cover read-only and no-access defaults, policy overrides by `keystone:userid` and `keystone:role`, `Deny` precedence and cross-project isolation.
+
+## Behavior change
+
+Two new options, empty by default:
+
+- `rgw keystone accepted project reader roles` — user is read-only within its project.
+- `rgw keystone implicit deny roles` — user has no access without an explicit policy grant.
+
+The most permissive accepted role wins (`member` or `admin` gives full control). A bucket policy naming the user or role can grant more, for that bucket only. Bucket creation and STS `AssumeRole` are always refused for these users.
+
+## Upgrade notes
+
+None. Both options default to empty.