Skip to content

feat(storage): Support GCS Get Service Account#5738

Closed
ccbcqbz wants to merge 9 commits into
googleapis:mainfrom
ccbcqbz:branch-get-service-account
Closed

feat(storage): Support GCS Get Service Account#5738
ccbcqbz wants to merge 9 commits into
googleapis:mainfrom
ccbcqbz:branch-get-service-account

Conversation

@ccbcqbz
Copy link
Copy Markdown

@ccbcqbz ccbcqbz commented May 26, 2026

feat(storage): Support GCS Get Service Account REST endpoint

Closes a GCS feature gap by implementing the control plane feature. This retrieves the email address of the Google Cloud Storage service agent for a given project, creating it under the hood if it does not yet exist.

Main changes:

  1. Data Modeling: Created JSON deserializer in using for robust unknown fields preservation (forward compatibility).
  2. Request Builder: Added a public fluent request builder and runner in utilizing standard GAX execution pipelines.
  3. Client Integration: Wired the API cleanly into the mockable trait (), handwritten REST client (), and (plain and traced paths).
  4. Unit & E2E Tests: Added mock HTTP REST server unit tests, deserialization unit tests, and wired up a live end-to-end integration test in hitting the live GCS control plane.

@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label May 26, 2026
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements the get_service_account API for the GCS client, allowing users to fetch the service account email address for a given project. The review feedback highlights three main issues: a compilation error when passing a LazyLock<String> to HeaderValue::from_static, a potential double-prefix bug in tracing if the projects/ prefix is not stripped early, and a lack of percent-encoding for project IDs containing domain names.

Comment thread src/storage/src/storage/get_service_account.rs
Comment thread src/storage/src/storage/get_service_account.rs
Comment thread src/storage/src/storage/get_service_account.rs
@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Codecov Report

❌ Patch coverage is 77.10280% with 49 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.80%. Comparing base (2f2c1e7) to head (cfe6e6d).

Files with missing lines Patch % Lines
src/storage/src/storage/get_service_account.rs 83.33% 25 Missing ⚠️
src/storage/src/storage/transport.rs 54.05% 17 Missing ⚠️
src/storage/src/storage/stub.rs 0.00% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5738      +/-   ##
==========================================
- Coverage   97.88%   97.80%   -0.08%     
==========================================
  Files         226      227       +1     
  Lines       55797    56011     +214     
==========================================
+ Hits        54619    54784     +165     
- Misses       1178     1227      +49     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ccbcqbz ccbcqbz force-pushed the branch-get-service-account branch from f1af51a to 4965ea4 Compare May 26, 2026 01:19
@joshuatants
Copy link
Copy Markdown
Contributor

In Rust, it is possible to have impl blocks for a given struct split across different files, as long as all the files are in the same crate.

https://doc.rust-lang.org/reference/items/implementations.html#r-items.impl.inherent.coherence
https://doc.rust-lang.org/book/ch05-03-method-syntax.html#multiple-impl-blocks

Therefore, even if there is an impl block for StorageControl in generated/client.rs, you can still add hand-written code to StorageControl by starting another impl block, say in control/client.rs itself.

I would not want to put a control-plane function in the data-plane client, unless there is really no way to put it in StorageControl.

Can you try putting this on StorageControl instead?

@ccbcqbz
Copy link
Copy Markdown
Author

ccbcqbz commented May 26, 2026

starting another impl block, say in control/client.rs

Hi @joshuatants , thanks for the comments. You're right, we can add new methods to handwritten control/client.rs via a separate impl block, but the blocker issue is that we cannot add new fields (like a REST HTTP client) to the StorageControl struct without causing compile-time failures in the auto-generated control/generated/client.rs, due to the fact that all fields of a struct must be declared inside the original struct definition block itself in Rust.

For example, compilation will fail for generated client when adding a new field to the handwritten client:

error[E0063]: missing field `test_field` in initializer of `control::client::StorageControl`
   --> src/storage/src/control/generated/client.rs:530:12
    |
530 |         Ok(Self { storage, control })
    |            ^^^^ missing `test_field`

And the project service account retrieval is a GCS JSON/REST-only feature (GET /storage/v1/projects/{project}/serviceAccount). It is not supported by the GCS gRPC control plane services. This is why get_service_account must reside on Storage, which already contains a fully initialized, handwritten REST engine.

This also likely explains why other language SDKs have implemented this feature in the storage client, despite it functionally being more of a control plane operation.

@ccbcqbz ccbcqbz marked this pull request as ready for review May 26, 2026 04:25
@ccbcqbz ccbcqbz requested review from a team as code owners May 26, 2026 04:25
Copy link
Copy Markdown
Contributor

@coryan coryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it may be better to implement this function in the gRPC API and then let the generator do its job. @joshuatants this is your call.

Comment thread src/storage/src/storage.rs Outdated
pub(crate) mod checksum;
pub(crate) mod client;
pub(crate) mod common_options;
pub mod get_service_account;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes the module public, which you may not want to do.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I've changed it to internal.

Comment thread tests/storage/src/lib.rs
let project = project_id()?;

tracing::info!("testing get_service_account() for project: {project}");
let email = client.get_service_account(project).send().await?;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the other APIs consume projects/_/buckets/{bucket_id} or anywhere cache names that are fully qualified. That is intentional, to keep the generated StorageControl working without a hand-written layer. Consider changing this to projects/{project}.

Comment thread src/storage/src/storage/v1.rs Outdated
Comment on lines +367 to +368
#[serde(flatten)]
pub unknown_fields: std::collections::HashMap<String, serde_json::Value>,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this public?

Comment thread src/storage/src/storage/v1.rs Outdated
Comment on lines +362 to +365
#[serde(default, rename_all = "camelCase")]
pub(crate) struct ProjectServiceAccount {
pub kind: String,
#[serde(rename = "email_address")]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe

Suggested change
#[serde(default, rename_all = "camelCase")]
pub(crate) struct ProjectServiceAccount {
pub kind: String,
#[serde(rename = "email_address")]
pub(crate) struct ProjectServiceAccount {
pub kind: String,

No sense in specifying the field name format and then overriding the format for the one field name where it matters?

Comment thread src/storage/src/storage/stub.rs Outdated
/// Implements [crate::client::Storage::get_service_account].
fn get_service_account(
&self,
_project: String,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You want something like:

Suggested change
_project: String,
_request: GetServiceAccountRequest,

Because the thing may receive a request in the future.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread src/storage/src/storage/client.rs Outdated
Comment on lines +228 to +231
/// # Design Note (Architectural Exception):
/// Logically, this administrative operation belongs to the control plane (`StorageControl`).
/// However, since the GCS gRPC control plane does not support project service account retrieval,
/// we place it here in the handwritten REST client to avoid altering machine-generated code.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not belong in the public documentation...

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

Comment thread src/storage/src/storage/client.rs Outdated
Comment on lines +233 to +234
/// # Parameters
/// * `project` - the Project ID or project number.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// # Parameters
/// * `project` - the Project ID or project number.
/// # Parameters
/// * `project` - the project name, either `projects/{projectId}` or `projects/{projectNumber}`.
/// See [The project resource] for more information about projects and their identifiers.
///
/// [The project resource]: https://docs.cloud.google.com/resource-manager/docs/cloud-platform-resource-hierarchy#projects

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@joshuatants
Copy link
Copy Markdown
Contributor

i think it may be better to implement this function in the gRPC API and then let the generator do its job. @joshuatants this is your call.

Thanks @coryan . Chatted with the rest of the SDK team. We agreed that we'll skip this API for now and try to get it implemented in the gRPC API.

Appreciate the work @ccbcqbz !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants