diff --git a/.github/workflows/test-cli.yml b/.github/workflows/test-cli.yml index e60e0d2c..00755f45 100644 --- a/.github/workflows/test-cli.yml +++ b/.github/workflows/test-cli.yml @@ -42,8 +42,16 @@ jobs: - name: Clippy without default features run: cargo clippy -p clickhousectl --all-targets --no-default-features -- -D warnings + # Clippy only compiles all features; these tests only run with the + # experimental access-negotiation feature enabled. + - name: Clippy with experimental access negotiation + run: cargo clippy -p clickhousectl --all-targets --features experimental-access-negotiation -- -D warnings + - name: Test run: cargo test -p clickhousectl + - name: Test with experimental access negotiation + run: cargo test -p clickhousectl --features experimental-access-negotiation + - name: Verify install workflow path mapping run: python3 scripts/tests/test_classify_install_integration.py diff --git a/README.md b/README.md index b01aebe2..68e05899 100644 --- a/README.md +++ b/README.md @@ -132,6 +132,48 @@ clickhousectl cloud postgres certs get --output ca.pem Save any initial password and connection string, wait for `state=running`, then connect with `psql` using verified TLS. The [ClickHouse Managed Postgres guide](docs/guides/managed-postgres.md) covers that flow, configuration changes, read-only OAuth queries, and recovery. ClickHouse Managed Postgres commands are beta. +### Discover assignable roles + +```bash +clickhousectl cloud access roles +clickhousectl cloud access roles --org --json +``` + +Discovery uses your refreshed OAuth user identity even when an API key is saved, with the same organization selectors and endpoint precedence as `cloud access request`. Human output identifies the selected organization and lists role IDs/names with policy effects and resource scopes. `--json` returns exactly the complete role array, including all ALLOW/DENY policies and optional SQL tags; an eligible empty menu is `[]`. SQL grant/revoke strings and unknown values remain opaque. Incomplete required discovery data fails before any menu is printed, rather than implying empty permissions or falling back to the administrative catalog. Discovery describes current authority, not a guarantee of later approval or a role recommendation. + +### Request a scoped API key (access request) + +`cloud access request` obtains a scoped API key through the control plane's PKCE approval handshake, so an agent can get a narrowly-scoped key that a human approves in a browser rather than reusing a long-lived one. + +```bash +clickhousectl cloud access request +clickhousectl cloud access request --key-name agent-key --purpose "CI deploy" --tool claude +clickhousectl cloud access request --org --perms role:,role: +clickhousectl cloud access request --no-browser --json +``` + +Use IDs from `cloud access roles` to propose roles with `--perms role:` (repeat the flag or separate tokens with commas). UUIDs are normalized to lowercase and deduplicated by identity in first-seen order; at most 100 distinct roles are allowed. Empty tokens, whitespace, wrong prefixes and malformed UUIDs fail before authentication or network access. Role names, raw permissions and resource overrides are not supported. Omit `--perms` for manual browser selection; an explicitly empty value is invalid. A proposal is not a grant: the server validates the whole proposal atomically, and the human's eventual selection may differ. The CLI neither filters rejected proposals nor retries a subset. + +The command requires your OAuth user identity from `cloud auth login` and refreshes it when needed; API keys from flags, saved credentials, or environment variables are never used for negotiation. Missing or expired identity that cannot be refreshed fails with authentication exit code 4 and login guidance. It generates a PKCE verifier/challenge and a CSRF `state`, starts a loopback redirect server on `127.0.0.1`, and opens the approval URL (printed as a fallback). After you approve it, the CLI claims the issued key with the PKCE verifier and stores the key id and secret in `.clickhouse/credentials.json`. The secret is never printed or placed in a URL. If the browser cannot reach the loopback server, the CLI falls back to polling the request's status until it is approved. Pending approval waits until the server pending deadline or 15 minutes, whichever comes first. One final status check can still discover an already-approved request. + +Use `--no-browser` when another person or device will open the approval link: it suppresses only automatic browser launch, not approval, the loopback listener or polling. The approval URL and progress always go to stderr, including with `--json`; JSON stdout contains only the final success object. Denied, expired and authentication failures stop the wait promptly without replacing saved credentials. Polling retries HTTP 429/5xx within the existing wait; other HTTP failures and malformed status responses terminate. An approved request uses its separate server claim deadline, even after pending expiry; servers missing deadline metadata are unsupported. Each access HTTP operation is bounded to 30 seconds, shortened by its phase deadline. Filing and claiming are not retried: a timeout may leave a filed request or a completed server claim without local credentials. Neither pending nor claim expiry is key expiry. + +Successful requests report the organization and request IDs, the server-saved proposal, and the actual roles returned by the claim. JSON retains `status`, `keyId`, `key`, and `credentialsPath`, and adds `requestId`, `organizationId`, `requestedRoleIds`, and `grantedRoleIds`. With no proposal, `requestedRoleIds` is `[]` (manual selection); the human may trim or add roles, so actual grants need not match the proposal. Missing or invalid server role arrays fail before local credential installation; older servers without this reporting contract are unsupported. + +The API URL comes from `--url` when provided, otherwise the saved OAuth login environment, or the normal Cloud default (`https://api.clickhouse.cloud/v1`) for older tokens without a saved URL. Local development requires an explicit local URL at login or through `--url`; there is no implicit localhost default. Other Cloud commands retain their normal credential precedence, so a saved negotiated key remains available for subsequent operations. Optional `--purpose`, `--tool`, `--version`, and `--session-id` flags record agent context with the request. + +Access organization selection uses your OAuth user's complete, paginated membership list, not the saved API key's organization. A single membership is selected automatically; zero or multiple memberships require action (no remembered default or prompt). Find IDs with `cloud org list --json` and select `cloud access request --org ` (`--org-id` remains supported), or use exact `--org-name`. Conflicting selectors, including across command depths, are usage errors. + +### Debugging which credential source was used + +Pass `--debug` to a Cloud resource command to print the resolved credential source (and the API URL) to stderr before the command runs. This works with and without `--json`. + +Cloud notices on stderr are best-effort: if their reader goes away, the operation still runs and reports its outcome through the exit status. + +When environment credentials are configured but a credentials file or explicit CLI flags take precedence, clickhousectl prints a one-line note to stderr. `cloud auth status` also marks the environment credentials as configured but inactive and identifies the source that outranked them. + +`--url` is an advanced debugging override for targeting a non-production Cloud control plane. Release builds accept the option but omit it from `--help`; development builds show it. For `cloud service query`, set `CLICKHOUSE_CLOUD_QUERY_HOST` separately to override the Query API host. + ### Ingest with ClickPipes Use ClickPipes to replicate database changes or load files into ClickHouse Cloud. Both examples require API-key authentication and a running ClickHouse destination. diff --git a/crates/clickhouse-cloud-api/README.md b/crates/clickhouse-cloud-api/README.md index 8be00485..3a48c6cc 100644 --- a/crates/clickhouse-cloud-api/README.md +++ b/crates/clickhouse-cloud-api/README.md @@ -178,6 +178,26 @@ callers must supply the new fields or use `..Default::default()`. The API requir CDC settings and table mapping fields to match the replication method; snapshot requests omit them. +`organization_assignable_roles_get_list(organization_id)` uses a CLI user Bearer +token to read the complete, unpaginated `ApiResponse>` array +from `/v1/organizations/{organizationId}/roles`. Basic clients fail locally with +`Error::AuthMismatch`; the server verifies the actual CLI user and organization +authority. `organization_roles_get_list` retains its administrative catalog API. +There is no catalog fallback or heuristic choice between the auth-dependent DTOs. +The analyzer allowlists the distinct typed helper for this existing route. + +Assignable roles retain identity strings (including non-v4 system UUIDs), role +type, all ALLOW/DENY policies, permission/resource strings and optional tags. +`grants` strings, including REVOKE statements, are opaque; no SQL parsing or +filtering occurs. This menu describes read-time authority, not an approval promise. + +Transport DTOs follow the library's all-`Option` compatibility policy: missing or +null required fields remain `None`, distinct from explicit empty arrays. Consumers +**must validate** result, role id/name/type/policies, and each policy's +allowDeny/permissions/resources before human **or JSON** success output. Never +convert absent fields into an empty/safe permission claim. Optional tags and their +fields retain absence; unknown enum strings round-trip losslessly. + `openapi_key_get_list(organization_id, limit, cursor)` returns one page of API keys. Migrate existing calls by passing `None, None` for the new arguments. The server accepts limits from 1–250 and defaults to 250; continue with the response's diff --git a/crates/clickhouse-cloud-api/src/client/organizations.rs b/crates/clickhouse-cloud-api/src/client/organizations.rs index 2405a55b..ce526536 100644 --- a/crates/clickhouse-cloud-api/src/client/organizations.rs +++ b/crates/clickhouse-cloud-api/src/client/organizations.rs @@ -87,6 +87,7 @@ impl Client { } Ok(serde_json::from_str(&body_text)?) } + /// Get organization details pub async fn organization_get( &self, @@ -654,6 +655,41 @@ impl Client { Ok(serde_json::from_str(&body_text)?) } + /// Discover the complete, unpaginated caller-assignable role array. + /// + /// Requires Bearer authentication; the server verifies the token's CLI user + /// identity and current organization authority. This is a read-time snapshot, + /// not an approval guarantee. Never falls back to the administrative catalog. + /// + /// Transport fields tolerate missing/null values. Callers must validate the + /// required discovery fields before human or JSON success output: absent + /// result/policies/permissions/resources are not empty permission guarantees. + pub async fn organization_assignable_roles_get_list( + &self, + organization_id: &str, + ) -> Result>, Error> { + if !matches!(&self.auth, super::Auth::Bearer { .. }) { + return Err(Error::AuthMismatch( + "organization_assignable_roles_get_list requires a CLI user Bearer token".into(), + )); + } + let path = format!("/v1/organizations/{organization_id}/roles"); + let req = self.request(reqwest::Method::GET, &path); + let resp = req.send().await?; + let status = resp.status(); + let body_text = resp.text().await?; + if !status.is_success() { + return Err(Error::Api { + status: status.as_u16(), + message: serde_json::from_str::>(&body_text) + .ok() + .and_then(|r| r.error) + .unwrap_or(body_text.clone()), + }); + } + Ok(serde_json::from_str(&body_text)?) + } + /// List all available roles for an organization pub async fn organization_roles_get_list( &self, diff --git a/crates/clickhouse-cloud-api/src/models.rs b/crates/clickhouse-cloud-api/src/models.rs index 59c15b6d..48c2fad8 100644 --- a/crates/clickhouse-cloud-api/src/models.rs +++ b/crates/clickhouse-cloud-api/src/models.rs @@ -223,9 +223,10 @@ pub use query_api_endpoints::{ }; pub use quotas::{OrganizationQuota, OrganizationQuotaQuotacode, OrganizationQuotaScope}; pub use rbac::{ - RBACPolicy, RBACPolicyAllowdeny, RBACPolicyCreateRequest, RBACPolicyCreateRequestAllowdeny, - RBACPolicyTags, RBACPolicyTagsResponse, RBACPolicyTagsRolev2, RBACRole, RBACRoleType, - RoleCreateRequest, RoleUpdateRequest, + AssignableRole, AssignableRolePolicy, AssignableRolePolicyTags, RBACPolicy, + RBACPolicyAllowdeny, RBACPolicyCreateRequest, RBACPolicyCreateRequestAllowdeny, RBACPolicyTags, + RBACPolicyTagsResponse, RBACPolicyTagsRolev2, RBACRole, RBACRoleType, RoleCreateRequest, + RoleUpdateRequest, }; pub use saved_queries::{PublicSavedQuery, PublicSavedQueryListItem, PublicSavedQueryRequest}; pub use scim::{ diff --git a/crates/clickhouse-cloud-api/src/models/rbac.rs b/crates/clickhouse-cloud-api/src/models/rbac.rs index bc3f6a2e..4ad6bf28 100644 --- a/crates/clickhouse-cloud-api/src/models/rbac.rs +++ b/crates/clickhouse-cloud-api/src/models/rbac.rs @@ -162,6 +162,46 @@ pub struct RBACRole { pub updated_at: Option>, } +/// A caller-assignable role from verified CLI user discovery, not the administrative catalog. +/// +/// Fields remain optional for transport compatibility. Before presenting a successful +/// discovery result, callers must require `id`, `name`, `type`, `policies`, and each +/// policy's effect, permissions and resources. Absence is not an empty permission set. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub struct AssignableRole { + /// Opaque identity string, including system identifiers that are not UUID v4. + #[serde(skip_serializing_if = "Option::is_none")] + pub id: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub name: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub r#type: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub policies: Option>, +} + +/// A complete policy in an assignable role; no effects or resource scopes are filtered. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub struct AssignableRolePolicy { + #[serde(rename = "allowDeny", skip_serializing_if = "Option::is_none")] + pub allow_deny: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub permissions: Option>, + #[serde(skip_serializing_if = "Option::is_none")] + pub resources: Option>, + #[serde(skip_serializing_if = "Option::is_none")] + pub tags: Option, +} + +/// Meaningful policy tags. SQL statements are opaque, including GRANT and REVOKE text. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub struct AssignableRolePolicyTags { + #[serde(skip_serializing_if = "Option::is_none")] + pub grants: Option>, + #[serde(rename = "roleV2", skip_serializing_if = "Option::is_none")] + pub role_v2: Option, +} + /// `RoleCreateRequest` from the ClickHouse Cloud API. #[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] pub struct RoleCreateRequest { diff --git a/crates/clickhouse-cloud-api/tests/client_test.rs b/crates/clickhouse-cloud-api/tests/client_test.rs index 2797ed8a..f336dfce 100644 --- a/crates/clickhouse-cloud-api/tests/client_test.rs +++ b/crates/clickhouse-cloud-api/tests/client_test.rs @@ -70,6 +70,141 @@ async fn list_organizations() { assert_eq!(orgs[0].name.as_deref(), Some("Test Org")); } +#[tokio::test] +async fn organization_assignable_roles_get_list_preserves_full_policy_array() { + let server = MockServer::start().await; + let roles = serde_json::json!([ + {"id": "00000000-0000-0000-0000-000000000001", "name": "System reader", "type": "system", "policies": [ + {"allowDeny": "ALLOW", "permissions": ["control-plane:service:view", "future:permission"], "resources": ["*", "instance/service-a"], "tags": {"roleV2": "sql-console-readonly"}}, + {"allowDeny": "DENY", "permissions": ["control-plane:service:delete"], "resources": ["instance/service-b"]} + ]}, + {"id": "55555555-5555-4555-8555-555555555555", "name": "Custom SQL", "type": "custom", "policies": [ + {"allowDeny": "ALLOW", "permissions": ["sql-console:database:access"], "resources": ["instance/service-a"], "tags": {"roleV2": "sql-console-admin", "grants": ["GRANT SELECT ON db.* TO role", "REVOKE SELECT ON db.secret FROM role"]}} + ]} + ]); + Mock::given(method("GET")) + .and(path( + "/v1/organizations/11111111-1111-4111-8111-111111111111/roles", + )) + .and(bearer_token("cli-user-token")) + .respond_with(ok_json(roles.clone())) + .expect(1) + .mount(&server) + .await; + let client = + Client::with_http_client_bearer(reqwest::Client::new(), server.uri(), "cli-user-token"); + let response = client + .organization_assignable_roles_get_list("11111111-1111-4111-8111-111111111111") + .await + .unwrap(); + assert_eq!(response.status, Some(200)); + assert_eq!(response.request_id.as_deref(), Some("req-test")); + let result = response.result.unwrap(); + assert_eq!(result[0].r#type, Some(RBACRoleType::System)); + assert_eq!(result[1].r#type, Some(RBACRoleType::Custom)); + assert_eq!( + result[0].policies.as_ref().unwrap()[1].allow_deny, + Some(RBACPolicyAllowdeny::DENY) + ); + assert_eq!(serde_json::to_value(result).unwrap(), roles); + let requests = server.received_requests().await.unwrap(); + assert_eq!(requests.len(), 1); + assert!(requests[0].url.query().is_none()); + assert!(requests[0].body.is_empty()); +} + +#[tokio::test] +async fn organization_assignable_roles_get_list_rejects_basic_without_catalog_fallback() { + let (server, client) = setup().await; + assert!(matches!( + client.organization_assignable_roles_get_list("org").await, + Err(clickhouse_cloud_api::Error::AuthMismatch(_)) + )); + assert!(server.received_requests().await.unwrap().is_empty()); + + // The pre-existing administrative method still uses Basic and its legacy DTO. + Mock::given(method("GET")) + .and(path("/v1/organizations/org/roles")) + .and(basic_auth("key", "secret")) + .respond_with(ok_json( + serde_json::json!([{"id": "role", "actors": ["actor"], "ownerId": "owner"}]), + )) + .expect(1) + .mount(&server) + .await; + let response = client.organization_roles_get_list("org").await.unwrap(); + let result = response.result.unwrap(); + assert_eq!(result[0].actors.as_ref().unwrap(), &["actor"]); + assert_eq!(result[0].owner_id.as_deref(), Some("owner")); +} + +#[tokio::test] +async fn organization_assignable_roles_get_list_preserves_errors_and_incomplete_envelopes() { + for (status, body) in [ + (401, r#"{"error":"login required"}"#), + (403, r#"{"error":"not authorized"}"#), + (400, r#"{"error":"RBAC v2 required"}"#), + (500, "upstream failure"), + (200, "not-json"), + (200, r#"{"result":{"roles":[]}}"#), + (200, "[]"), + (200, r#"{"result":[{"policies":"wrong-type"}]}"#), + (200, r#"{"error":"envelope error"}"#), + (200, "{}"), + (200, r#"{"result":null}"#), + (200, r#"{"result":[]}"#), + ] { + let server = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/v1/organizations/org/roles")) + .and(bearer_token("cli-user-token")) + .respond_with(ResponseTemplate::new(status).set_body_string(body)) + .expect(1) + .mount(&server) + .await; + let client = + Client::with_http_client_bearer(reqwest::Client::new(), server.uri(), "cli-user-token"); + let response = client.organization_assignable_roles_get_list("org").await; + if status != 200 { + let Err(clickhouse_cloud_api::Error::Api { + status: actual, + message, + }) = response + else { + panic!("expected API error") + }; + assert_eq!(actual, status); + let expected = serde_json::from_str::(body) + .ok() + .and_then(|body| body["error"].as_str().map(str::to_owned)) + .unwrap_or_else(|| body.into()); + assert_eq!(message, expected); + } else if matches!( + body, + "{}" | r#"{"result":null}"# | r#"{"error":"envelope error"}"# + ) { + let response = response.unwrap(); + assert!(response.result.is_none()); + assert_eq!( + response.error.as_deref(), + if body.contains("envelope error") { + Some("envelope error") + } else { + None + } + ); + } else if body == r#"{"result":[]}"# { + assert_eq!(response.unwrap().result, Some(vec![])); + } else { + assert!( + matches!(response, Err(clickhouse_cloud_api::Error::Json(_))), + "{body}" + ); + } + assert_eq!(server.received_requests().await.unwrap().len(), 1); + } +} + #[tokio::test] async fn get_organization() { let mock_server = MockServer::start().await; diff --git a/crates/clickhouse-cloud-api/tests/models_test.rs b/crates/clickhouse-cloud-api/tests/models_test.rs index 506a0680..e7f18408 100644 --- a/crates/clickhouse-cloud-api/tests/models_test.rs +++ b/crates/clickhouse-cloud-api/tests/models_test.rs @@ -75,6 +75,94 @@ fn aws_backup_external_id_update_preserves_omit_set_and_clear() { ); } +#[test] +fn assignable_role_required_fields_preserve_missing_null_and_empty_distinctions() { + let role = serde_json::json!({"id": "00000000-0000-0000-0000-000000000001", "name": "role", "type": "system", "policies": []}); + for field in ["id", "name", "type", "policies"] { + for null in [false, true] { + let mut wire = role.clone(); + if null { + wire[field] = serde_json::Value::Null; + } else { + wire.as_object_mut().unwrap().remove(field); + } + let parsed: AssignableRole = serde_json::from_value(wire).unwrap(); + match field { + "id" => assert!(parsed.id.is_none()), + "name" => assert!(parsed.name.is_none()), + "type" => assert!(parsed.r#type.is_none()), + "policies" => assert!(parsed.policies.is_none()), + _ => unreachable!(), + } + assert!(serde_json::to_value(parsed).unwrap().get(field).is_none()); + } + } + let parsed: AssignableRole = serde_json::from_value(role).unwrap(); + assert_eq!(parsed.policies, Some(vec![])); + let policy = serde_json::json!({"allowDeny": "DENY", "permissions": [], "resources": []}); + for field in ["allowDeny", "permissions", "resources", "tags"] { + for null in [false, true] { + let mut wire = policy.clone(); + if null { + wire[field] = serde_json::Value::Null; + } else { + wire.as_object_mut().unwrap().remove(field); + } + let parsed: AssignableRolePolicy = serde_json::from_value(wire).unwrap(); + match field { + "allowDeny" => assert!(parsed.allow_deny.is_none()), + "permissions" => assert!(parsed.permissions.is_none()), + "resources" => assert!(parsed.resources.is_none()), + "tags" => assert!(parsed.tags.is_none()), + _ => unreachable!(), + } + assert!(serde_json::to_value(parsed).unwrap().get(field).is_none()); + } + } + let parsed: AssignableRolePolicy = serde_json::from_value(policy).unwrap(); + assert_eq!(parsed.permissions, Some(vec![])); + assert_eq!(parsed.resources, Some(vec![])); + for wire in [ + serde_json::json!({}), + serde_json::json!({"grants": null, "roleV2": null}), + ] { + let parsed: AssignableRolePolicyTags = serde_json::from_value(wire).unwrap(); + assert!(parsed.grants.is_none()); + assert!(parsed.role_v2.is_none()); + assert_eq!(serde_json::to_value(parsed).unwrap(), serde_json::json!({})); + } + let parsed: AssignableRolePolicyTags = + serde_json::from_value(serde_json::json!({"grants": []})).unwrap(); + assert_eq!(parsed.grants, Some(vec![])); +} + +#[test] +fn assignable_role_preserves_unknown_enum_values_without_legacy_catalog_fields() { + let wire = serde_json::json!({"id": "role", "name": "future", "type": "future-type", "policies": [ + {"allowDeny": "future-effect", "permissions": ["future:permission"], "resources": ["future/resource"], "tags": {"roleV2": "future-role", "grants": ["REVOKE ALL ON *.* FROM role"]}} + ]}); + let mut extra = wire.clone(); + extra["actors"] = serde_json::json!(["internal"]); + extra["tenantId"] = "tenant".into(); + extra["policies"][0]["id"] = "policy-db-id".into(); + extra["policies"][0]["futureField"] = true.into(); + let parsed: AssignableRole = serde_json::from_value(extra).unwrap(); + assert_eq!( + parsed.r#type, + Some(RBACRoleType::Unknown("future-type".into())) + ); + let policy = &parsed.policies.as_ref().unwrap()[0]; + assert_eq!( + policy.allow_deny, + Some(RBACPolicyAllowdeny::Unknown("future-effect".into())) + ); + assert_eq!( + policy.tags.as_ref().unwrap().role_v2, + Some(RBACPolicyTagsRolev2::Unknown("future-role".into())) + ); + assert_eq!(serde_json::to_value(parsed).unwrap(), wire); +} + #[test] fn snapshot_configuration_tolerates_missing_null_and_extra_fields() { for wire in [ diff --git a/crates/clickhouse-openapi-analyzer/src/config.rs b/crates/clickhouse-openapi-analyzer/src/config.rs index 6adf2053..8e6e73cd 100644 --- a/crates/clickhouse-openapi-analyzer/src/config.rs +++ b/crates/clickhouse-openapi-analyzer/src/config.rs @@ -51,6 +51,9 @@ pub fn clickhouse_cloud_config() -> AnalyzerConfig { "run_query", "run_query_bearer", "run_postgres_query_bearer", + // Explicit verified-CLI auth lane of organizationRolesGetList; the + // administrative method keeps its legacy RBACRole response contract. + "organization_assignable_roles_get_list", ]), optionality_exemptions: pairs(OPTIONALITY_EXEMPTIONS), fractional_response_exemptions: pairs(FRACTIONAL_RESPONSE_EXEMPTIONS), diff --git a/crates/clickhousectl/Cargo.toml b/crates/clickhousectl/Cargo.toml index ccecbebb..1de0813e 100644 --- a/crates/clickhousectl/Cargo.toml +++ b/crates/clickhousectl/Cargo.toml @@ -26,6 +26,11 @@ telemetry = [] # fields are present on the response structs and surfaced in CLI output. Off by # default so the CLI never shows a field the API has deprecated. deprecated-fields = ["clickhouse-cloud-api/deprecated-fields"] +# Compiles in the experimental `cloud access` flow (assignable-role discovery +# and the scoped-API-key PKCE approval handshake). Off by default so the +# published binary carries none of it; opt in locally with +# `--features experimental-access-negotiation`. +experimental-access-negotiation = [] [dependencies] clap = { version = "4", features = ["derive"] } diff --git a/crates/clickhousectl/src/cloud/access.rs b/crates/clickhousectl/src/cloud/access.rs new file mode 100644 index 00000000..945b3f16 --- /dev/null +++ b/crates/clickhousectl/src/cloud/access.rs @@ -0,0 +1,2342 @@ +//! `cloud access`: discover assignable roles and obtain scoped API keys. +//! Role discovery uses the typed API client; requests use the PKCE handshake. +//! +//! The user authenticates with their existing CLI identity (OAuth Bearer from +//! `cloud auth login`), files an access request carrying a PKCE +//! challenge and a CSRF `state`, approves it in a browser, and the CLI claims +//! the issued key over a loopback redirect — falling back to polling the +//! request's status. The claimed key id and secret are stored with the CLI's +//! existing credential storage; the secret is never printed or placed in a URL. +//! +//! The handshake endpoints are not modelled by the typed `clickhouse-cloud-api` client, +//! so the handshake calls them directly with a plain `reqwest` client carrying +//! the credentials resolved by [`CloudClient`] ([`CloudClient::request_auth`]). + +use crate::cloud::client::{CloudClient, CloudError, RequestAuth, Result as CloudResult}; +use crate::cloud::credentials; +use crate::cloud::output::eprint_line; +use crate::cloud::permissions::Declaration as Permission; + +// Access negotiation runs on the user's OAuth (Bearer) identity, not an API +// key, and reaches lanes the typed client does not model as metered +// operations: `access roles` reads the verified-CLI assignable-role lane and +// `access request` runs the PKCE handshake. Declared as authorization, not API +// operations, and unscoped because org selection is resolved from the user's +// own memberships rather than the `--org-id`/organization-list API path. +pub(super) const PERMISSIONS: &[Permission] = &[ + Permission::non_api( + "access roles", + "Requires OAuth user identity (`cloud auth login`); reads the verified-CLI assignable-role lane.", + ) + .unscoped(), + Permission::non_api( + "access request", + "Requires OAuth user identity (`cloud auth login`); runs the browser PKCE approval handshake.", + ) + .unscoped(), +]; +use base64::Engine; +use base64::engine::general_purpose::URL_SAFE_NO_PAD; +use clap::{Args, Subcommand}; +use clickhouse_cloud_api::models::{AssignableRole, RBACPolicyAllowdeny, RBACRoleType}; +use serde::Serialize; +use serde::de::DeserializeOwned; +use sha2::{Digest, Sha256}; +use std::collections::{HashMap, HashSet}; +use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; +use tabled::{Table, Tabled, settings::Style}; +use tokio::io::{AsyncReadExt, AsyncWriteExt}; +use tokio::net::TcpListener; + +/// How long to wait for a human to approve the request before giving up. +const APPROVAL_TIMEOUT: Duration = Duration::from_secs(15 * 60); + +/// How often to poll the request's status as a fallback to the loopback redirect. +const POLL_INTERVAL: Duration = Duration::from_secs(3); + +/// The access-operation bound: every single access HTTP request — discovery +/// through the typed client, and each handshake step — must give up after +/// this long so a server holding a response open cannot stall the command. +pub(super) const OPERATION_TIMEOUT: Duration = Duration::from_secs(30); +const MAX_EPOCH_MILLIS: i64 = 8_640_000_000_000_000; + +#[derive(Subcommand)] +pub enum AccessCommands { + /// List caller-assignable roles with policy scopes + #[command(after_help = "\ +CONTEXT FOR AGENTS: + Returns current assignable roles, not a guarantee of later approval. + JSON output is the complete role array, including all policies and tags.")] + Roles, + /// Request a scoped API key via browser approval + #[command(after_help = "\ +CONTEXT FOR AGENTS: + Requires OAuth from `cloud auth login`, even with a saved API key; roles are per organization: pass --org-id. + Each command's `--help` declares `API key permission:` needs; take conditionals only if their condition holds. + `Authorization:` needs have no granting role. + `cloud access roles --json` lists roles (empty: stop); policies `ALLOW`/`DENY` permissions on `*` or named resources. + Choose roles covering every need with `ALLOW`, none denied by chosen roles, and the fewest extras. + Propose with `--perms` and `--purpose`; a human approves the stderr URL within 15 minutes; check `grantedRoleIds`. + The approved key becomes the active credential; changing permissions means a new flow (API keys cannot read roles).")] + Request(AccessRequestArgs), +} + +impl AccessCommands { + /// The access-request flow authenticates as the user to *request* access; + /// OAuth (read-only) identity is the intended credential, so it is not a + /// write command and must not be blocked by the OAuth write gate. + pub fn is_write(&self) -> bool { + match self { + AccessCommands::Request(_) | AccessCommands::Roles => false, + } + } +} + +#[derive(Args)] +pub struct AccessRequestArgs { + /// Print the approval URL without opening a browser + #[arg(long)] + pub no_browser: bool, + + /// Proposed role: tokens, comma-separated or repeated (max 100) + #[arg(long, value_name = "role:UUID", value_delimiter = ',', value_parser = parse_requested_role)] + pub perms: Vec, + + /// Name to give the issued API key + #[arg(long)] + pub key_name: Option, + + /// Agent purpose recorded with the request + #[arg(long)] + pub purpose: Option, + + /// Agent tool name recorded with the request + #[arg(long)] + pub tool: Option, + + /// Agent tool version recorded with the request + #[arg(long)] + pub version: Option, + + /// Agent session id recorded with the request + #[arg(long)] + pub session_id: Option, +} + +// Keep the token grammar narrower than Uuid::parse_str (which also accepts +// compact, braced and URN representations). UUID version is unrestricted. +fn parse_requested_role(token: &str) -> Result { + let id = token.strip_prefix("role:").ok_or("expected role:")?; + let parsed = uuid::Uuid::parse_str(id).map_err(|_| "expected role:")?; + if !id.eq_ignore_ascii_case(&parsed.hyphenated().to_string()) { + return Err("expected role: without whitespace"); + } + Ok(parsed) +} + +impl AccessRequestArgs { + pub fn requested_role_ids(&self) -> Result>, &'static str> { + if self.perms.is_empty() { + return Ok(None); + } + let mut seen = HashSet::new(); + let ids: Vec<_> = self + .perms + .iter() + .filter(|id| seen.insert(**id)) + .map(|id| id.hyphenated().to_string()) + .collect(); + if ids.len() > 100 { + return Err("--perms accepts at most 100 distinct roles"); + } + Ok(Some(ids)) + } +} + +// ── request/response bodies ──────────────────────────────────────────────── + +#[derive(Serialize)] +#[serde(rename_all = "camelCase")] +struct AgentContext<'a> { + #[serde(skip_serializing_if = "Option::is_none")] + purpose: Option<&'a str>, + #[serde(skip_serializing_if = "Option::is_none")] + tool: Option<&'a str>, + #[serde(skip_serializing_if = "Option::is_none")] + version: Option<&'a str>, + #[serde(skip_serializing_if = "Option::is_none")] + session_id: Option<&'a str>, +} + +#[derive(Serialize)] +#[serde(rename_all = "camelCase")] +struct CreateAccessRequestBody<'a> { + #[serde(skip_serializing_if = "Option::is_none")] + requested_role_ids: Option<&'a [String]>, + #[serde(skip_serializing_if = "Option::is_none")] + key_name: Option<&'a str>, + #[serde(skip_serializing_if = "Option::is_none")] + agent: Option>, + code_challenge: &'a str, + code_challenge_method: &'a str, + state: &'a str, + redirect_uri: &'a str, +} + +/// Every `/v1` response from the openapi app wraps its payload as +/// `{ "result": , "requestId": }`; the handshake reads +/// `result` and ignores the correlation id. +#[derive(serde::Deserialize)] +struct Envelope { + result: T, + error: Option, + status: Option, +} + +#[derive(serde::Deserialize)] +#[serde(rename_all = "camelCase")] +struct CreateAccessRequestResponse { + requested_role_ids: Option>, + request_id: String, + status: Option, + approval_url: String, + /// Pending approval deadline only, in epoch milliseconds. + expires_at: Option, +} + +#[derive(serde::Deserialize)] +#[serde(rename_all = "camelCase")] +struct AccessRequestStatus { + requested_role_ids: Option>, + request_id: Option, + expires_at: Option, + status: Option, + claim_code: Option, + claim_code_expires_at: Option, + issued_key_id: Option, +} + +#[derive(Serialize)] +#[serde(rename_all = "camelCase")] +struct ClaimBody<'a> { + claim_code: &'a str, + code_verifier: &'a str, +} + +#[derive(serde::Deserialize)] +#[serde(rename_all = "camelCase")] +struct ClaimResponse { + granted_role_ids: Option>, + key_id: String, + key: String, + secret: String, +} + +// Server arrays are authoritative: reject duplicate identities rather than silently +// deduplicating them as we do for user proposal input. Never echo invalid values. +fn canonical_role_ids(ids: Option<&[String]>, proposal: bool) -> CloudResult> { + let field = if proposal { + "requestedRoleIds" + } else { + "grantedRoleIds" + }; + let invalid = || { + CloudError::new(format!( + "access request: missing or invalid {field}; server role reporting support is required" + )) + }; + let ids = ids.ok_or_else(invalid)?; + if (proposal && ids.len() > 100) || (!proposal && ids.is_empty()) { + return Err(invalid()); + } + let mut seen = HashSet::new(); + ids.iter() + .map(|id| { + let parsed = uuid::Uuid::parse_str(id).map_err(|_| invalid())?; + let canonical = parsed.hyphenated().to_string(); + if !id.eq_ignore_ascii_case(&canonical) || !seen.insert(parsed) { + return Err(invalid()); + } + Ok(canonical) + }) + .collect() +} + +// ── PKCE ─────────────────────────────────────────────────────────────────── + +struct Pkce { + verifier: String, + challenge: String, +} + +/// PKCE pair (RFC 7636): a random 32-byte verifier, base64url-encoded, and its +/// S256 challenge `base64url(SHA256(verifier))`. +fn generate_pkce() -> Pkce { + let verifier = URL_SAFE_NO_PAD.encode(random_bytes::<32>()); + let challenge = URL_SAFE_NO_PAD.encode(Sha256::digest(verifier.as_bytes())); + Pkce { + verifier, + challenge, + } +} + +/// A random, URL-safe CSRF nonce tying the filed request to its redirect. +fn generate_state() -> String { + URL_SAFE_NO_PAD.encode(random_bytes::<16>()) +} + +fn random_bytes() -> [u8; N] { + use rand::Rng; + let mut bytes = [0u8; N]; + rand::rng().fill_bytes(&mut bytes); + bytes +} + +// ── entry point ───────────────────────────────────────────────────────────── + +impl CloudClient { + async fn list_assignable_roles(&self, org_id: &str) -> CloudResult> { + let response = self + .api() + .organization_assignable_roles_get_list(org_id) + .await + .map_err(|error| self.convert_error_for_organization(error, org_id))?; + if let Some(error) = &response.error { + return Err(CloudError::new(format!("Role discovery failed: {error}"))); + } + if let Some(status) = response.status.filter(|status| *status >= 400) { + return Err(CloudError::new(format!( + "Role discovery failed with envelope status {status}" + ))); + } + let roles = Self::unwrap_response(response)?; + validate_assignable_roles(&roles)?; + Ok(roles) + } + + /// Read all user memberships before auto-selection or exact-name selection. + async fn resolve_access_organization_id( + &self, + org_id: Option, + org_name: Option, + ) -> CloudResult { + if org_id.is_some() && org_name.is_some() { + return Err(CloudError::usage( + "Organization ID conflicts with --org-name", + )); + } + if let Some(id) = org_id { + return Ok(id); + } + let mut organizations = Vec::new(); + let mut cursor = None; + let mut seen_cursors = std::collections::HashSet::new(); + let mut seen_ids = std::collections::HashSet::new(); + loop { + let response = self + .api() + .organization_get_list(None, cursor.as_deref()) + .await + .map_err(|error| self.convert_error(error))?; + let next_cursor = response.next_cursor.clone(); + let page = Self::unwrap_response(response)?; + let had_rows = !page.is_empty(); + let previous_count = organizations.len(); + for organization in page { + let id = organization.id.ok_or_else(|| CloudError::new( + "Organization response is missing its id; cannot safely select an organization", + ))?; + if seen_ids.insert(id) { + organizations.push((id, organization)); + } + } + match next_cursor { + None => break, + Some(next) => { + if (had_rows && previous_count == organizations.len()) + || !seen_cursors.insert(next.clone()) + { + return Err(CloudError::new( + "Organization pagination made no progress; cannot safely select an organization", + )); + } + cursor = Some(next); + } + } + } + if let Some(name) = org_name { + return crate::cloud::shared::select_named_id( + "organization", + &name, + organizations + .iter() + .map(|(id, org)| (org.name.as_deref(), Some(id))), + ); + } + match organizations.as_slice() { + [] => Err(CloudError::new( + "Your OAuth user has no organizations. Join or create an organization, then run `clickhousectl cloud org list --json`.", + )), + [(id, _)] => Ok(id.to_string()), + _ => Err(CloudError::new( + "Your OAuth user belongs to multiple organizations. Run `clickhousectl cloud org list --json` and select an ID with --org (or --org-id).", + )), + } + } +} + +pub async fn run( + command: AccessCommands, + url_override: Option<&str>, + org_id: Option, + org_name: Option, + debug: bool, + json: bool, +) -> CloudResult<()> { + match command { + AccessCommands::Request(args) => { + request(args, url_override, org_id, org_name, debug, json).await + } + AccessCommands::Roles => roles(url_override, org_id, org_name, debug, json).await, + } +} + +/// Validate the entire menu before either output mode can claim success. +fn validate_assignable_roles(roles: &[AssignableRole]) -> CloudResult<()> { + fn required(value: &Option, field: &str) -> CloudResult<()> { + if value.is_none() { + return Err(CloudError::new(format!( + "Incomplete role discovery response: missing {field}" + ))); + } + Ok(()) + } + for role in roles { + required(&role.id, "role.id")?; + if let Some(id) = &role.id { + uuid::Uuid::parse_str(id).map_err(|_| { + CloudError::new("Invalid role discovery response: role.id is not a UUID") + })?; + } + required(&role.name, "role.name")?; + required(&role.r#type, "role.type")?; + required(&role.policies, "role.policies")?; + if let Some(policies) = &role.policies { + for policy in policies { + required(&policy.allow_deny, "policy.allowDeny")?; + required(&policy.permissions, "policy.permissions")?; + required(&policy.resources, "policy.resources")?; + } + } + } + Ok(()) +} + +async fn roles( + url_override: Option<&str>, + org_id: Option, + org_name: Option, + debug: bool, + json: bool, +) -> CloudResult<()> { + let client = CloudClient::for_access(url_override).await?; + if debug { + super::print_client_diagnostics(&client); + } + let org_id = client + .resolve_access_organization_id(org_id, org_name) + .await?; + let roles = client.list_assignable_roles(&org_id).await?; + if json { + println!("{}", serde_json::to_string_pretty(&roles)?); + } else { + print_assignable_roles(&org_id, &roles); + } + Ok(()) +} + +fn print_assignable_roles(org_id: &str, roles: &[AssignableRole]) { + use crate::cloud::output::or_absent; + #[derive(Tabled)] + struct RoleRow { + id: String, + name: String, + r#type: String, + policies: String, + } + let rows = roles + .iter() + .map(|role| { + let role_type = match &role.r#type { + Some(RBACRoleType::Unknown(value)) => format!("Unknown ({value})"), + other => or_absent(other.as_ref()).to_string(), + }; + let mut summaries = Vec::new(); + if let Some(policies) = &role.policies { + if policies.is_empty() { + summaries.push("No policies".into()); + } + for policy in policies { + let effect = match &policy.allow_deny { + Some(RBACPolicyAllowdeny::Unknown(value)) => { + format!("Unknown effect ({value})") + } + other => or_absent(other.as_ref()).to_string(), + }; + if let (Some(permissions), Some(resources)) = + (&policy.permissions, &policy.resources) + { + summaries.push(format!( + "{effect}: permissions [{}]; resources [{}]", + permissions.join(", "), + resources.join(", ") + )); + } + } + } + RoleRow { + id: or_absent(role.id.as_ref()).to_string(), + name: or_absent(role.name.as_ref()).to_string(), + r#type: role_type, + policies: summaries.join("\n"), + } + }) + .collect::>(); + println!("Organization: {org_id}"); + if rows.is_empty() { + println!("No assignable roles for your user in this organization."); + } else { + println!("{}", Table::new(rows).with(Style::markdown())); + } +} + +/// The client for filing, polling, and claiming: like `oauth_client()` in +/// `auth.rs`, it never follows redirects. A 307/308 from the claim endpoint +/// would replay the JSON body — `claimCode` and the secret PKCE +/// `code_verifier` — to another origin, and dropping the `Authorization` +/// header on cross-origin redirects does not protect those body fields. +/// Reject redirects instead of forwarding secrets or replaying the claim POST. +fn handshake_client() -> CloudResult { + crate::http::client_builder() + .redirect(reqwest::redirect::Policy::none()) + .build() + .map_err(|error| CloudError::new(format!("failed to build HTTP client: {error}"))) +} + +async fn request( + args: AccessRequestArgs, + url_override: Option<&str>, + org_id: Option, + org_name: Option, + debug: bool, + json: bool, +) -> CloudResult<()> { + let requested_role_ids = args.requested_role_ids().map_err(CloudError::usage)?; + // Resolve the user identity and endpoint once for organization discovery + // and every step of the existing PKCE handshake. + let client = CloudClient::for_access(url_override).await?; + if debug { + super::print_client_diagnostics(&client); + } + let org_id = client + .resolve_access_organization_id(org_id, org_name) + .await?; + let auth = client.request_auth(); + let base_url = client.base_url().to_string(); + + let http = handshake_client()?; + + // Start the loopback redirect target before filing, so the port is known. + let listener = TcpListener::bind(("127.0.0.1", 0)) + .await + .map_err(|error| CloudError::new(format!("failed to start loopback server: {error}")))?; + let port = listener + .local_addr() + .map_err(|error| CloudError::new(format!("failed to read loopback address: {error}")))? + .port(); + let redirect_uri = format!("http://127.0.0.1:{port}/callback"); + + let pkce = generate_pkce(); + let state = generate_state(); + + let agent = agent_context(&args); + let create_url = format!("{base_url}/organizations/{org_id}/accessRequests"); + let create_body = CreateAccessRequestBody { + requested_role_ids: requested_role_ids.as_deref(), + key_name: args.key_name.as_deref(), + agent, + code_challenge: &pkce.challenge, + code_challenge_method: "S256", + state: &state, + redirect_uri: &redirect_uri, + }; + let created: CreateAccessRequestResponse = operation( + OPERATION_TIMEOUT, + send_json( + http.post(&create_url), + &auth, + &create_body, + "file access request", + true, + ), + ) + .await + .map_err(|error| { + error.into_cloud("file access request timed out; the request may have been filed") + })?; + let pending = PendingApproval::new(&created)?; + if pending.requested_role_ids.as_slice() != requested_role_ids.as_deref().unwrap_or_default() { + return Err(CloudError::new( + "file access request: saved proposal differs from submitted proposal", + )); + } + let saved_requested_role_ids = pending.requested_role_ids.clone(); + + approval_handoff(&created.approval_url, args.no_browser, |url| { + open::that(url) + }); + + let status_url = format!( + "{base_url}/organizations/{org_id}/accessRequests/{}", + created.request_id + ); + + // A callback only wakes the single status/claim owner; GET supplies authority. + let permit = wait_for_approval(&http, &status_url, &auth, pending, listener, &state).await?; + let claim_url = format!("{status_url}/claim"); + let claimed = claim_access_key(&http, &claim_url, &auth, &permit, &pkce.verifier).await?; + + if claimed.key_id.is_empty() || claimed.key.is_empty() || claimed.secret.is_empty() { + return Err(CloudError::new("claim access key: incomplete response")); + } + + let granted_role_ids = canonical_role_ids(claimed.granted_role_ids.as_deref(), false)?; + let credentials_path = credentials::credentials_path().display().to_string(); + // Prepare the complete output before the only credential write. Actual roles + // come only from the successful claim, never from the proposal or status GET. + let json_result = if json { + Some(serde_json::to_string_pretty(&serde_json::json!({ + "status": "approved", + "keyId": claimed.key_id, + "key": claimed.key, + "credentialsPath": credentials_path, + "requestId": created.request_id, + "organizationId": org_id, + "requestedRoleIds": saved_requested_role_ids, + "grantedRoleIds": granted_role_ids, + }))?) + } else { + None + }; + + // Never print the secret; store it with the CLI's existing credential file. + credentials::set_api_credentials(claimed.key.clone(), claimed.secret)?; + + if let Some(out) = json_result { + println!("{out}"); + } else { + println!("Access approved. Scoped API key stored."); + println!(" Organization: {org_id}"); + println!(" Request ID: {}", created.request_id); + if saved_requested_role_ids.is_empty() { + println!(" Requested roles: none (manual selection)"); + } else { + println!(" Requested roles: {}", saved_requested_role_ids.join(", ")); + } + println!(" Granted roles: {}", granted_role_ids.join(", ")); + println!(" Key ID: {}", claimed.key_id); + println!(" Saved to: {credentials_path}"); + } + + Ok(()) +} + +fn approval_handoff( + approval_url: &str, + no_browser: bool, + open_browser: impl FnOnce(&str) -> std::io::Result<()>, +) { + eprint_line(format!( + "Approve this access request in your browser:\n {approval_url}" + )); + if !no_browser && open_browser(approval_url).is_err() { + eprint_line("Could not open a browser automatically. Open the URL above to continue."); + } + eprint_line("Waiting for approval (up to 15 minutes)..."); +} + +fn agent_context(args: &AccessRequestArgs) -> Option> { + if args.purpose.is_none() + && args.tool.is_none() + && args.version.is_none() + && args.session_id.is_none() + { + return None; + } + Some(AgentContext { + purpose: args.purpose.as_deref(), + tool: args.tool.as_deref(), + version: args.version.as_deref(), + session_id: args.session_id.as_deref(), + }) +} + +fn authorized(builder: reqwest::RequestBuilder, auth: &RequestAuth) -> reqwest::RequestBuilder { + match auth { + RequestAuth::Basic { key, secret } => builder.basic_auth(key, Some(secret)), + RequestAuth::Bearer { token } => builder.bearer_auth(token), + } +} + +async fn send_json( + builder: reqwest::RequestBuilder, + auth: &RequestAuth, + body: &impl Serialize, + action: &str, + include_error_body: bool, +) -> CloudResult { + let response = authorized(builder, auth) + .json(body) + .send() + .await + .map_err(|error| CloudError::new(format!("{action}: {}", error.without_url())))?; + parse_response(response, action, include_error_body).await +} + +async fn parse_response( + response: reqwest::Response, + action: &str, + include_error_body: bool, +) -> CloudResult { + let status = response.status(); + let bytes = response + .bytes() + .await + .map_err(|error| CloudError::new(format!("{action}: {}", error.without_url())))?; + if !status.is_success() { + let message = if include_error_body { + format!( + "{action} failed ({status}): {}", + String::from_utf8_lossy(&bytes).trim() + ) + } else { + format!("{action} failed ({status})") + }; + return Err(if status.as_u16() == 401 || status.as_u16() == 403 { + CloudError::auth(message) + } else { + CloudError::new(message) + }); + } + // Deserializer diagnostics can quote a secret-bearing unexpected value. + let envelope: Envelope = serde_json::from_slice(&bytes) + .map_err(|_| CloudError::new(format!("{action}: invalid response")))?; + if envelope.error.is_some() || envelope.status.is_some_and(|status| status >= 400) { + return Err(CloudError::new(format!( + "{action}: invalid response envelope" + ))); + } + Ok(envelope.result) +} + +// ── loopback redirect server ──────────────────────────────────────────────── + +/// Accept loopback connections until one hits `/callback` with a `code` and the +/// matching `state`, then return the code. A `state` mismatch is a CSRF failure +/// and aborts; unrelated paths (e.g. `/favicon.ico`) are answered and ignored. +async fn wait_for_callback(listener: TcpListener, expected_state: &str) -> CloudResult { + loop { + let (mut stream, _) = listener + .accept() + .await + .map_err(|error| CloudError::new(format!("loopback accept failed: {error}")))?; + + let target = match read_request_target(&mut stream).await { + Some(target) => target, + None => { + write_response(&mut stream, "400 Bad Request", BAD_REQUEST_HTML).await; + continue; + } + }; + + let (path, query) = split_target(&target); + if path != "/callback" { + write_response(&mut stream, "404 Not Found", NOT_FOUND_HTML).await; + continue; + } + + let params = parse_query(query); + match (params.get("code"), params.get("state")) { + (Some(code), Some(got_state)) if got_state == expected_state => { + if !valid_claim_code(code) { + write_response(&mut stream, "400 Bad Request", BAD_REQUEST_HTML).await; + return Err(CloudError::new("invalid access request callback code")); + } + write_response(&mut stream, "200 OK", SUCCESS_HTML).await; + return Ok(code.clone()); + } + (_, Some(_)) => { + write_response(&mut stream, "400 Bad Request", BAD_REQUEST_HTML).await; + return Err(CloudError::auth( + "access request callback failed CSRF check (state mismatch)", + )); + } + _ => { + write_response(&mut stream, "400 Bad Request", BAD_REQUEST_HTML).await; + continue; + } + } + } +} + +/// The largest request line (terminator included) accepted from the loopback +/// callback before the connection is treated as malformed and ignored. +const MAX_REQUEST_LINE: usize = 8 * 1024; + +/// Read the HTTP request line and return its request target (the second token). +/// +/// A TCP read may split the request line anywhere — including midway through +/// the `state` value — so bytes are buffered through the request-line +/// terminator and only then parsed; parsing whatever a single read happened to +/// deliver would treat a partial `state` as a CSRF mismatch and abort approval. +/// An oversized line without a terminator is reported as no target. +async fn read_request_target(stream: &mut tokio::net::TcpStream) -> Option { + let mut buffer = Vec::new(); + let mut chunk = [0u8; 2048]; + loop { + let read = stream.read(&mut chunk).await.ok()?; + if read == 0 { + return None; + } + buffer.extend_from_slice(&chunk[..read]); + if let Some(line_end) = buffer.iter().position(|byte| *byte == b'\n') { + let text = String::from_utf8_lossy(&buffer[..line_end]); + let line = text.strip_suffix('\r').unwrap_or(&text); + return line.split_whitespace().nth(1).map(str::to_string); + } + if buffer.len() > MAX_REQUEST_LINE { + return None; + } + } +} + +fn split_target(target: &str) -> (&str, &str) { + match target.split_once('?') { + Some((path, query)) => (path, query), + None => (target, ""), + } +} + +fn parse_query(query: &str) -> HashMap { + query + .split('&') + .filter(|pair| !pair.is_empty()) + .filter_map(|pair| { + let (key, value) = pair.split_once('=')?; + let value = urlencoding::decode(value).ok()?.into_owned(); + Some((key.to_string(), value)) + }) + .collect() +} + +async fn write_response(stream: &mut tokio::net::TcpStream, status: &str, body: &str) { + let response = format!( + "HTTP/1.1 {status}\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", + body.len() + ); + let _ = stream.write_all(response.as_bytes()).await; + let _ = stream.flush().await; +} + +const SUCCESS_HTML: &str = "Access approved

Access approved

You can return to your terminal.

"; +const NOT_FOUND_HTML: &str = "Not found"; +const BAD_REQUEST_HTML: &str = "Invalid request. You can close this tab and return to your terminal."; + +// ── status polling fallback ────────────────────────────────────────────────── + +/// An immutable wall deadline with a monotonic ceiling: clock rollback cannot +/// extend it, while a forward adjustment may shorten it. +struct Deadline { + epoch_millis: i64, + monotonic: Instant, +} + +fn epoch_millis() -> CloudResult { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .ok() + .and_then(|duration| i64::try_from(duration.as_millis()).ok()) + .ok_or_else(|| CloudError::new("invalid system clock for access request")) +} + +impl Deadline { + fn new(value: Option, now: i64, instant: Instant) -> CloudResult { + let epoch_millis = value.filter(|value| (1..=MAX_EPOCH_MILLIS).contains(value)) + .ok_or_else(|| CloudError::new("access request has a missing or invalid deadline; server deadline support is required"))?; + let remaining = Duration::from_millis(epoch_millis.saturating_sub(now).max(0) as u64); + let monotonic = instant + .checked_add(remaining) + .ok_or_else(|| CloudError::new("access request deadline is not representable"))?; + Ok(Self { + epoch_millis, + monotonic, + }) + } + + fn remaining_at(&self, now: i64, instant: Instant) -> Duration { + Duration::from_millis(self.epoch_millis.saturating_sub(now).max(0) as u64) + .min(self.monotonic.saturating_duration_since(instant)) + } + + fn remaining(&self) -> CloudResult { + Ok(self.remaining_at(epoch_millis()?, Instant::now())) + } +} + +struct PendingApproval<'a> { + requested_role_ids: Vec, + request_id: &'a str, + deadline: Deadline, +} + +impl<'a> PendingApproval<'a> { + fn new(created: &'a CreateAccessRequestResponse) -> CloudResult { + if created.request_id.trim().is_empty() + || created.approval_url.trim().is_empty() + || created.status.as_deref() != Some("pending") + { + return Err(CloudError::new("file access request: incomplete response")); + } + let now = Instant::now(); + let mut deadline = Deadline::new(created.expires_at, epoch_millis()?, now)?; + deadline.monotonic = deadline.monotonic.min(now + APPROVAL_TIMEOUT); + Ok(Self { + requested_role_ids: canonical_role_ids(created.requested_role_ids.as_deref(), true)?, + request_id: &created.request_id, + deadline, + }) + } +} + +// Contains a secret. Do not derive Debug or include it in diagnostics. +struct ClaimPermit { + code: String, + deadline: Deadline, +} + +fn valid_claim_code(code: &str) -> bool { + (1..=512).contains(&code.len()) + && code + .bytes() + .all(|c| c.is_ascii_alphanumeric() || c == b'_' || c == b'-') +} + +fn interpret_status( + status: AccessRequestStatus, + pending: &PendingApproval<'_>, + callback_code: Option<&str>, +) -> CloudResult> { + if status.request_id.as_deref() != Some(pending.request_id) + || status.expires_at != Some(pending.deadline.epoch_millis) + { + return Err(CloudError::new( + "access request status has inconsistent identity or pending deadline", + )); + } + if canonical_role_ids(status.requested_role_ids.as_deref(), true)? != pending.requested_role_ids + { + return Err(CloudError::new( + "access request status has inconsistent saved proposal", + )); + } + match status.status.as_deref() { + Some("pending") => Ok(None), + Some("denied") => Err(CloudError::new("access request denied")), + Some("expired") => Err(CloudError::new("access request expired")), + Some("approved") => { + let deadline = Deadline::new( + status.claim_code_expires_at, + epoch_millis()?, + Instant::now(), + )?; + if let Some(key_id) = status.issued_key_id { + return Err(CloudError::new(if key_id.trim().is_empty() { + "access request has invalid issued key identity" + } else { + "access request has already been claimed" + })); + } + if deadline.remaining()?.is_zero() { + return Err(CloudError::new("access request claim window expired")); + } + let code = status + .claim_code + .filter(|code| valid_claim_code(code)) + .ok_or_else(|| { + CloudError::new("approved access request has no usable claim code") + })?; + if callback_code.is_some_and(|callback| callback != code) { + return Err(CloudError::new( + "access request callback does not match authoritative status", + )); + } + Ok(Some(ClaimPermit { code, deadline })) + } + _ => Err(CloudError::new("invalid access request status")), + } +} + +enum OperationError { + Timeout, + Failed(CloudError), +} + +impl OperationError { + fn into_cloud(self, timeout_message: &str) -> CloudError { + match self { + Self::Timeout => CloudError::new(timeout_message), + Self::Failed(error) => error, + } + } +} + +/// Covers send, body and decoding. The elapsed check also catches synchronous +/// decoding that finishes after the timeout without yielding to Tokio. +async fn operation( + budget: Duration, + future: impl std::future::Future>, +) -> Result { + if budget.is_zero() { + return Err(OperationError::Timeout); + } + let start = Instant::now(); + match tokio::time::timeout(budget, future).await { + Ok(result) if start.elapsed() < budget => result.map_err(OperationError::Failed), + _ => Err(OperationError::Timeout), + } +} + +async fn fetch_status( + http: &reqwest::Client, + status_url: &str, + auth: &RequestAuth, +) -> CloudResult> { + let response = authorized(http.get(status_url), auth) + .send() + .await + .map_err(|error| { + CloudError::new(format!("poll access request: {}", error.without_url())) + })?; + if response.status().is_server_error() + || response.status() == reqwest::StatusCode::TOO_MANY_REQUESTS + { + return Ok(None); + } + parse_response(response, "poll access request", false) + .await + .map(Some) +} + +async fn wait_for_approval( + http: &reqwest::Client, + status_url: &str, + auth: &RequestAuth, + pending: PendingApproval<'_>, + listener: TcpListener, + state: &str, +) -> CloudResult { + let callback = wait_for_callback(listener, state); + tokio::pin!(callback); + let mut callback_code = None; + loop { + let remaining = pending.deadline.remaining()?; + if !remaining.is_zero() { + tokio::select! { + result = &mut callback, if callback_code.is_none() => callback_code = Some(result?), + _ = tokio::time::sleep(POLL_INTERVAL.min(remaining)) => {}, + } + } + let remaining = pending.deadline.remaining()?; + let final_observation = remaining.is_zero(); + let budget = if final_observation { + OPERATION_TIMEOUT + } else { + OPERATION_TIMEOUT.min(remaining) + }; + let fetch = operation(budget, fetch_status(http, status_url, auth)); + tokio::pin!(fetch); + let fetched = tokio::select! { + biased; + result = &mut callback, if callback_code.is_none() => { + callback_code = Some(result?); + fetch.await + } + result = &mut fetch => result, + }; + let status = match fetched { + Ok(status) => status, + Err(OperationError::Timeout) + if !final_observation && pending.deadline.remaining()?.is_zero() => + { + continue; + } + Err(error) => return Err(error.into_cloud("poll access request timed out")), + }; + if let Some(status) = status + && let Some(permit) = interpret_status(status, &pending, callback_code.as_deref())? + { + return Ok(permit); + } + if final_observation { + return Err(CloudError::new( + "access request approval wait ended without a usable approval", + )); + } + } +} + +async fn claim_access_key( + http: &reqwest::Client, + claim_url: &str, + auth: &RequestAuth, + permit: &ClaimPermit, + verifier: &str, +) -> CloudResult { + let remaining = permit.deadline.remaining()?; + if remaining.is_zero() { + return Err(CloudError::new("access request claim window expired")); + } + operation(OPERATION_TIMEOUT.min(remaining), send_json( + http.post(claim_url), auth, + &ClaimBody { claim_code: &permit.code, code_verifier: verifier }, + "claim access key", false, + )).await.map_err(|error| error.into_cloud("claim access key timed out; the claim may have completed on the server; no credentials were installed")) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::cli::{Cli, Commands}; + use clap::Parser; + + fn created(expires_at: i64) -> CreateAccessRequestResponse { + CreateAccessRequestResponse { + requested_role_ids: Some(vec![]), + request_id: "request-1".into(), + status: Some("pending".into()), + approval_url: "https://example.invalid/approve".into(), + expires_at: Some(expires_at), + } + } + + fn status_body(status: &str, expires_at: i64) -> serde_json::Value { + serde_json::json!({"result": {"requestedRoleIds":[], "requestId":"request-1", "expiresAt":expires_at, "status":status, "claimCode":"test-code", "claimCodeExpiresAt":4102444800000_i64}}) + } + + async fn poll_until_approved( + http: &reqwest::Client, + url: &str, + auth: &RequestAuth, + ) -> CloudResult { + let created = created(4102444800000); + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + wait_for_approval( + http, + url, + auth, + PendingApproval::new(&created)?, + listener, + "state", + ) + .await + .map(|permit| permit.code) + } + + #[test] + fn canonical_role_ids_validate_server_arrays_without_inferring_grants() { + let system = "21111111-1111-1111-9999-000000000007".to_string(); + let upper = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA".to_string(); + assert_eq!( + canonical_role_ids(Some(&[system.clone(), upper.clone()]), true).unwrap(), + vec![system, upper.to_lowercase()] + ); + assert_eq!( + canonical_role_ids(Some(&[]), true).unwrap(), + Vec::::new() + ); + assert!(canonical_role_ids(Some(&[]), false).is_err()); + for proposal in [true, false] { + assert!(canonical_role_ids(None, proposal).is_err()); + assert!( + canonical_role_ids(Some(&[upper.clone(), upper.to_lowercase()]), proposal).is_err() + ); + for invalid in [ + "", + "role-name", + "*", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "{aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa}", + "urn:uuid:aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", + " aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", + ] { + let error = canonical_role_ids(Some(&[invalid.into()]), proposal).unwrap_err(); + assert!(!error.message.contains("aaaaaaaa")); + } + } + let many: Vec<_> = (1..=101) + .map(|id| format!("00000000-0000-0000-0000-{id:012x}")) + .collect(); + assert!(canonical_role_ids(Some(&many[..100]), true).is_ok()); + assert!(canonical_role_ids(Some(&many), true).is_err()); + assert_eq!(canonical_role_ids(Some(&many), false).unwrap().len(), 101); + } + + #[test] + fn role_response_fields_reject_malformed_types_and_status_proposal_changes() { + for value in [ + serde_json::json!(42), + serde_json::json!("secret-value"), + serde_json::json!([null]), + serde_json::json!([1]), + serde_json::json!({}), + ] { + let create = serde_json::json!({"requestId":"request-1", "status":"pending", "approvalUrl":"https://example.invalid", "expiresAt":4102444800000_i64, "requestedRoleIds":value}); + assert!(serde_json::from_value::(create).is_err()); + assert!( + serde_json::from_value::( + serde_json::json!({"requestedRoleIds":value}) + ) + .is_err() + ); + let claim = serde_json::json!({"keyId":"key-id", "key":"key", "secret":"secret", "grantedRoleIds":value}); + assert!(serde_json::from_value::(claim).is_err()); + } + let a = "21111111-1111-1111-9999-000000000007".to_string(); + let b = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa".to_string(); + let mut created = created(4102444800000); + created.requested_role_ids = Some(vec![a.clone(), b.to_uppercase()]); + let pending = PendingApproval::new(&created).unwrap(); + let mut status = status_body("approved", 4102444800000); + for ids in [ + serde_json::json!(null), + serde_json::json!([]), + serde_json::json!([b, a]), + serde_json::json!([b, b.to_uppercase()]), + ] { + status["result"]["requestedRoleIds"] = ids; + assert!( + interpret_status( + serde_json::from_value(status["result"].clone()).unwrap(), + &pending, + None + ) + .is_err() + ); + } + status["result"]["requestedRoleIds"] = serde_json::json!([a, b]); + assert!( + interpret_status( + serde_json::from_value(status["result"].clone()).unwrap(), + &pending, + None + ) + .unwrap() + .is_some() + ); + } + + #[test] + fn deadline_boundaries_clock_changes_and_epoch_domain() { + let instant = Instant::now(); + for (epoch, remaining) in [(999, 0), (1000, 0), (1001, 1)] { + let deadline = Deadline::new(Some(epoch), 1000, instant).unwrap(); + assert_eq!( + deadline.remaining_at(1000, instant), + Duration::from_millis(remaining) + ); + } + let deadline = Deadline::new(Some(2000), 1000, instant).unwrap(); + assert_eq!( + deadline.remaining_at(500, instant + Duration::from_millis(600)), + Duration::from_millis(400) + ); + assert_eq!( + deadline.remaining_at(1900, instant), + Duration::from_millis(100) + ); + for value in [ + None, + Some(-1), + Some(0), + Some(MAX_EPOCH_MILLIS + 1), + Some(i64::MAX), + ] { + assert!(Deadline::new(value, 1000, instant).is_err()); + } + // Platforms may reject a huge Instant, but must never panic or clamp. + if let Ok(deadline) = Deadline::new(Some(MAX_EPOCH_MILLIS), 1000, instant) { + assert_eq!(deadline.epoch_millis, MAX_EPOCH_MILLIS); + } + for value in [ + serde_json::json!(null), + serde_json::json!(1.5), + serde_json::json!("secret"), + serde_json::json!(18446744073709551615_u64), + ] { + let wire = serde_json::json!({"requestId":"request-1", "status":"pending", "approvalUrl":"https://example.invalid", "expiresAt":value}); + if let Ok(created) = serde_json::from_value::(wire) { + assert!(PendingApproval::new(&created).is_err()); + } + } + let future = created(4102444800000); + assert!( + PendingApproval::new(&future) + .unwrap() + .deadline + .remaining() + .unwrap() + <= APPROVAL_TIMEOUT + ); + } + + #[test] + fn interpret_status_requires_independent_claim_deadline_and_valid_code() { + let created = created(1); // pending expiry cannot cut off an approved claim + let pending = PendingApproval::new(&created).unwrap(); + let body = status_body("approved", 1); + let parse = |body: serde_json::Value| { + serde_json::from_value::(body["result"].clone()).unwrap() + }; + assert!( + interpret_status(parse(body.clone()), &pending, Some("test-code")) + .unwrap() + .is_some() + ); + for (field, value) in [ + ("claimCodeExpiresAt", serde_json::json!(null)), + ("claimCodeExpiresAt", serde_json::json!(1)), + ("claimCode", serde_json::json!(null)), + ("claimCode", serde_json::json!(" ")), + ("claimCode", serde_json::json!("a".repeat(513))), + ("issuedKeyId", serde_json::json!("issued")), + ("issuedKeyId", serde_json::json!("")), + ("requestId", serde_json::json!("other")), + ("expiresAt", serde_json::json!(2)), + ] { + let mut invalid = body.clone(); + invalid["result"][field] = value; + assert!( + interpret_status(parse(invalid), &pending, None).is_err(), + "{field}" + ); + } + assert!(interpret_status(parse(body), &pending, Some("different-secret")).is_err()); + for code in ["a", "_-09AZaz"] { + assert!(valid_claim_code(code)); + } + for code in ["", "a=", " a", "a\n", "é"] { + assert!(!valid_claim_code(code)); + } + assert!(valid_claim_code(&"a".repeat(512))); + } + + #[tokio::test] + async fn wait_for_approval_final_observation_is_once_and_can_claim_after_pending_expiry() { + use wiremock::{Mock, MockServer, ResponseTemplate}; + for (http_status, status, success) in [ + (200, "approved", true), + (200, "pending", false), + (429, "pending", false), + (503, "pending", false), + ] { + let server = MockServer::start().await; + Mock::given(wiremock::matchers::method("GET")) + .respond_with( + ResponseTemplate::new(http_status).set_body_json(status_body(status, 1)), + ) + .expect(1) + .mount(&server) + .await; + let created = created(1); + let result = wait_for_approval( + &reqwest::Client::new(), + &server.uri(), + &RequestAuth::Bearer { + token: "fake".into(), + }, + PendingApproval::new(&created).unwrap(), + TcpListener::bind("127.0.0.1:0").await.unwrap(), + "state", + ) + .await; + assert_eq!(result.is_ok(), success); + } + } + + #[tokio::test] + async fn operation_timeout_covers_body_and_claim_is_not_retried() { + use wiremock::{Mock, MockServer, ResponseTemplate}; + let server = MockServer::start().await; + Mock::given(wiremock::matchers::method("POST")) + .respond_with(ResponseTemplate::new(200).set_delay(Duration::from_secs(1))) + .expect(1) + .mount(&server) + .await; + let permit = ClaimPermit { + code: "secret-code".into(), + deadline: Deadline::new( + Some(epoch_millis().unwrap() + 50), + epoch_millis().unwrap(), + Instant::now(), + ) + .unwrap(), + }; + let error = claim_access_key( + &reqwest::Client::new(), + &server.uri(), + &RequestAuth::Bearer { + token: "fake".into(), + }, + &permit, + "secret-verifier", + ) + .await + .err() + .unwrap(); + assert!(error.message.contains("may have completed")); + assert!(!error.message.contains("secret")); + let result = operation::<()>(Duration::from_millis(5), async { + tokio::time::sleep(Duration::from_secs(1)).await; + Ok(()) + }) + .await; + assert!(matches!(result, Err(OperationError::Timeout))); + } + + #[tokio::test] + async fn wait_for_approval_callback_is_only_a_signal_and_idle_socket_is_bounded() { + use std::sync::{ + Arc, + atomic::{AtomicUsize, Ordering}, + }; + use wiremock::{Mock, MockServer, ResponseTemplate}; + for callback in [true, false] { + let server = MockServer::start().await; + let attempts = Arc::new(AtomicUsize::new(0)); + let count = attempts.clone(); + Mock::given(wiremock::matchers::method("GET")) + .respond_with(move |_: &wiremock::Request| { + let status = if callback && count.fetch_add(1, Ordering::SeqCst) == 0 { + "pending" + } else { + "approved" + }; + ResponseTemplate::new(200).set_body_json(status_body(status, 4102444800000)) + }) + .expect(if callback { 2 } else { 1 }) + .mount(&server) + .await; + let created = created(4102444800000); + let mut pending = PendingApproval::new(&created).unwrap(); + // Exercise the local wait cap without waiting fifteen minutes. + pending.deadline.monotonic = Instant::now() + Duration::from_millis(100); + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + let client = reqwest::Client::new(); + let auth = RequestAuth::Bearer { + token: "fake".into(), + }; + let url = server.uri(); + let wait = wait_for_approval(&client, &url, &auth, pending, listener, "state"); + let notify = async { + let mut stream = tokio::net::TcpStream::connect(address).await.unwrap(); + if callback { + stream + .write_all(b"GET /callback?code=test-code&state=state HTTP/1.1\r\n\r\n") + .await + .unwrap(); + let mut response = Vec::new(); + stream.read_to_end(&mut response).await.unwrap(); + assert!(String::from_utf8_lossy(&response).starts_with("HTTP/1.1 200")); + } else { + // No request line: the outer pending clock must cancel this read. + let mut response = Vec::new(); + stream.read_to_end(&mut response).await.unwrap(); + } + }; + let (permit, ()) = + tokio::time::timeout(Duration::from_secs(2), async { tokio::join!(wait, notify) }) + .await + .unwrap(); + assert_eq!(permit.unwrap().code, "test-code"); + } + } + + #[tokio::test] + async fn wait_for_callback_reassembles_a_request_line_split_across_writes() { + use wiremock::{Mock, MockServer, ResponseTemplate}; + let server = MockServer::start().await; + Mock::given(wiremock::matchers::method("GET")) + .respond_with( + ResponseTemplate::new(200).set_body_json(status_body("approved", 4102444800000)), + ) + .expect(1) + .mount(&server) + .await; + let created = created(4102444800000); + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + let client = reqwest::Client::new(); + let auth = RequestAuth::Bearer { + token: "fake".into(), + }; + let url = server.uri(); + let wait = wait_for_approval( + &client, + &url, + &auth, + PendingApproval::new(&created).unwrap(), + listener, + "state", + ); + let notify = async { + let mut stream = tokio::net::TcpStream::connect(address).await.unwrap(); + stream + .write_all(b"GET /callback?code=test-code&sta") + .await + .unwrap(); + // Keep the halves in separate reads; a read need not cover the + // whole line, and this split lands midway through the state value. + tokio::time::sleep(Duration::from_millis(50)).await; + stream + .write_all(b"te=state HTTP/1.1\r\n\r\n") + .await + .unwrap(); + let mut response = Vec::new(); + stream.read_to_end(&mut response).await.unwrap(); + assert!(String::from_utf8_lossy(&response).starts_with("HTTP/1.1 200")); + }; + let (permit, ()) = + tokio::time::timeout(Duration::from_secs(2), async { tokio::join!(wait, notify) }) + .await + .unwrap(); + assert_eq!(permit.unwrap().code, "test-code"); + } + + #[tokio::test] + async fn read_request_target_skips_an_oversized_line_without_a_terminator() { + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + let mut client = tokio::net::TcpStream::connect(address).await.unwrap(); + let (mut server, _) = listener.accept().await.unwrap(); + let oversized = "GET /".to_string() + &"a".repeat(MAX_REQUEST_LINE + 1); + let writer = tokio::spawn(async move { + // The reader may stop before the tail lands; delivery is best-effort. + let _ = client.write_all(oversized.as_bytes()).await; + }); + let target = tokio::time::timeout(Duration::from_secs(2), read_request_target(&mut server)) + .await + .unwrap(); + assert_eq!(target, None); + writer.await.unwrap(); + } + + #[tokio::test] + async fn handshake_client_never_follows_redirects() { + use wiremock::matchers::{method, path}; + use wiremock::{Mock, MockServer, ResponseTemplate}; + for status in [301, 302, 303, 307, 308] { + let server = MockServer::start().await; + let destination = MockServer::start().await; + Mock::given(method("POST")) + .and(path("/v1/organizations/org/accessRequests/request-1/claim")) + .respond_with( + ResponseTemplate::new(status) + .insert_header("Location", format!("{}/stolen", destination.uri())), + ) + .expect(1) + .mount(&server) + .await; + let claimed = claim_access_key( + &handshake_client().unwrap(), + &format!( + "{}/v1/organizations/org/accessRequests/request-1/claim", + server.uri() + ), + &RequestAuth::Bearer { + token: "fake".into(), + }, + &ClaimPermit { + code: "claim-code-secret".into(), + deadline: Deadline::new( + Some(epoch_millis().unwrap() + 50), + epoch_millis().unwrap(), + Instant::now(), + ) + .unwrap(), + }, + "secret-verifier", + ) + .await; + let error = claimed.err().unwrap(); + assert!( + !error.message.contains("secret"), + "redirect handling must not leak the verifier: {error}" + ); + assert!(destination.received_requests().await.unwrap().is_empty()); + } + } + + #[tokio::test] + async fn wait_for_approval_pending_transport_crosses_boundary_once() { + use std::sync::{ + Arc, + atomic::{AtomicUsize, Ordering}, + }; + use wiremock::{Mock, MockServer, ResponseTemplate}; + let server = MockServer::start().await; + let count = Arc::new(AtomicUsize::new(0)); + let attempts = count.clone(); + Mock::given(wiremock::matchers::method("GET")) + .respond_with(move |_: &wiremock::Request| { + let response = ResponseTemplate::new(200) + .set_body_json(status_body("approved", 4102444800000)); + if attempts.fetch_add(1, Ordering::SeqCst) == 0 { + response.set_delay(Duration::from_secs(1)) + } else { + response + } + }) + .expect(2) + .mount(&server) + .await; + let created = created(4102444800000); + let mut pending = PendingApproval::new(&created).unwrap(); + pending.deadline.monotonic = Instant::now() + Duration::from_millis(100); + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + let client = reqwest::Client::new(); + let auth = RequestAuth::Bearer { + token: "fake".into(), + }; + let url = server.uri(); + let wait = wait_for_approval(&client, &url, &auth, pending, listener, "state"); + let notify = async { + client + .get(format!( + "http://{address}/callback?code=test-code&state=state" + )) + .send() + .await + .unwrap(); + }; + let (permit, ()) = + tokio::time::timeout(Duration::from_secs(2), async { tokio::join!(wait, notify) }) + .await + .unwrap(); + assert_eq!(permit.unwrap().code, "test-code"); + assert_eq!(count.load(Ordering::SeqCst), 2); + } + + #[tokio::test] + async fn operation_bounds_response_body_read_and_sanitizes_envelope() { + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + let server = tokio::spawn(async move { + let (mut stream, _) = listener.accept().await.unwrap(); + let mut request = [0; 4096]; + assert!(stream.read(&mut request).await.unwrap() > 0); + stream + .write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 1000\r\n\r\n{") + .await + .unwrap(); + std::future::pending::<()>().await; + }); + let client = reqwest::Client::new(); + let result = operation(Duration::from_millis(50), async { + let response = client + .get(format!("http://{address}")) + .send() + .await + .unwrap(); + parse_response::(response, "poll access request", false).await + }) + .await; + server.abort(); + assert!(matches!(result, Err(OperationError::Timeout))); + + use wiremock::{Mock, MockServer, ResponseTemplate}; + for field in ["error", "status"] { + let server = MockServer::start().await; + let mut body = status_body("approved", 1); + body[field] = if field == "error" { + serde_json::json!("secret-echo") + } else { + serde_json::json!(500) + }; + Mock::given(wiremock::matchers::method("GET")) + .respond_with(ResponseTemplate::new(200).set_body_json(body)) + .mount(&server) + .await; + let error = fetch_status( + &client, + &server.uri(), + &RequestAuth::Bearer { + token: "fake".into(), + }, + ) + .await + .err() + .unwrap(); + assert!(!error.message.contains("secret-echo")); + assert!(error.message.contains("envelope")); + } + } + + #[test] + fn approval_handoff_no_browser_skips_launcher_and_default_launches_once() { + approval_handoff("https://example.invalid/approve", true, |_| { + panic!("must not launch") + }); + let mut calls = 0; + approval_handoff("https://example.invalid/approve", false, |url| { + assert_eq!(url, "https://example.invalid/approve"); + calls += 1; + Ok(()) + }); + assert_eq!(calls, 1); + // Launcher failure still permits manual approval and polling. + approval_handoff("https://example.invalid/approve", false, |_| { + Err(std::io::Error::other("fake launcher failure")) + }); + assert!(!parse(&["chctl", "cloud", "access", "request"]).no_browser); + assert!(parse(&["chctl", "cloud", "access", "request", "--no-browser"]).no_browser); + assert!( + Cli::try_parse_from(["chctl", "cloud", "access", "roles", "--no-browser"]).is_err() + ); + } + + #[tokio::test] + async fn poll_until_approved_propagates_terminal_and_invalid_responses() { + use wiremock::{Mock, MockServer, ResponseTemplate}; + let mut tasks = Vec::new(); + for (http_status, mut body, auth_error) in [ + (401, serde_json::json!({"error":"echo-claim-code"}), true), + (403, serde_json::json!({"error":"echo-claim-code"}), true), + (400, serde_json::json!({"error":"echo-claim-code"}), false), + (404, serde_json::json!({"error":"echo-claim-code"}), false), + ( + 200, + serde_json::json!({"result":{"status":"denied"}}), + false, + ), + ( + 200, + serde_json::json!({"result":{"status":"expired"}}), + false, + ), + ( + 200, + serde_json::json!({"result":{"status":"future"}}), + false, + ), + (200, serde_json::json!({"result":{}}), false), + ( + 200, + serde_json::json!({"result":{"status":"approved"}}), + false, + ), + ( + 200, + serde_json::json!({"result":{"status":"approved","claimCode":""}}), + false, + ), + ( + 200, + serde_json::json!({"result":{"status":"approved","claimCode":{"echo-claim-code":1}}}), + false, + ), + ] { + if let Some(result) = body + .get_mut("result") + .and_then(serde_json::Value::as_object_mut) + { + result.insert("requestedRoleIds".into(), serde_json::json!([])); + result.insert("requestId".into(), serde_json::json!("request-1")); + result.insert("expiresAt".into(), serde_json::json!(4102444800000_i64)); + result.insert( + "claimCodeExpiresAt".into(), + serde_json::json!(4102444800000_i64), + ); + } + tasks.push(tokio::spawn(async move { + let server = MockServer::start().await; + Mock::given(wiremock::matchers::method("GET")) + .respond_with(ResponseTemplate::new(http_status).set_body_json(body)) + .expect(1) + .mount(&server) + .await; + let result = tokio::time::timeout( + Duration::from_secs(8), + poll_until_approved( + &reqwest::Client::new(), + &server.uri(), + &RequestAuth::Bearer { + token: "fake-token".into(), + }, + ), + ) + .await + .expect("terminal response must not wait for approval timeout") + .unwrap_err(); + assert_eq!( + matches!(result.kind, crate::cloud::client::CloudErrorKind::Auth), + auth_error + ); + assert!(!result.message.contains("echo-claim-code")); + })); + } + for task in tasks { + task.await.unwrap(); + } + } + + #[tokio::test] + async fn poll_until_approved_retries_only_transient_http_and_pending() { + use std::sync::{ + Arc, + atomic::{AtomicUsize, Ordering}, + }; + use wiremock::{Mock, MockServer, ResponseTemplate}; + let server = MockServer::start().await; + let count = Arc::new(AtomicUsize::new(0)); + let attempts = count.clone(); + Mock::given(wiremock::matchers::method("GET")) + .respond_with(move |_: &wiremock::Request| { + match attempts.fetch_add(1, Ordering::SeqCst) { + 0 => ResponseTemplate::new(429), + 1 => ResponseTemplate::new(503), + 2 => ResponseTemplate::new(200) + .set_body_json(status_body("pending", 4102444800000)), + _ => ResponseTemplate::new(200) + .set_body_json(status_body("approved", 4102444800000)), + } + }) + .expect(4) + .mount(&server) + .await; + let code = tokio::time::timeout( + Duration::from_secs(18), + poll_until_approved( + &reqwest::Client::new(), + &server.uri(), + &RequestAuth::Bearer { + token: "fake-token".into(), + }, + ), + ) + .await + .unwrap() + .unwrap(); + assert_eq!(code, "test-code"); + assert_eq!(count.load(Ordering::SeqCst), 4); + } + + #[tokio::test] + async fn claim_response_errors_do_not_echo_secret_values() { + use wiremock::{Mock, MockServer, ResponseTemplate}; + for status in [200, 400, 401, 500] { + let server = MockServer::start().await; + Mock::given(wiremock::matchers::method("POST")) + .respond_with( + ResponseTemplate::new(status).set_body_json(serde_json::json!({ + "result": "secret-verifier-code", "error": "secret-verifier-code" + })), + ) + .mount(&server) + .await; + let error = send_json::( + reqwest::Client::new().post(server.uri()), + &RequestAuth::Bearer { + token: "fake-token".into(), + }, + &serde_json::json!({}), + "claim access key", + false, + ) + .await + .err() + .unwrap(); + assert!(!error.message.contains("secret-verifier-code")); + } + } + + fn parse(args: &[&str]) -> AccessRequestArgs { + let cli = Cli::try_parse_from(args).expect("parse"); + let Commands::Cloud(cloud) = cli.command else { + panic!("expected cloud command"); + }; + let crate::cloud::cli::CloudCommands::Access { command, .. } = cloud.command else { + panic!("expected access command"); + }; + let AccessCommands::Request(args) = command else { + panic!("expected request") + }; + args + } + + #[test] + fn request_perms_parse_deduplicates_uuid_identity_and_preserves_order() { + let first = "abcdefab-1234-1234-1234-123456789abc"; + let system = "21111111-1111-1111-9999-000000000007"; + let args = parse(&[ + "chctl", + "cloud", + "access", + "request", + "--perms", + &format!("role:{first},role:{system}"), + "--perms", + &format!("role:{}", first.to_uppercase()), + ]); + assert_eq!( + args.requested_role_ids().unwrap(), + Some(vec![first.into(), system.into()]) + ); + assert_eq!( + parse(&["chctl", "cloud", "access", "request"]) + .requested_role_ids() + .unwrap(), + None + ); + let ids: Vec<_> = (0..100) + .map(|i| format!("role:00000000-0000-0000-0000-{i:012x}")) + .collect(); + let repeated = format!("{},{}", ids.join(","), ids.join(",")); + assert_eq!( + parse(&["chctl", "cloud", "access", "request", "--perms", &repeated]) + .requested_role_ids() + .unwrap() + .unwrap() + .len(), + 100 + ); + let oversized = format!( + "{},role:00000000-0000-0000-0000-000000000100", + ids.join(",") + ); + assert!( + parse(&["chctl", "cloud", "access", "request", "--perms", &oversized]) + .requested_role_ids() + .is_err() + ); + } + + #[test] + fn request_perms_rejects_empty_malformed_and_unsupported_tokens() { + let valid = "role:00000000-0000-0000-0000-000000000001"; + for value in [ + "".to_string(), + ",".into(), + format!("{valid},"), + format!(",{valid}"), + format!("{valid},,{valid}"), + format!(" {valid}"), + format!("{valid} "), + format!("{valid}\n"), + format!("{valid}, {valid}"), + "role:00000000-0000-0000-0000-00000000000 1".into(), + "role:00000000000000000000000000000001".into(), + "role:{00000000-0000-0000-0000-000000000001}".into(), + "role:urn:uuid:00000000-0000-0000-0000-000000000001".into(), + "Role:00000000-0000-0000-0000-000000000001".into(), + "00000000-0000-0000-0000-000000000001".into(), + "role:Admin".into(), + "control-plane:service:view".into(), + "role:*".into(), + ] { + assert!( + Cli::try_parse_from(["chctl", "cloud", "access", "request", "--perms", &value]) + .is_err(), + "{value:?}" + ); + } + assert!( + Cli::try_parse_from(["chctl", "cloud", "access", "roles", "--perms", valid]).is_err() + ); + } + + #[test] + fn roles_selectors_and_help_preserve_command_grammar() { + use clap::{CommandFactory, FromArgMatches}; + for (selectors, conflict) in [ + (vec!["--org-id=one", "access", "roles", "--org=one"], false), + (vec!["--org-id=one", "access", "roles", "--org=two"], true), + ( + vec!["--org-name=name", "access", "roles", "--org-name=name"], + false, + ), + (vec!["--org-name=name", "access", "roles", "--org=id"], true), + (vec!["access", "--org=one", "roles", "--org=two"], true), + ( + vec![ + "--json", + "access", + "roles", + "--url=http://localhost:1/v1", + "--", + ], + false, + ), + ] { + let argv: Vec = ["chctl", "cloud"] + .into_iter() + .chain(selectors) + .map(Into::into) + .collect(); + let mut cmd = Cli::command(); + let matches = cmd.try_get_matches_from_mut(&argv).unwrap(); + let cli = Cli::from_arg_matches(&matches).unwrap(); + let Commands::Cloud(args) = cli.command else { + panic!("cloud expected") + }; + let result = args.validate_access_selectors(&argv); + if conflict { + assert_eq!( + result.unwrap_err().kind(), + clap::error::ErrorKind::ArgumentConflict + ); + } else { + result.unwrap(); + } + assert!(!args.command.is_write_command()); + } + for args in [ + vec!["chctl", "cloud", "access", "--help"], + vec!["chctl", "cloud", "access", "roles", "--help"], + ] { + assert_eq!( + Cli::try_parse_from(args).err().unwrap().kind(), + clap::error::ErrorKind::DisplayHelp + ); + } + assert!(Cli::try_parse_from(["chctl", "cloud", "org", "list", "--org=id"]).is_err()); + assert!( + Cli::try_parse_from(["chctl", "cloud", "access", "roles", "--purpose=test"]).is_err() + ); + } + + #[test] + fn access_organization_selectors_validate_original_command_depths() { + use clap::{CommandFactory, FromArgMatches}; + fn validate(args: &[&str]) -> Result<(), clap::Error> { + let argv: Vec = std::iter::once("chctl") + .chain(args.iter().copied()) + .map(Into::into) + .collect(); + let mut command = Cli::command(); + let matches = command.try_get_matches_from_mut(&argv)?; + let cli = Cli::from_arg_matches(&matches)?; + let Commands::Cloud(args) = cli.command else { + panic!("cloud expected") + }; + args.validate_access_selectors(&argv) + } + for args in [ + vec!["cloud", "access", "request", "--org", "id"], + vec!["cloud", "access", "request", "--org=id", "--org-id=id"], + vec!["cloud", "access", "--org=id", "request", "--org=id"], + vec!["cloud", "access", "--org=id", "request"], + vec!["cloud", "--org-id=id", "access", "request", "--org=id"], + vec!["cloud", "--org-id=id", "access", "request", "--org-id=id"], + vec![ + "cloud", + "--org-name=name", + "access", + "request", + "--org-name=name", + ], + vec![ + "cloud", + "access", + "request", + "--purpose=--org=other", + "--org=id", + ], + vec![ + "cloud", + "access", + "request", + "--purpose", + "text --org=other", + "--org=id", + ], + vec!["cloud", "access", "request", "--org=id", "--"], + vec![ + "cloud", + "--json", + "--url=http://localhost:1/v1", + "--org-id=id", + "access", + "request", + "--org=id", + ], + vec![ + "cloud", + "access", + "request", + "--json", + "--url=http://localhost:1/v1", + "--org=id", + ], + vec!["cloud", "--org-id=first", "org", "list", "--org-id=second"], + ] { + validate(&args).unwrap_or_else(|error| panic!("{args:?}: {error}")); + } + for args in [ + vec![ + "cloud", + "access", + "request", + "--org=first", + "--org-id=second", + ], + vec!["cloud", "access", "request", "--org=first", "--org=first"], + vec![ + "cloud", + "--org-id=first", + "access", + "request", + "--org-id=second", + ], + vec![ + "cloud", "access", "--org", "first", "request", "--org", "second", + ], + vec![ + "cloud", + "--org-id=first", + "access", + "request", + "--org=second", + ], + vec![ + "cloud", + "access", + "--org=first", + "request", + "--org-id=second", + ], + vec![ + "cloud", + "--org-name=first", + "access", + "request", + "--org-name=second", + ], + vec![ + "cloud", + "--org-name=same", + "access", + "request", + "--org=same", + ], + vec![ + "cloud", + "--org-id=same", + "access", + "request", + "--org-name=same", + ], + vec![ + "cloud", + "access", + "--org=same", + "request", + "--org-name=same", + ], + ] { + let error = validate(&args).expect_err("conflict"); + assert_eq!( + error.kind(), + clap::error::ErrorKind::ArgumentConflict, + "{args:?}: {error}" + ); + assert_eq!(error.exit_code(), 2); + } + for args in [ + vec!["cloud", "access", "--help"], + vec!["cloud", "access", "request", "--help"], + ] { + assert_eq!( + validate(&args).unwrap_err().kind(), + clap::error::ErrorKind::DisplayHelp + ); + } + assert!(validate(&["cloud", "access", "request", "--", "--org=id"]).is_err()); + assert!(validate(&["cloud", "org", "list", "--org=id"]).is_err()); + } + + #[test] + fn pkce_challenge_is_s256_of_verifier() { + let pkce = generate_pkce(); + let expected = URL_SAFE_NO_PAD.encode(Sha256::digest(pkce.verifier.as_bytes())); + assert_eq!(pkce.challenge, expected); + // 32 random bytes base64url-encode (no pad) to 43 chars. + assert_eq!(pkce.verifier.len(), 43); + assert!(!pkce.verifier.contains('=')); + assert!(!pkce.verifier.contains('+')); + assert!(!pkce.verifier.contains('/')); + } + + #[test] + fn pkce_and_state_are_unique_per_call() { + assert_ne!(generate_pkce().verifier, generate_pkce().verifier); + assert_ne!(generate_state(), generate_state()); + } + + #[test] + fn split_target_separates_path_and_query() { + assert_eq!( + split_target("/callback?code=a&state=b"), + ("/callback", "code=a&state=b") + ); + assert_eq!(split_target("/callback"), ("/callback", "")); + } + + #[test] + fn parse_query_url_decodes_values() { + let params = parse_query("code=ab%20cd&state=xyz"); + assert_eq!(params.get("code").map(String::as_str), Some("ab cd")); + assert_eq!(params.get("state").map(String::as_str), Some("xyz")); + assert!(parse_query("").is_empty()); + } + + #[test] + fn agent_context_is_absent_when_no_fields_set() { + let args = parse(["chctl", "cloud", "access", "request"].as_slice()); + assert!(agent_context(&args).is_none()); + } + + #[test] + fn create_body_serializes_camel_case_with_agent() { + let args = parse( + [ + "chctl", + "cloud", + "access", + "request", + "--key-name", + "agent-key", + "--purpose", + "ci", + "--tool", + "claude", + "--version", + "1.2", + "--session-id", + "sess-1", + ] + .as_slice(), + ); + let agent = agent_context(&args).expect("agent present"); + let body = CreateAccessRequestBody { + requested_role_ids: None, + key_name: args.key_name.as_deref(), + agent: Some(agent), + code_challenge: "challenge", + code_challenge_method: "S256", + state: "state", + redirect_uri: "http://127.0.0.1:1234/callback", + }; + let value = serde_json::to_value(&body).unwrap(); + assert_eq!(value["keyName"], "agent-key"); + assert_eq!(value["codeChallenge"], "challenge"); + assert_eq!(value["codeChallengeMethod"], "S256"); + assert_eq!(value["state"], "state"); + assert_eq!(value["redirectUri"], "http://127.0.0.1:1234/callback"); + assert_eq!(value["agent"]["purpose"], "ci"); + assert_eq!(value["agent"]["tool"], "claude"); + assert_eq!(value["agent"]["version"], "1.2"); + assert_eq!(value["agent"]["sessionId"], "sess-1"); + } + + #[test] + fn request_is_classified_read_only() { + // OAuth (read-only) identity is the intended credential for filing. + assert!( + !AccessCommands::Request(parse(["chctl", "cloud", "access", "request"].as_slice())) + .is_write() + ); + } + + fn unwrap_result(body: &str) -> T { + serde_json::from_str::>(body) + .expect("deserialize envelope") + .result + } + + #[test] + fn file_response_reads_payload_from_result_envelope() { + // `/v1` responses wrap the payload: { "result": , "requestId": ... }. + let created: CreateAccessRequestResponse = unwrap_result( + r#"{ + "result": { + "requestId": "req-1", + "status": "pending", + "approvalUrl": "https://console.example.com/approve/req-1", + "expiresAt": 1739000000000 + }, + "requestId": "corr-1" + }"#, + ); + assert_eq!(created.request_id, "req-1"); + assert_eq!(created.status.as_deref(), Some("pending")); + assert_eq!( + created.approval_url, + "https://console.example.com/approve/req-1" + ); + // expiresAt is epoch millis (a number), not a string. + assert_eq!(created.expires_at, Some(1_739_000_000_000)); + } + + #[test] + fn poll_response_reads_claim_code_from_result_envelope() { + let approved: AccessRequestStatus = unwrap_result( + r#"{ + "result": { + "requestId": "req-1", + "status": "approved", + "grantedRoleIds": ["role-a", "role-b"], + "issuedKeyId": "key-1", + "claimCode": "claim-xyz", + "expiresAt": 1739000000000 + }, + "requestId": "corr-2" + }"#, + ); + assert_eq!(approved.status.as_deref(), Some("approved")); + assert_eq!(approved.claim_code.as_deref(), Some("claim-xyz")); + } + + #[test] + fn pending_poll_response_has_no_claim_code() { + let pending: AccessRequestStatus = unwrap_result( + r#"{"result":{"requestId":"req-1","status":"pending","expiresAt":1739000000000},"requestId":"corr-3"}"#, + ); + assert_eq!(pending.status.as_deref(), Some("pending")); + assert!(pending.claim_code.is_none()); + } + + #[test] + fn claim_response_reads_key_and_secret_from_result_envelope() { + let claimed: ClaimResponse = unwrap_result( + r#"{ + "result": { "keyId": "mgmt-1", "key": "PUBLICKEY", "secret": "SH_secret" }, + "requestId": "corr-4" + }"#, + ); + assert_eq!(claimed.key_id, "mgmt-1"); + assert_eq!(claimed.key, "PUBLICKEY"); + assert_eq!(claimed.secret, "SH_secret"); + } +} diff --git a/crates/clickhousectl/src/cloud/cli.rs b/crates/clickhousectl/src/cloud/cli.rs index 1f3c2704..3d4621cb 100644 --- a/crates/clickhousectl/src/cloud/cli.rs +++ b/crates/clickhousectl/src/cloud/cli.rs @@ -1,3 +1,5 @@ +#[cfg(feature = "experimental-access-negotiation")] +pub(crate) use crate::cloud::access::AccessCommands; pub(crate) use crate::cloud::activity::ActivityCommands; pub(crate) use crate::cloud::api_keys::KeyCommands; pub(crate) use crate::cloud::auth::AuthCommands; @@ -63,6 +65,75 @@ pub struct CloudArgs { } impl CloudArgs { + /// Reparse only access invocations without selector propagation so clap + /// cannot erase conflicting values supplied at different command depths. + #[cfg(feature = "experimental-access-negotiation")] + pub fn validate_access_selectors( + &self, + argv: &[std::ffi::OsString], + ) -> Result<(), clap::Error> { + if !matches!(self.command, CloudCommands::Access { .. }) { + return Ok(()); + } + fn without_selector_globals( + mut command: clap::Command, + inherited: &[clap::Arg], + ) -> clap::Command { + for arg in inherited { + if !command + .get_arguments() + .any(|local| local.get_id() == arg.get_id()) + { + command = command.arg(arg.clone()); + } + } + let selectors: Vec<_> = command + .get_arguments() + .filter(|arg| { + ["org", "org_id", "org_name"].contains(&arg.get_id().as_str()) + && arg.is_global_set() + }) + .cloned() + .collect(); + for arg in &selectors { + command = command.mut_arg(arg.get_id().clone(), |arg| arg.global(false)); + } + for child in command.get_subcommands_mut() { + *child = without_selector_globals(child.clone(), &selectors); + } + command + } + // Never mutate a built clap command: its cached parser indexes would + // describe the old globals. Reuse fresh derive definitions instead. + use clap::CommandFactory; + let mut validation = without_selector_globals(crate::cli::Cli::command(), &[]); + let matches = validation.try_get_matches_from_mut(argv)?; + let mut current = &matches; + let mut selected: Option<(&str, &str)> = None; + loop { + for id in ["org", "org_id", "org_name"] { + if current.try_get_one::(id).ok().flatten().is_none() + || current.value_source(id) != Some(clap::parser::ValueSource::CommandLine) + { + continue; + } + let value = current.get_one::(id).expect("explicit selector"); + let kind = if id == "org_name" { "name" } else { "id" }; + if selected.is_some_and(|previous| previous != (kind, value.as_str())) { + return Err(validation.error( + clap::error::ErrorKind::ArgumentConflict, + "Conflicting organization selectors: use one organization ID (--org or --org-id) or one exact --org-name", + )); + } + selected = Some((kind, value)); + } + match current.subcommand() { + Some((_, child)) => current = child, + None => return Ok(()), + } + } + } + /// Clap validates conflicts before propagating globals from other depths. pub fn has_organization_selector_conflict(&self) -> bool { self.org_id.is_some() && self.org_name.is_some() @@ -127,6 +198,25 @@ pub enum CloudCommands { command: AuthCommands, }, + /// Discover roles and request scoped API keys + #[cfg(feature = "experimental-access-negotiation")] + #[command(after_help = "\ +CONTEXT FOR AGENTS: + Requires user OAuth identity from `cloud auth login`, even when an API key is saved. + Discover assignable roles with `cloud access roles`; policies describe current authority. + Request a scoped key with `cloud access request --perms role:`; a human approves it. + Use request --no-browser for manual handoff; the approval URL goes to stderr. + API URL: explicit --url, then saved OAuth login environment, then the normal Cloud default. + Exactly one user organization is selected automatically; otherwise choose --org or --org-name. + Organization IDs: `cloud org list --json`. --org-id remains supported.")] + Access { + /// Organization ID (conflicts with --org-name) + #[arg(long, global = true, value_name = "ID", conflicts_with = "org_name")] + org: Option, + #[command(subcommand)] + command: AccessCommands, + }, + /// Manage organizations #[command(after_help = "\ CONTEXT FOR AGENTS: @@ -285,6 +375,8 @@ impl CloudCommands { pub fn is_write_command(&self) -> bool { match self { CloudCommands::Auth { command } => command.is_write(), + #[cfg(feature = "experimental-access-negotiation")] + CloudCommands::Access { command, .. } => command.is_write(), CloudCommands::Org { command } => command.is_write(), CloudCommands::Service { command } => command.is_write(), CloudCommands::Backup { command } => command.is_write(), diff --git a/crates/clickhousectl/src/cloud/client.rs b/crates/clickhousectl/src/cloud/client.rs index 84bdeaff..18b2c2d6 100644 --- a/crates/clickhousectl/src/cloud/client.rs +++ b/crates/clickhousectl/src/cloud/client.rs @@ -116,8 +116,27 @@ impl From for CloudError { pub type Result = std::result::Result; enum AuthMode { + Basic { + key: String, + secret: String, + }, + // The token is read only by `request_auth` (experimental-access-negotiation), + // but both builds keep the same variant shape. + #[cfg_attr(not(feature = "experimental-access-negotiation"), allow(dead_code))] + Bearer { + token: String, + }, +} + +/// The authentication a bespoke control-plane request (one made with a plain +/// `reqwest` client, outside the typed Cloud API client) should carry. Mirrors +/// [`AuthMode`] but owns its values so callers can build request headers after +/// the borrow on the client ends. Used by the access-request PKCE flow +/// (`cloud access request`), which talks to endpoints the library does not model. +#[cfg(feature = "experimental-access-negotiation")] +pub(crate) enum RequestAuth { Basic { key: String, secret: String }, - Bearer, + Bearer { token: String }, } /// The resolved credential source that won precedence for a `CloudClient`. @@ -577,8 +596,7 @@ impl CloudClient { api_secret: Option<&str>, url_override: Option<&str>, ) -> Result { - let resolved = resolve_auth(api_key, api_secret, url_override)?; - Self::from_resolved(resolved, None) + Self::from_resolved_auth(resolve_auth(api_key, api_secret, url_override)?, None) } /// Like [`CloudClient::new`], but every request gives up after `timeout`, @@ -589,8 +607,10 @@ impl CloudClient { url_override: Option<&str>, timeout: std::time::Duration, ) -> Result { - let resolved = resolve_auth(api_key, api_secret, url_override)?; - Self::from_resolved(resolved, Some(timeout)) + Self::from_resolved_auth( + resolve_auth(api_key, api_secret, url_override)?, + Some(timeout), + ) } /// A client for exactly this API key pair, bypassing the precedence @@ -611,7 +631,7 @@ impl CloudClient { .map(crate::cloud::auth::normalize_api_url) .unwrap_or_else(|| DEFAULT_BASE_URL.to_string()), }; - Self::from_resolved(resolved, Some(timeout)) + Self::from_resolved_auth(resolved, Some(timeout)) } /// A client for exactly these OAuth tokens, bypassing the precedence @@ -626,12 +646,66 @@ impl CloudClient { token: tokens.access_token.clone(), }, source: AuthSource::OAuthTokens, - base_url: tokens.api_url.clone(), + base_url: crate::cloud::auth::normalize_api_url(&tokens.api_url), }; - Self::from_resolved(resolved, Some(timeout)) + Self::from_resolved_auth(resolved, Some(timeout)) } - fn from_resolved(resolved: ResolvedAuth, timeout: Option) -> Result { + /// Access negotiation identifies the user, never a previously issued API key. + #[cfg(feature = "experimental-access-negotiation")] + pub(super) async fn for_access(url_override: Option<&str>) -> Result { + // Suppress the normal refresh warning's API-key fallback advice: this + // lane requires OAuth even when other credentials are configured. + crate::cloud::auth::ensure_fresh_tokens(true) + .await + .map_err(|error| { + CloudError::auth(format!( + "Unable to refresh OAuth identity: {error}. Run `clickhousectl cloud auth login` to re-authenticate." + )) + })?; + Self::from_access_tokens( + crate::cloud::auth::load_tokens() + .map_err(|_| CloudError::auth(crate::cloud::auth::OAUTH_LOGIN_REQUIRED))?, + url_override, + // Membership and assignable-role discovery use the typed client + // directly, outside the handshake's `operation` wrapper, so this + // client carries the same 30-second access-operation bound; a + // server holding a response open must not stall either command. + Some(crate::cloud::access::OPERATION_TIMEOUT), + ) + } + + #[cfg(feature = "experimental-access-negotiation")] + fn from_access_tokens( + tokens: Option, + url_override: Option<&str>, + timeout: Option, + ) -> Result { + let tokens = tokens + .filter(crate::cloud::auth::is_token_valid) + .ok_or_else(|| { + CloudError::auth( + "Access negotiation requires OAuth identity. Run `clickhousectl cloud auth login` to authenticate. API keys cannot be used for access negotiation.", + ) + })?; + Self::from_resolved_auth( + ResolvedAuth { + creds: ResolvedCreds::Bearer { + token: tokens.access_token, + }, + source: AuthSource::OAuthTokens, + base_url: crate::cloud::auth::normalize_api_url( + url_override.unwrap_or(&tokens.api_url), + ), + }, + timeout, + ) + } + + fn from_resolved_auth( + resolved: ResolvedAuth, + timeout: Option, + ) -> Result { let mut builder = crate::http::client_builder(); if let Some(timeout) = timeout { builder = builder.timeout(timeout); @@ -652,8 +726,8 @@ impl CloudClient { AuthMode::Basic { key, secret }, ), ResolvedCreds::Bearer { token } => ( - clickhouse_cloud_api::Client::with_http_client_bearer(http, lib_url, token), - AuthMode::Bearer, + clickhouse_cloud_api::Client::with_http_client_bearer(http, lib_url, token.clone()), + AuthMode::Bearer { token }, ), }; @@ -699,7 +773,7 @@ impl CloudClient { /// Returns true if the client is using OAuth Bearer token authentication. /// Bearer auth is read-only and cannot perform write operations. pub fn is_bearer_auth(&self) -> bool { - matches!(&self.auth_mode, AuthMode::Bearer) + matches!(&self.auth_mode, AuthMode::Bearer { .. }) } /// The active API key pair, for authenticating directly to a Query API @@ -707,7 +781,23 @@ impl CloudClient { pub(crate) fn basic_auth_credentials(&self) -> Option<(&str, &str)> { match &self.auth_mode { AuthMode::Basic { key, secret } => Some((key, secret)), - AuthMode::Bearer => None, + AuthMode::Bearer { .. } => None, + } + } + + /// The resolved credentials in a form a bespoke `reqwest` request can carry, + /// for control-plane endpoints the typed library client does not model (the + /// access-request PKCE flow). Clones so the value outlives the client borrow. + #[cfg(feature = "experimental-access-negotiation")] + pub(crate) fn request_auth(&self) -> RequestAuth { + match &self.auth_mode { + AuthMode::Basic { key, secret } => RequestAuth::Basic { + key: key.clone(), + secret: secret.clone(), + }, + AuthMode::Bearer { token } => RequestAuth::Bearer { + token: token.clone(), + }, } } @@ -887,6 +977,82 @@ mod tests { const DEFAULT_LIB_BASE_URL: &str = "https://api.clickhouse.cloud"; + /// The access client's per-request bound must actually reach reqwest and + /// classify as a timeout: a response held open past the bound fails with + /// `FailureKind::Timeout` instead of stalling. `for_access` supplies + /// `access::OPERATION_TIMEOUT`; a short bound here keeps the test fast. + #[tokio::test] + #[cfg(feature = "experimental-access-negotiation")] + async fn access_client_applies_the_access_operation_bound_to_discovery() { + use std::time::Duration; + use wiremock::matchers::{method, path}; + use wiremock::{Mock, MockServer, ResponseTemplate}; + let server = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/v1/organizations")) + .respond_with(ResponseTemplate::new(200).set_delay(Duration::from_secs(60))) + .mount(&server) + .await; + let tokens = serde_json::from_value(serde_json::json!({ + "access_token": "test-token", "refresh_token": "unused", + "expires_at": 4102444800_i64, + "api_url": format!("{}/v1", server.uri()), + "auth_config": {"auth_url": server.uri(), "client_id": "test-client"}, + })) + .unwrap(); + let client = + CloudClient::from_access_tokens(Some(tokens), None, Some(Duration::from_millis(300))) + .unwrap(); + let error = client + .api() + .organization_get_list(None, None) + .await + .map(|_| ()) + .map_err(|error| client.convert_error(error)) + .unwrap_err(); + assert!( + matches!( + error.failure.as_ref().map(|f| &f.kind), + Some(crate::failure::FailureKind::Timeout) + ), + "expected a timeout classification: {error}" + ); + } + + #[test] + #[cfg(feature = "experimental-access-negotiation")] + fn access_client_uses_explicit_or_saved_api_base() { + // Records are complete: both `api_url` and `auth_config` are required + // at deserialization (rejection covered by the authentication tests), + // and endpoint selection only runs on a loaded record. + for (saved, explicit, expected) in [ + ( + "https://saved.example/v1/", + None, + "https://saved.example/v1", + ), + ( + "https://saved.example/v1", + Some("http://localhost:2023/"), + "http://localhost:2023/v1", + ), + (DEFAULT_BASE_URL, None, DEFAULT_BASE_URL), + ] { + let value = serde_json::json!({ + "access_token": "test-token", "refresh_token": "unused", + "expires_at": 4102444800_i64, + "api_url": saved, + "auth_config": {"auth_url": "https://auth.example.com", "client_id": "test-client"}, + }); + let tokens = serde_json::from_value(value).unwrap(); + let client = CloudClient::from_access_tokens(Some(tokens), explicit, None).unwrap(); + assert_eq!(client.base_url(), expected); + assert_eq!(client.auth_source(), AuthSource::OAuthTokens); + assert!(client.is_bearer_auth()); + assert!(client.basic_auth_credentials().is_none()); + } + } + #[test] fn typed_udf_dependency_error_keeps_message_and_http_classification() { let error = @@ -967,7 +1133,9 @@ mod tests { organization_id: None, organization_name: None, resolved_organization_id: tokio::sync::OnceCell::new(), - auth_mode: AuthMode::Bearer, + auth_mode: AuthMode::Bearer { + token: "test_token".into(), + }, auth_source: AuthSource::OAuthTokens, base_url: DEFAULT_BASE_URL.to_string(), }; @@ -1123,7 +1291,9 @@ mod tests { organization_id: None, organization_name: None, resolved_organization_id: tokio::sync::OnceCell::new(), - auth_mode: AuthMode::Bearer, + auth_mode: AuthMode::Bearer { + token: "test_token".into(), + }, auth_source: AuthSource::OAuthTokens, base_url: DEFAULT_BASE_URL.to_string(), }; diff --git a/crates/clickhousectl/src/cloud/mod.rs b/crates/clickhousectl/src/cloud/mod.rs index acda62e5..ec14e7c1 100644 --- a/crates/clickhousectl/src/cloud/mod.rs +++ b/crates/clickhousectl/src/cloud/mod.rs @@ -1,3 +1,5 @@ +#[cfg(feature = "experimental-access-negotiation")] +pub mod access; pub mod activity; pub mod api_keys; pub mod auth; @@ -71,6 +73,21 @@ pub async fn run(args: CloudArgs, json: bool) -> Result<()> { ) .await; } + // Access negotiation refreshes and uses only the OAuth user identity, + // bypassing normal API-key precedence and the ordinary OAuth write gate. + #[cfg(feature = "experimental-access-negotiation")] + CloudCommands::Access { command, org } => { + return access::run( + command, + args.url.as_deref(), + org.or(args.org_id), + args.org_name, + args.debug, + json, + ) + .await + .map_err(cloud_error_to_top_level); + } command => command, }; @@ -108,11 +125,7 @@ pub async fn run(args: CloudArgs, json: bool) -> Result<()> { } if args.debug { - eprint_line(format!( - "[debug] auth source: {}", - client.auth_source().describe() - )); - eprint_line(format!("[debug] api url: {}", client.base_url())); + print_client_diagnostics(&client); } // OAuth (Bearer) tokens are read-only. Block write commands early @@ -137,6 +150,14 @@ pub async fn run(args: CloudArgs, json: bool) -> Result<()> { .map_err(cloud_error_to_top_level) } +fn print_client_diagnostics(client: &CloudClient) { + eprint_line(format!( + "[debug] auth source: {}", + client.auth_source().describe() + )); + eprint_line(format!("[debug] api url: {}", client.base_url())); +} + fn cloud_error_to_top_level(e: CloudError) -> Error { match (e.kind, e.details) { // Auth failures own the exit code (4) and have their own remediation @@ -164,6 +185,8 @@ fn cloud_error_to_top_level(e: CloudError) -> Error { async fn dispatch(client: &CloudClient, command: CloudCommands, json: bool) -> client::Result<()> { match command { CloudCommands::Auth { command } => auth::run_with_client(client, command, json).await, + #[cfg(feature = "experimental-access-negotiation")] + CloudCommands::Access { .. } => unreachable!("handled above"), CloudCommands::Org { command } => organizations::run_org(client, command, json).await, CloudCommands::Service { command } => services::run(client, command, json).await, CloudCommands::Member { command } => organizations::run_member(client, command, json).await, diff --git a/crates/clickhousectl/src/cloud/permissions.rs b/crates/clickhousectl/src/cloud/permissions.rs index f458dcc7..acb33067 100644 --- a/crates/clickhousectl/src/cloud/permissions.rs +++ b/crates/clickhousectl/src/cloud/permissions.rs @@ -101,6 +101,12 @@ impl Declaration { } fn declarations() -> impl Iterator { + // Feature-gated declarations stay a compile-time constant in both builds + // so the iterator's shape is feature-agnostic. + #[cfg(feature = "experimental-access-negotiation")] + const ACCESS_DECLARATIONS: &[&[Declaration]] = &[super::access::PERMISSIONS]; + #[cfg(not(feature = "experimental-access-negotiation"))] + const ACCESS_DECLARATIONS: &[&[Declaration]] = &[]; [ super::activity::PERMISSIONS, super::api_keys::PERMISSIONS, @@ -117,6 +123,7 @@ fn declarations() -> impl Iterator { super::udfs::PERMISSIONS, ] .into_iter() + .chain(ACCESS_DECLARATIONS.iter().copied()) .flatten() } diff --git a/crates/clickhousectl/src/main.rs b/crates/clickhousectl/src/main.rs index a2c6b95c..5f09fea3 100644 --- a/crates/clickhousectl/src/main.rs +++ b/crates/clickhousectl/src/main.rs @@ -70,18 +70,26 @@ async fn main() { let cli = Cli::from_arg_matches(&matches) .expect("Cli::from_arg_matches must accept matches from Cli::command()"); let read_only_telemetry_status = is_read_only_telemetry_status(&cli.command); - let run_result = match validate_post_parse(&cli, &mut cmd) { - Ok(()) => run_parsed(cli, read_only_telemetry_status).await, - Err(e) => { - stdout::record(e.print()); - RunOutcome { - exit_code: e.exit_code(), - is_child_exit: false, - defer_telemetry_notice: false, - local_failure: None, - } - } + #[cfg(feature = "experimental-access-negotiation")] + let access_validation = match &cli.command { + Commands::Cloud(args) => args.validate_access_selectors(&argv), + _ => Ok(()), }; + #[cfg(not(feature = "experimental-access-negotiation"))] + let access_validation = Ok::<(), clap::Error>(()); + let run_result = + match access_validation.and_then(|()| validate_post_parse(&cli, &mut cmd)) { + Ok(()) => run_parsed(cli, read_only_telemetry_status).await, + Err(e) => { + stdout::record(e.print()); + RunOutcome { + exit_code: e.exit_code(), + is_child_exit: false, + defer_telemetry_notice: false, + local_failure: None, + } + } + }; let RunOutcome { exit_code, is_child_exit, @@ -205,6 +213,21 @@ fn validate_post_parse(cli: &Cli, cmd: &mut clap::Command) -> std::result::Resul )); } + #[cfg(feature = "experimental-access-negotiation")] + if let cloud::cli::CloudCommands::Access { + command: cloud::cli::AccessCommands::Request(request), + .. + } = &args.command + && let Err(message) = request.requested_role_ids() + { + let command = cmd + .find_subcommand_mut("cloud") + .and_then(|cloud| cloud.find_subcommand_mut("access")) + .and_then(|access| access.find_subcommand_mut("request")) + .expect("cloud access request command must exist"); + return Err(command.error(ErrorKind::ValueValidation, message)); + } + // Login's credentials are global arguments. Validate after propagation so // a pair split across command levels remains valid. if let cloud::cli::CloudCommands::Auth { command } = &args.command diff --git a/crates/clickhousectl/tests/cli_request_shape_test.rs b/crates/clickhousectl/tests/cli_request_shape_test.rs index 58eab68c..72d20f8a 100644 --- a/crates/clickhousectl/tests/cli_request_shape_test.rs +++ b/crates/clickhousectl/tests/cli_request_shape_test.rs @@ -34703,3 +34703,1337 @@ async fn org_notifications_accepts_an_empty_cursor_without_fetching_more_pages() }) ); } + +// The access-negotiation flow is experimental and compiled only with +// `--features experimental-access-negotiation`. +#[cfg(feature = "experimental-access-negotiation")] +mod access_negotiation { + #![allow(clippy::too_many_lines)] + + use super::*; + + // ── Access negotiation's OAuth-only identity lane ────────────────────────── + + fn access_roles_command(dir: &Path, json: bool) -> Command { + let mut command = Command::new(clickhousectl_binary()); + command + .env_clear() + .env("DO_NOT_TRACK", "1") + .env("HOME", dir.join("home")) + .current_dir(dir) + .args(["cloud", "access", "roles"]); + if json { + command.arg("--json"); + } + command + } + + fn assignable_roles_fixture() -> Value { + serde_json::json!([ + {"id": "00000000-0000-0000-0000-000000000001", "name": "System reader", "type": "system", "policies": [ + {"allowDeny": "ALLOW", "permissions": ["service:view"], "resources": ["*"], "tags": {"roleV2": "sql-console-readonly"}}, + {"allowDeny": "DENY", "permissions": ["service:delete"], "resources": ["instance/private"]} + ]}, + {"id": "55555555-5555-4555-8555-555555555555", "name": "Custom SQL", "type": "custom", "policies": [ + {"allowDeny": "ALLOW", "permissions": ["sql-console:database:access"], "resources": ["instance/service-a"], "tags": {"roleV2": "sql-console-admin", "grants": ["GRANT SELECT ON db.* TO role", "REVOKE SELECT ON db.secret FROM role"]}}, + {"allowDeny": "future-effect", "permissions": [], "resources": [], "tags": {"roleV2": "future-role", "grants": []}} + ]}, + {"id": "22222222-2222-2222-2222-222222222222", "name": "Future type", "type": "future-type", "policies": []} + ]) + } + + #[tokio::test] + async fn access_roles_outputs_complete_oauth_menu_with_selected_organization() { + const ORG: &str = "11111111-2222-3333-4444-555555555555"; + for selector in ["explicit", "auto", "name"] { + for json in [false, true] { + let server = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/v1/organizations")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(ResponseTemplate::new(200).set_body_json( + serde_json::json!({"result": [{"id": ORG, "name": "Selected org"}]}), + )) + .expect(if selector == "explicit" { 0 } else { 1 }) + .mount(&server) + .await; + let roles = assignable_roles_fixture(); + Mock::given(method("GET")).and(path(format!("/v1/organizations/{ORG}/roles"))) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"result": roles, "requestId": "correlation", "nextCursor": "must-not-page"}))) + .expect(1).mount(&server).await; + let dir = tempfile::tempdir().unwrap(); + let tokens = access_credentials(dir.path()); + write_oauth_tokens(&tokens, &server.uri()); + let mut command = access_roles_command(dir.path(), json); + let debug = selector != "explicit"; + if debug { + command.arg("--debug"); + } + if selector == "explicit" { + command.args(["--org", ORG]); + } + if selector == "name" { + command.args(["--org-name", "Selected org"]); + } + let output = command.output().unwrap(); + assert_eq!( + output.status.code(), + Some(0), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert_access_output_has_no_credentials(&output); + if debug { + assert_access_debug_diagnostics(&output, &server.uri()); + } else { + assert!(!String::from_utf8_lossy(&output.stderr).contains("[debug]")); + } + if json { + assert_eq!( + serde_json::from_slice::(&output.stdout).unwrap(), + roles + ); + } else { + let text = String::from_utf8_lossy(&output.stdout); + for value in [ + ORG, + "System reader", + "Custom SQL", + "DENY", + "instance/private", + "Unknown effect (future-effect)", + "Unknown (future-type)", + "00000000-0000-0000-0000-000000000001", + ] { + assert!(text.contains(value), "missing {value}: {text}"); + } + } + assert_eq!( + server.received_requests().await.unwrap().len(), + if selector == "explicit" { 1 } else { 2 } + ); + } + } + } + + #[tokio::test] + async fn access_roles_empty_and_optional_tags_are_valid() { + for roles in [ + serde_json::json!([]), + serde_json::json!([ + {"id": "00000000-0000-0000-0000-000000000001", "name": "empty", "type": "system", "policies": [ + {"allowDeny": "DENY", "permissions": [], "resources": [], "tags": null}, + {"allowDeny": "ALLOW", "permissions": [], "resources": [], "tags": {"grants": null, "roleV2": null}} + ]} + ]), + ] { + for json in [false, true] { + let server = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/v1/organizations/org/roles")) + .respond_with( + ResponseTemplate::new(200) + .set_body_json(serde_json::json!({"result": roles})), + ) + .expect(1) + .mount(&server) + .await; + let dir = tempfile::tempdir().unwrap(); + write_oauth_tokens(&access_credentials(dir.path()), &server.uri()); + let output = access_roles_command(dir.path(), json) + .args(["--org", "org"]) + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(0), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + if json { + let expected: Vec = + serde_json::from_value(roles.clone()).unwrap(); + assert_eq!( + serde_json::from_slice::(&output.stdout).unwrap(), + serde_json::to_value(expected).unwrap() + ); + } else if roles.as_array().unwrap().is_empty() { + assert!( + String::from_utf8_lossy(&output.stdout).contains("No assignable roles") + ); + } + } + } + } + + #[tokio::test] + async fn access_roles_rejects_incomplete_menus_before_any_success_output() { + let fixture = assignable_roles_fixture(); + let mut invalid = vec![ + serde_json::json!({}), + serde_json::json!({"result": null}), + serde_json::json!({"result": fixture, "error": "envelope failure"}), + serde_json::json!({"result": fixture, "status": 500}), + serde_json::json!({"result": {"roles": []}}), + ]; + for field in ["id", "name", "type", "policies"] { + for null in [false, true] { + let mut roles = fixture.clone(); + if null { + roles[1][field] = Value::Null; + } else { + roles[1].as_object_mut().unwrap().remove(field); + } + invalid.push(serde_json::json!({"result": roles})); + } + } + for field in ["allowDeny", "permissions", "resources"] { + for null in [false, true] { + let mut roles = fixture.clone(); + if null { + roles[1]["policies"][0][field] = Value::Null; + } else { + roles[1]["policies"][0] + .as_object_mut() + .unwrap() + .remove(field); + } + invalid.push(serde_json::json!({"result": roles})); + } + } + let mut roles = fixture.clone(); + roles[1]["id"] = "not-a-uuid".into(); + invalid.push(serde_json::json!({"result": roles})); + for response in invalid { + for json in [false, true] { + let server = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/v1/organizations/org/roles")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(ResponseTemplate::new(200).set_body_json(response.clone())) + .expect(1) + .mount(&server) + .await; + let dir = tempfile::tempdir().unwrap(); + write_oauth_tokens(&access_credentials(dir.path()), &server.uri()); + let output = access_roles_command(dir.path(), json) + .args(["--org", "org"]) + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(1), + "{response}: {}", + String::from_utf8_lossy(&output.stderr) + ); + assert!( + output.stdout.is_empty(), + "partial success: {}", + String::from_utf8_lossy(&output.stdout) + ); + assert!(!output.stderr.is_empty()); + assert_access_output_has_no_credentials(&output); + assert_eq!(server.received_requests().await.unwrap().len(), 1); + } + } + } + + #[tokio::test] + async fn access_roles_preserves_server_denials_without_fallback() { + for (status, message, exit) in [ + (401, "non-CLI identity", 4), + (403, "cross-org denial", 4), + (400, "RBAC prerequisite", 1), + (500, "upstream error", 1), + ] { + for json in [false, true] { + let server = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/v1/organizations/other-org/roles")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with( + ResponseTemplate::new(status) + .set_body_json(serde_json::json!({"error": message})), + ) + .expect(1) + .mount(&server) + .await; + let dir = tempfile::tempdir().unwrap(); + write_oauth_tokens(&access_credentials(dir.path()), &server.uri()); + let output = access_roles_command(dir.path(), json) + .args(["--org", "other-org"]) + .output() + .unwrap(); + assert_eq!(output.status.code(), Some(exit)); + assert!(output.stdout.is_empty()); + assert!(String::from_utf8_lossy(&output.stderr).contains(message)); + assert_eq!(server.received_requests().await.unwrap().len(), 1); + } + } + } + + #[tokio::test] + async fn access_roles_requires_and_refreshes_oauth_despite_basic_credentials() { + for state in ["missing", "unrefreshable", "refresh"] { + let issuer = MockServer::start().await; + let target = MockServer::start().await; + let dir = tempfile::tempdir().unwrap(); + let tokens = access_credentials(dir.path()); + if state != "missing" { + std::fs::write(tokens.join("tokens.json"), serde_json::to_vec(&serde_json::json!({ + "access_token": "test-bearer-token", "refresh_token": "refresh-secret", "expires_at": 1, + "api_url": format!("{}/v1", issuer.uri()), + "auth_config": {"auth_url": issuer.uri(), "client_id": "test-client"} + })).unwrap()).unwrap(); + Mock::given(method("POST")).and(path("/oauth/token")) + .respond_with(if state == "refresh" { + ResponseTemplate::new(200).set_body_json(serde_json::json!({"access_token": "refreshed-bearer-token", "expires_in": 3600, "token_type": "Bearer"})) + } else { ResponseTemplate::new(401).set_body_string("refresh-secret") }) + .expect(1).mount(&issuer).await; + } + Mock::given(method("GET")) + .and(path("/v1/organizations/org/roles")) + .and(header("authorization", "Bearer refreshed-bearer-token")) + .respond_with( + ResponseTemplate::new(200).set_body_json(serde_json::json!({"result": []})), + ) + .expect(if state == "refresh" { 1 } else { 0 }) + .mount(&target) + .await; + let output = access_roles_command(dir.path(), true) + .args([ + "--org", + "org", + "--url", + &target.uri(), + "--api-key", + "flag-basic-key", + "--api-secret", + "flag-basic-secret", + ]) + .env("CLICKHOUSECTL_AUTH_URL", issuer.uri()) + .env("CLICKHOUSECTL_CLIENT_ID", "test-client") + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(if state == "refresh" { 0 } else { 4 }) + ); + assert_access_output_has_no_credentials(&output); + if state == "refresh" { + assert_eq!( + serde_json::from_slice::(&output.stdout).unwrap(), + serde_json::json!([]) + ); + } else { + assert!(output.stdout.is_empty()); + assert!(String::from_utf8_lossy(&output.stderr).contains("cloud auth login")); + } + assert_eq!( + target.received_requests().await.unwrap().len(), + usize::from(state == "refresh") + ); + } + } + + #[test] + fn access_roles_selector_conflicts_exit_usage_before_oauth() { + for args in [ + vec!["cloud", "--org-id=one", "access", "roles", "--org=two"], + vec![ + "cloud", + "--org-name=one", + "access", + "roles", + "--org-name=two", + ], + vec!["cloud", "access", "--org=one", "roles", "--org-id=two"], + vec!["cloud", "--org-name=one", "access", "roles", "--org=one"], + ] { + let dir = tempfile::tempdir().unwrap(); + access_credentials(dir.path()); + let output = Command::new(clickhousectl_binary()) + .env_clear() + .env("DO_NOT_TRACK", "1") + .env("HOME", dir.path().join("home")) + .current_dir(dir.path()) + .args(args) + .output() + .unwrap(); + assert_eq!(output.status.code(), Some(2)); + assert!(output.stdout.is_empty()); + } + } + + #[tokio::test] + async fn access_request_no_browser_handoff_keeps_stdout_final_and_credentials_safe() { + use base64::Engine as _; + use sha2::{Digest as _, Sha256}; + const A: &str = "21111111-1111-1111-9999-000000000007"; + const B: &str = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; + const C: &str = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"; + let mut tasks = Vec::new(); + for json in [false, true] { + for scenario in [ + "callback", + "poll", + "denied", + "expired", + "401", + "403", + "unknown", + "malformed", + "empty-code", + "claim-error", + "claim-malformed", + "claim-empty", + "csrf", + "callback-mismatch", + "claim-deadline-missing", + "claim-deadline-expired", + "already-claimed", + "pending-expired-approved", + "pending-expired-pending", + "create-deadline-missing", + "create-envelope-error", + "create-proposal-missing", + "create-proposal-null", + "create-proposal-malformed", + "create-proposal-duplicate", + "create-proposal-oversized", + "create-proposal-mismatch", + "create-proposal-unexpected", + "get-proposal-missing", + "get-proposal-null", + "get-proposal-mismatch", + "get-proposal-duplicate", + "get-identity-mismatch", + "claim-grants-missing", + "claim-grants-null", + "claim-grants-empty", + "claim-grants-malformed", + "claim-grants-duplicate", + "claim-grants-many", + ] { + tasks.push(tokio::spawn(async move { + let server = MockServer::start().await; + let dir = tempfile::tempdir().unwrap(); + let tokens = access_credentials(dir.path()); + write_oauth_tokens(&tokens, &server.uri()); + let credentials_path = dir.path().join(".clickhouse/credentials.json"); + let before = std::fs::read(&credentials_path).unwrap(); + let oauth_before = std::fs::read(tokens.join("tokens.json")).unwrap(); + let approval_url = format!("{}/approve/request-1", server.uri()); + let pending_expiry = if scenario.starts_with("pending-expired") {1_i64} else {4102444800000_i64}; + let proposed = matches!(scenario, "callback" | "poll") || scenario.starts_with("create-proposal") || scenario.starts_with("get-proposal"); + let proposal = if proposed {serde_json::json!([A, B.to_uppercase()])} else {serde_json::json!([])}; + let mut create = serde_json::json!({"result": { + "requestedRoleIds":proposal, "requestId":"request-1", "status":"pending", "approvalUrl":approval_url, "expiresAt":pending_expiry + }}); + if scenario == "create-deadline-missing" { create["result"]["expiresAt"] = Value::Null; } + if scenario == "create-envelope-error" { create["error"] = serde_json::json!("secret-envelope-error"); } + match scenario { + "create-proposal-missing" => { create["result"].as_object_mut().unwrap().remove("requestedRoleIds"); }, + "create-proposal-null" => create["result"]["requestedRoleIds"] = Value::Null, + "create-proposal-malformed" => create["result"]["requestedRoleIds"] = serde_json::json!(["secret-invalid-role"]), + "create-proposal-duplicate" => create["result"]["requestedRoleIds"] = serde_json::json!([B, B.to_uppercase()]), + "create-proposal-oversized" => create["result"]["requestedRoleIds"] = serde_json::json!((1..=101).map(|id| format!("00000000-0000-0000-0000-{id:012x}")).collect::>()), + "create-proposal-mismatch" => create["result"]["requestedRoleIds"] = serde_json::json!([B, A]), + _ => {}, + } + Mock::given(method("POST")).and(path("/v1/organizations/org/accessRequests")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(ResponseTemplate::new(200).set_body_json(create)).expect(1).mount(&server).await; + let mut approved = serde_json::json!({"result":{"requestedRoleIds":proposal,"grantedRoleIds":[A],"requestId":"request-1","expiresAt":pending_expiry,"claimCodeExpiresAt":4102444800000_i64,"status":"approved","claimCode":"claim-code-secret"}}); + match scenario { + "get-proposal-missing" => {approved["result"].as_object_mut().unwrap().remove("requestedRoleIds");}, + "get-proposal-null" => approved["result"]["requestedRoleIds"] = Value::Null, + "get-proposal-mismatch" => approved["result"]["requestedRoleIds"] = serde_json::json!([B, A]), + "get-proposal-duplicate" => approved["result"]["requestedRoleIds"] = serde_json::json!([B, B.to_uppercase()]), + "get-identity-mismatch" => approved["result"]["requestId"] = serde_json::json!("other-request"), + "claim-deadline-missing" => approved["result"]["claimCodeExpiresAt"] = Value::Null, + "claim-deadline-expired" => approved["result"]["claimCodeExpiresAt"] = serde_json::json!(1), + "already-claimed" => approved["result"]["issuedKeyId"] = serde_json::json!("issued-id"), + "pending-expired-pending" => approved["result"]["status"] = serde_json::json!("pending"), + _ => {}, + } + let status = match scenario { + "401" | "403" => ResponseTemplate::new(scenario.parse::().unwrap()).set_body_string("claim-code-secret"), + "denied" | "expired" | "unknown" => ResponseTemplate::new(200).set_body_json(serde_json::json!({"result":{"requestedRoleIds":[],"requestId":"request-1","expiresAt":4102444800000_i64,"status":scenario}})), + "malformed" => ResponseTemplate::new(200).set_body_json(serde_json::json!({"result":"claim-code-secret"})), + "empty-code" => ResponseTemplate::new(200).set_body_json(serde_json::json!({"result":{"requestedRoleIds":[],"requestId":"request-1","expiresAt":4102444800000_i64,"claimCodeExpiresAt":4102444800000_i64,"status":"approved","claimCode":""}})), + "csrf" => ResponseTemplate::new(200).set_body_json(serde_json::json!({"result":{"requestedRoleIds":[],"requestId":"request-1","expiresAt":4102444800000_i64,"status":"pending"}})), + _ => ResponseTemplate::new(200).set_body_json(approved), + }; + Mock::given(method("GET")).and(path("/v1/organizations/org/accessRequests/request-1")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(status).mount(&server).await; + let successful = matches!(scenario, "callback" | "poll" | "pending-expired-approved" | "claim-grants-many"); + let claims = usize::from(successful || scenario.starts_with("claim-grants-") || matches!(scenario, "claim-error" | "claim-malformed" | "claim-empty")); + Mock::given(method("POST")).and(path("/v1/organizations/org/accessRequests/request-1/claim")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(move |request: &wiremock::Request| { + if scenario == "claim-error" || scenario == "claim-malformed" { + // Deliberately echo the actual submitted verifier and claim code. + let echoed = String::from_utf8_lossy(&request.body).to_string(); + ResponseTemplate::new(if scenario == "claim-error" {401} else {200}) + .set_body_json(serde_json::json!({"error":echoed,"result":echoed})) + } else { + let mut claimed = serde_json::json!({"result": { + "keyId":"issued-id", "key":"issued-key", "secret":if scenario == "claim-empty" {""} else {"issued-secret"}, + "grantedRoleIds":[B.to_uppercase(), C] + }}); + match scenario { + "claim-grants-missing" => {claimed["result"].as_object_mut().unwrap().remove("grantedRoleIds");}, + "claim-grants-null" => claimed["result"]["grantedRoleIds"] = Value::Null, + "claim-grants-empty" => claimed["result"]["grantedRoleIds"] = serde_json::json!([]), + "claim-grants-malformed" => claimed["result"]["grantedRoleIds"] = serde_json::json!(["secret-invalid-role"]), + "claim-grants-duplicate" => claimed["result"]["grantedRoleIds"] = serde_json::json!([B, B.to_uppercase()]), + "claim-grants-many" => claimed["result"]["grantedRoleIds"] = serde_json::json!((1..=101).map(|id| format!("00000000-0000-0000-0000-{id:012x}")).collect::>()), + _ => {}, + } + ResponseTemplate::new(200).set_body_json(claimed) + } + }).expect(claims as u64).mount(&server).await; + let mut command = Command::new(clickhousectl_binary()); + command.env_clear().env("DO_NOT_TRACK", "1").env("HOME", dir.path().join("home")) + .env("PATH", dir.path().join("empty-path")).current_dir(dir.path()) + .args(["cloud", "access", "request", "--org", "org", "--no-browser", "--debug"]) + .stdout(Stdio::piped()).stderr(Stdio::piped()); + if json { command.arg("--json"); } + if proposed && scenario != "create-proposal-unexpected" { command.args(["--perms", &format!("role:{A},role:{B}")]); } + let child = command.spawn().unwrap(); + let output_task = tokio::task::spawn_blocking(move || { + let mut child = child; + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(12); + while child.try_wait().unwrap().is_none() { + if std::time::Instant::now() >= deadline { + child.kill().unwrap(); + let output = child.wait_with_output().unwrap(); + panic!("request failed to terminate promptly: {}", String::from_utf8_lossy(&output.stderr)); + } + std::thread::sleep(std::time::Duration::from_millis(20)); + } + child.wait_with_output().unwrap() + }); + if matches!(scenario, "callback" | "csrf" | "callback-mismatch") { + let filed = tokio::time::timeout(std::time::Duration::from_secs(5), async { + loop { + if let Some(request) = server.received_requests().await.unwrap().first() { + break serde_json::from_slice::(&request.body).unwrap(); + } + tokio::time::sleep(std::time::Duration::from_millis(10)).await; + } + }).await.unwrap(); + let response = reqwest::Client::new().get(filed["redirectUri"].as_str().unwrap()) + .query(&[("code", if scenario == "callback-mismatch" {"mismatched-secret-code"} else {"claim-code-secret"}), ("state", if scenario == "csrf" {"wrong-state"} else {filed["state"].as_str().unwrap()})]) + .send().await.unwrap(); + assert_eq!(response.status().as_u16(), if scenario == "csrf" {400} else {200}); + } + let output = output_task.await.unwrap(); + let expected_exit = if successful {0} else if matches!(scenario, "401" | "403" | "claim-error" | "csrf") {4} else {1}; + assert_eq!(output.status.code(), Some(expected_exit), "{scenario}: {}", String::from_utf8_lossy(&output.stderr)); + let stderr = String::from_utf8_lossy(&output.stderr); + assert_eq!(stderr.contains(&approval_url), !scenario.starts_with("create-"), "{scenario}: {stderr}"); + assert!(!String::from_utf8_lossy(&output.stdout).contains(&approval_url)); + assert_access_output_has_no_credentials(&output); + assert_access_debug_diagnostics(&output, &server.uri()); + let combined = format!("{}{}", String::from_utf8_lossy(&output.stdout), stderr); + assert!(!combined.contains("claim-code-secret")); + assert!(!combined.contains("issued-secret")); + assert!(!combined.contains("mismatched-secret-code")); + assert!(!combined.contains("secret-envelope-error")); + assert!(!combined.contains("secret-invalid-role")); + let requests = server.received_requests().await.unwrap(); + let filed: Value = serde_json::from_slice(&requests[0].body).unwrap(); + assert!(!combined.contains(filed["state"].as_str().unwrap())); + let claim_requests: Vec<_> = requests.iter().filter(|r| r.url.path().ends_with("/claim")).collect(); + assert_eq!(claim_requests.len(), claims); + for claim in claim_requests { + let body: Value = serde_json::from_slice(&claim.body).unwrap(); + assert_eq!(body.as_object().unwrap().len(), 2); + assert_eq!(body["claimCode"], "claim-code-secret"); + let verifier = body["codeVerifier"].as_str().unwrap(); + assert!(!combined.contains(verifier)); + assert_eq!(base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(Sha256::digest(verifier.as_bytes())), filed["codeChallenge"]); + } + if successful { + if json { + let result: Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(result.as_object().unwrap().len(), 8); + assert_eq!(result["requestId"], "request-1"); + assert_eq!(result["organizationId"], "org"); + assert_eq!(result["credentialsPath"], credentials_path.canonicalize().unwrap().display().to_string()); + assert_eq!(result["requestedRoleIds"], if proposed {serde_json::json!([A, B])} else {serde_json::json!([])}); + if scenario == "claim-grants-many" { + assert_eq!(result["grantedRoleIds"].as_array().unwrap().len(), 101); + } else { assert_eq!(result["grantedRoleIds"], serde_json::json!([B, C])); } + assert_eq!(result["status"], "approved"); + assert_eq!(result["keyId"], "issued-id"); + assert_eq!(result["key"], "issued-key"); + } else { + let stdout = String::from_utf8_lossy(&output.stdout); + assert!(stdout.contains("Organization: org")); + assert!(stdout.contains("Request ID: request-1")); + if proposed { assert!(stdout.contains(&format!("Requested roles: {A}, {B}"))); } + else { assert!(stdout.contains("Requested roles: none (manual selection)")); } + if scenario != "claim-grants-many" { assert!(stdout.contains(&format!("Granted roles: {B}, {C}"))); } + } + let stored: Value = serde_json::from_slice(&std::fs::read(&credentials_path).unwrap()).unwrap(); + assert_eq!(stored["api_key"], "issued-key"); + assert_eq!(stored["api_secret"], "issued-secret"); + } else { + assert!(output.stdout.is_empty()); + assert_eq!(std::fs::read(&credentials_path).unwrap(), before); + } + assert_eq!(std::fs::read(tokens.join("tokens.json")).unwrap(), oauth_before); + assert!(requests.iter().all(|r| !r.url.path().starts_with("/approve"))); + })); + } + } + for task in tasks { + task.await.unwrap(); + } + } + + fn access_command(dir: &Path) -> Command { + let mut command = Command::new(clickhousectl_binary()); + command + .env_clear() + .env("DO_NOT_TRACK", "1") + .env("HOME", dir.join("home")) + .current_dir(dir) + .args(["cloud", "access", "request", "--json", "--debug"]); + command + } + + fn access_credentials(dir: &Path) -> PathBuf { + let project = dir.join(".clickhouse"); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write( + project.join("credentials.json"), + r#"{"api_key":"saved-basic-key","api_secret":"saved-basic-secret"}"#, + ) + .unwrap(); + let tokens = dir.join("home/.clickhouse"); + std::fs::create_dir_all(&tokens).unwrap(); + tokens + } + + fn assert_access_debug_diagnostics(output: &std::process::Output, api_url: &str) { + let stderr = String::from_utf8_lossy(&output.stderr); + assert_eq!( + stderr + .matches("[debug] auth source: OAuth tokens (") + .count(), + 1, + "{stderr}" + ); + assert!(stderr.contains("home/.clickhouse/tokens.json)"), "{stderr}"); + assert_eq!(stderr.matches("[debug] api url:").count(), 1, "{stderr}"); + assert!( + stderr + .lines() + .any(|line| line == format!("[debug] api url: {api_url}/v1")), + "{stderr}" + ); + assert!(!String::from_utf8_lossy(&output.stdout).contains("[debug]")); + } + + fn assert_access_output_has_no_credentials(output: &std::process::Output) { + let text = format!( + "{}{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + for secret in [ + "saved-basic-key", + "saved-basic-secret", + "flag-basic-secret", + "env-basic-secret", + "test-bearer-token", + "refreshed-bearer-token", + "refresh-secret", + ] { + assert!( + !text.contains(secret), + "credential leaked in command output" + ); + } + } + + #[tokio::test] + async fn access_request_perms_files_canonical_proposal_once_without_grant_or_subset() { + let first = "abcdefab-1234-1234-1234-123456789abc"; + let system = "21111111-1111-1111-9999-000000000007"; + for json in [false, true] { + let server = MockServer::start().await; + Mock::given(method("POST")) + .and(path("/v1/organizations/selected/accessRequests")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(ResponseTemplate::new(403).set_body_json( + serde_json::json!({"error": "role selection is not assignable"}), + )) + .expect(1) + .mount(&server) + .await; + let dir = tempfile::tempdir().unwrap(); + let tokens = access_credentials(dir.path()); + write_oauth_tokens(&tokens, &server.uri()); + let credentials_path = dir.path().join(".clickhouse/credentials.json"); + let before = std::fs::read(&credentials_path).unwrap(); + let mut command = Command::new(clickhousectl_binary()); + command + .env_clear() + .env("DO_NOT_TRACK", "1") + .env("HOME", dir.path().join("home")) + .current_dir(dir.path()) + .args([ + "cloud", + "access", + "request", + "--debug", + "--org", + "selected", + "--perms", + &format!("role:{first},role:{system}"), + "--perms", + &format!("role:{}", first.to_uppercase()), + "--key-name", + "proposed-key", + "--purpose", + "bounded proposal", + ]); + if json { + command.arg("--json"); + } + let output = command.output().unwrap(); + assert_eq!(output.status.code(), Some(4)); + assert!(output.stdout.is_empty()); + assert!( + String::from_utf8_lossy(&output.stderr) + .contains("role selection is not assignable") + ); + assert_access_output_has_no_credentials(&output); + assert_eq!(std::fs::read(&credentials_path).unwrap(), before); + let requests = server.received_requests().await.unwrap(); + assert_eq!(requests.len(), 1, "no role catalog lookup or subset retry"); + let body: Value = serde_json::from_slice(&requests[0].body).unwrap(); + assert_eq!(body["requestedRoleIds"], serde_json::json!([first, system])); + assert_eq!(body["keyName"], "proposed-key"); + assert_eq!( + body["agent"], + serde_json::json!({"purpose": "bounded proposal"}) + ); + assert_eq!(body["codeChallengeMethod"], "S256"); + assert_eq!(body["codeChallenge"].as_str().unwrap().len(), 43); + assert!(!body["state"].as_str().unwrap().is_empty()); + assert!( + body["redirectUri"] + .as_str() + .unwrap() + .starts_with("http://127.0.0.1:") + ); + assert!(body["redirectUri"].as_str().unwrap().ends_with("/callback")); + assert_eq!(body.as_object().unwrap().len(), 7); + for field in ["codeChallenge", "state"] { + assert!( + !String::from_utf8_lossy(&output.stderr) + .contains(body[field].as_str().unwrap()) + ); + } + } + } + + #[tokio::test] + async fn access_request_perms_usage_validation_precedes_oauth_and_network() { + let server = MockServer::start().await; + let dir = tempfile::tempdir().unwrap(); + let tokens = access_credentials(dir.path()); + // Expired identity would attempt refresh if usage validation were late. + std::fs::write( + tokens.join("tokens.json"), + serde_json::to_vec(&serde_json::json!({ + "access_token": "test-bearer-token", "refresh_token": "refresh-secret", "expires_at": 1, + "api_url": format!("{}/v1", server.uri()) + })) + .unwrap(), + ) + .unwrap(); + let oversized = (0..101) + .map(|i| format!("role:00000000-0000-0000-0000-{i:012x}")) + .collect::>() + .join(","); + for value in [ + "", + ",", + "role:Admin", + "Role:00000000-0000-0000-0000-000000000001", + "role:00000000-0000-0000-0000-000000000001,", + "role:00000000-0000-0000-0000-000000000001 ,role:*", + &oversized, + ] { + let output = access_command(dir.path()) + .args(["--perms", value]) + .env("CLICKHOUSECTL_AUTH_URL", server.uri()) + .env("CLICKHOUSECTL_CLIENT_ID", "test-client") + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(2), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!(output.stdout.is_empty()); + assert_access_output_has_no_credentials(&output); + } + let output = Command::new(clickhousectl_binary()) + .env_clear() + .env("DO_NOT_TRACK", "1") + .env("HOME", dir.path().join("home")) + .current_dir(dir.path()) + .args([ + "cloud", + "--org-id=one", + "access", + "request", + "--org=two", + "--perms", + "role:00000000-0000-0000-0000-000000000001", + ]) + .output() + .unwrap(); + assert_eq!(output.status.code(), Some(2)); + assert!(server.received_requests().await.unwrap().is_empty()); + } + + /// A 3xx from the claim endpoint must not forward the claim POST's body — + /// `claimCode` and the secret PKCE verifier — to the redirect target. + #[tokio::test] + async fn access_request_claim_never_follows_redirects() { + use base64::Engine as _; + use sha2::{Digest as _, Sha256}; + for status in [301, 302, 303, 307, 308] { + let server = MockServer::start().await; + let destination = MockServer::start().await; + let dir = tempfile::tempdir().unwrap(); + let tokens = access_credentials(dir.path()); + write_oauth_tokens(&tokens, &server.uri()); + let credentials_path = dir.path().join(".clickhouse/credentials.json"); + let before = std::fs::read(&credentials_path).unwrap(); + let oauth_before = std::fs::read(tokens.join("tokens.json")).unwrap(); + Mock::given(method("POST")) + .and(path("/v1/organizations/org/accessRequests")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"result": { + "requestedRoleIds": [], "requestId": "request-1", "status": "pending", + "approvalUrl": format!("{}/approve/request-1", server.uri()), "expiresAt": 4102444800000_i64, + }}))) + .expect(1) + .mount(&server) + .await; + Mock::given(method("GET")) + .and(path("/v1/organizations/org/accessRequests/request-1")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"result": { + "requestedRoleIds": [], "grantedRoleIds": ["aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"], + "requestId": "request-1", "expiresAt": 4102444800000_i64, + "claimCodeExpiresAt": 4102444800000_i64, "status": "approved", "claimCode": "claim-code-secret", + }}))) + .expect(1) + .mount(&server) + .await; + Mock::given(method("POST")) + .and(path("/v1/organizations/org/accessRequests/request-1/claim")) + .respond_with( + ResponseTemplate::new(status) + .insert_header("Location", format!("{}/stolen", destination.uri())), + ) + .expect(1) + .mount(&server) + .await; + let output = access_command(dir.path()) + .args(["--org", "org", "--no-browser"]) + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(1), + "{status}: {}", + String::from_utf8_lossy(&output.stderr) + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(!stderr.contains("claim-code-secret")); + assert!(!stderr.contains("issued-secret")); + assert_access_output_has_no_credentials(&output); + assert!(output.stdout.is_empty()); + assert_eq!(std::fs::read(&credentials_path).unwrap(), before); + assert_eq!( + std::fs::read(tokens.join("tokens.json")).unwrap(), + oauth_before + ); + let requests = server.received_requests().await.unwrap(); + let filed: Value = serde_json::from_slice(&requests[0].body).unwrap(); + let claim_requests: Vec<_> = requests + .iter() + .filter(|request| request.url.path().ends_with("/claim")) + .collect(); + assert_eq!(claim_requests.len(), 1); + let body: Value = serde_json::from_slice(&claim_requests[0].body).unwrap(); + assert_eq!(body.as_object().unwrap().len(), 2); + assert_eq!(body["claimCode"], "claim-code-secret"); + let verifier = body["codeVerifier"].as_str().unwrap(); + assert_eq!( + base64::engine::general_purpose::URL_SAFE_NO_PAD + .encode(Sha256::digest(verifier.as_bytes())), + filed["codeChallenge"] + ); + assert!(destination.received_requests().await.unwrap().is_empty()); + assert!( + requests + .iter() + .all(|request| !request.url.path().starts_with("/approve")) + ); + } + } + + /// Discovery runs on the typed client directly, outside the handshake's + /// `operation` wrapper, so `for_access` must hand the client the 30-second + /// access-operation bound itself: a server holding the response open + /// fails the command near the bound instead of stalling until the mock's + /// delay elapses. The mock delays longer than the bound; without the + /// client timeout the command would only finish after it. + #[tokio::test] + async fn access_roles_discovery_gives_up_within_the_operation_bound() { + let started = std::time::Instant::now(); + let server = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/v1/organizations/org/roles")) + .respond_with( + ResponseTemplate::new(200) + .set_body_json(serde_json::json!([])) + .set_delay(std::time::Duration::from_secs(90)), + ) + .mount(&server) + .await; + let dir = tempfile::tempdir().unwrap(); + let tokens = access_credentials(dir.path()); + write_oauth_tokens(&tokens, &server.uri()); + let output = access_roles_command(dir.path(), true) + .args(["--org", "org"]) + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(1), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!( + started.elapsed() < std::time::Duration::from_secs(60), + "discovery must give up within the operation bound, not wait out the mock" + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(!stderr.contains("test-bearer-token")); + assert_access_output_has_no_credentials(&output); + } + + #[tokio::test] + async fn access_request_uses_oauth_for_discovery_and_filing_with_saved_or_explicit_url() { + for explicit_url in [false, true] { + let selected = MockServer::start().await; + let saved = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/v1/organizations")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "result": [{"id": "11111111-2222-3333-4444-555555555555", "name": "Only org"}] + }))) + .expect(1) + .mount(&selected) + .await; + // Stop before browser launch, while exercising the real PKCE POST. + Mock::given(method("POST")) + .and(path( + "/v1/organizations/11111111-2222-3333-4444-555555555555/accessRequests", + )) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(ResponseTemplate::new(403).set_body_json(serde_json::json!({ + "error": "fixture rejection" + }))) + .expect(1) + .mount(&selected) + .await; + let dir = tempfile::tempdir().unwrap(); + let token_dir = access_credentials(dir.path()); + write_oauth_tokens( + &token_dir, + &if explicit_url { + saved.uri() + } else { + selected.uri() + }, + ); + let mut command = access_command(dir.path()); + // Even explicit/environment Basic credentials cannot change the user lane. + command + .args(["--api-key", "flag-key", "--api-secret", "flag-basic-secret"]) + .env("CLICKHOUSE_CLOUD_API_KEY", "env-key") + .env("CLICKHOUSE_CLOUD_API_SECRET", "env-basic-secret"); + if explicit_url { + command.args(["--url", &format!("{}/v1/", selected.uri())]); + } + let output = command.output().unwrap(); + assert_eq!( + output.status.code(), + Some(4), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert_access_output_has_no_credentials(&output); + assert_access_debug_diagnostics(&output, &selected.uri()); + assert!(saved.received_requests().await.unwrap().is_empty()); + let requests = selected.received_requests().await.unwrap(); + assert_eq!(requests.len(), 2); + let body: Value = serde_json::from_slice(&requests[1].body).unwrap(); + assert_eq!(body["codeChallengeMethod"], "S256"); + assert_eq!(body["codeChallenge"].as_str().unwrap().len(), 43); + assert!(!body["state"].as_str().unwrap().is_empty()); + assert!( + body["redirectUri"] + .as_str() + .unwrap() + .starts_with("http://127.0.0.1:") + ); + assert!(body.get("codeVerifier").is_none()); + assert_eq!(body.as_object().unwrap().len(), 4); + let credentials = + std::fs::read_to_string(dir.path().join(".clickhouse/credentials.json")).unwrap(); + assert!(credentials.contains("saved-basic-secret")); + assert!(token_dir.join("tokens.json").exists()); + } + } + + #[tokio::test] + async fn access_request_missing_or_unrefreshable_identity_never_falls_back_to_basic() { + for token_state in ["missing", "no-refresh", "unknown-host", "rejected"] { + let server = MockServer::start().await; + let dir = tempfile::tempdir().unwrap(); + let token_dir = access_credentials(dir.path()); + if token_state != "missing" { + // `auth_config` is required for the record to load at all. The + // saved config only drives refresh for the states that must + // reach it; `unknown-host` keeps a legacy record without one, + // which no environment can repair, so the file must survive. + let mut record = serde_json::json!({ + "access_token": "test-bearer-token", + "refresh_token": if token_state == "no-refresh" { "" } else { "refresh-secret" }, + "expires_at": 1, + "api_url": format!("{}/v1", server.uri()), + }); + if token_state != "unknown-host" { + record["auth_config"] = + serde_json::json!({"auth_url": server.uri(), "client_id": "test-client"}); + } + std::fs::write( + token_dir.join("tokens.json"), + serde_json::to_vec(&record).unwrap(), + ) + .unwrap(); + } + let mut command = access_command(dir.path()); + command.args([ + "--url", + &server.uri(), + "--org-id", + "11111111-2222-3333-4444-555555555555", + ]); + if token_state == "rejected" { + Mock::given(method("POST")) + .and(path("/oauth/token")) + // Even an identity provider echoing a secret must not leak it. + .respond_with(ResponseTemplate::new(401).set_body_string("refresh-secret")) + .expect(1) + .mount(&server) + .await; + command + .env("CLICKHOUSECTL_AUTH_URL", server.uri()) + .env("CLICKHOUSECTL_CLIENT_ID", "test-client"); + } + let output = command.output().unwrap(); + assert_eq!(output.status.code(), Some(4), "{token_state}"); + let error: Value = serde_json::from_slice(&output.stderr).unwrap(); + assert!( + error["error"]["message"] + .as_str() + .unwrap() + .contains("cloud auth login") + ); + assert_access_output_has_no_credentials(&output); + let requests = server.received_requests().await.unwrap(); + assert_eq!(requests.len(), usize::from(token_state == "rejected")); + assert!(dir.path().join(".clickhouse/credentials.json").exists()); + assert_eq!( + token_dir.join("tokens.json").exists(), + token_state == "unknown-host" + ); + } + } + + #[tokio::test] + async fn access_request_refreshes_oauth_despite_stored_basic_and_preserves_login_environment() { + let issuer = MockServer::start().await; + let target = MockServer::start().await; + Mock::given(method("POST")) + .and(path("/oauth/token")) + .and(header("content-type", "application/x-www-form-urlencoded")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "access_token": "refreshed-bearer-token", "expires_in": 3600, "token_type": "Bearer" + }))) + .expect(1) + .mount(&issuer) + .await; + Mock::given(method("POST")) + .and(path( + "/v1/organizations/11111111-2222-3333-4444-555555555555/accessRequests", + )) + .and(header("authorization", "Bearer refreshed-bearer-token")) + .respond_with(ResponseTemplate::new(403)) + .expect(1) + .mount(&target) + .await; + let dir = tempfile::tempdir().unwrap(); + let token_dir = access_credentials(dir.path()); + std::fs::write( + token_dir.join("tokens.json"), + serde_json::to_vec(&serde_json::json!({ + "access_token": "test-bearer-token", "refresh_token": "refresh-secret", + "expires_at": 1, "api_url": format!("{}/v1", issuer.uri()), + "auth_config": {"auth_url": issuer.uri(), "client_id": "test-client"}, + })) + .unwrap(), + ) + .unwrap(); + let output = access_command(dir.path()) + .args([ + "--url", + &target.uri(), + "--org-id", + "11111111-2222-3333-4444-555555555555", + ]) + .env("CLICKHOUSECTL_AUTH_URL", format!("{}/", issuer.uri())) + .env("CLICKHOUSECTL_CLIENT_ID", "test-client") + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(4), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert_access_output_has_no_credentials(&output); + let requests = issuer.received_requests().await.unwrap(); + assert_eq!(requests.len(), 1); + let form: std::collections::HashMap<_, _> = url::form_urlencoded::parse(&requests[0].body) + .into_owned() + .collect(); + assert_eq!(form["grant_type"], "refresh_token"); + assert_eq!(form["client_id"], "test-client"); + assert_eq!(form["refresh_token"], "refresh-secret"); + let tokens: Value = + serde_json::from_slice(&std::fs::read(token_dir.join("tokens.json")).unwrap()).unwrap(); + assert_eq!(tokens["access_token"], "refreshed-bearer-token"); + assert_eq!(tokens["refresh_token"], "refresh-secret"); + assert_eq!(tokens["api_url"], format!("{}/v1", issuer.uri())); + assert!(tokens["expires_at"].as_i64().unwrap() > chrono::Utc::now().timestamp() + 60); + } + + #[tokio::test] + async fn access_organization_resolution_reads_complete_user_memberships() { + const ONE: &str = "11111111-2222-3333-4444-555555555555"; + const TWO: &str = "22222222-2222-3333-4444-555555555555"; + for scenario in [ + "zero", + "one", + "multi", + "paged-single", + "paged-multi", + "named", + "case-mismatch", + "ambiguous-name", + "repeat", + "no-progress", + "page-error", + "missing-result", + "missing-id", + "explicit", + ] { + let server = MockServer::start().await; + let first_rows = match scenario { + "zero" | "explicit" => serde_json::json!([]), + "multi" => { + serde_json::json!([{"id": ONE, "name": "First"}, {"id": TWO, "name": "Second"}]) + } + _ => serde_json::json!([{"id": ONE, "name": "First"}]), + }; + let paged = [ + "paged-single", + "paged-multi", + "named", + "case-mismatch", + "ambiguous-name", + "repeat", + "no-progress", + "page-error", + "missing-result", + "missing-id", + ] + .contains(&scenario); + let mut first = serde_json::json!({"result": first_rows}); + if paged { + first["nextCursor"] = "opaque+/= &?".into(); + } + Mock::given(method("GET")) + .and(path("/v1/organizations")) + .and(header("authorization", "Bearer test-bearer-token")) + .and(|request: &wiremock::Request| request.url.query().is_none()) + .respond_with(ResponseTemplate::new(200).set_body_json(first)) + .expect(if scenario == "explicit" { 0 } else { 1 }) + .mount(&server) + .await; + if paged { + let second = match scenario { + "paged-single" => serde_json::json!({"result": [], "nextCursor": null}), + "repeat" => serde_json::json!({"result": [], "nextCursor": "opaque+/= &?"}), + "no-progress" => { + serde_json::json!({"result": [{"id": ONE}], "nextCursor": "different"}) + } + "missing-result" => serde_json::json!({"nextCursor": null}), + "missing-id" => serde_json::json!({"result": [{"name": "Second"}]}), + "ambiguous-name" => { + serde_json::json!({"result": [{"id": TWO, "name": "First"}]}) + } + _ => serde_json::json!({"result": [{"id": TWO, "name": "Second"}]}), + }; + Mock::given(method("GET")) + .and(path("/v1/organizations")) + .and(query_param("cursor", "opaque+/= &?")) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with( + ResponseTemplate::new(if scenario == "page-error" { 500 } else { 200 }) + .set_body_json(second), + ) + .expect(1) + .mount(&server) + .await; + } + let files = ["one", "paged-single", "named", "explicit"].contains(&scenario); + let selected = if ["named", "explicit"].contains(&scenario) { + TWO + } else { + ONE + }; + Mock::given(method("POST")) + .and(path(format!("/v1/organizations/{selected}/accessRequests"))) + .and(header("authorization", "Bearer test-bearer-token")) + .respond_with(ResponseTemplate::new(403)) + .expect(if files { 1 } else { 0 }) + .mount(&server) + .await; + let dir = tempfile::tempdir().unwrap(); + let token_dir = access_credentials(dir.path()); + write_oauth_tokens(&token_dir, &server.uri()); + let mut command = access_command(dir.path()); + match scenario { + "named" => { + command.args(["--org-name", "Second"]); + } + "case-mismatch" => { + command.args(["--org-name", "second"]); + } + "ambiguous-name" => { + command.args(["--org-name", "First"]); + } + "explicit" => { + command.args(["--org", TWO]); + } + _ => {} + } + let output = command.output().unwrap(); + assert_eq!( + output.status.code(), + Some(if files { 4 } else { 1 }), + "{scenario}: {}", + String::from_utf8_lossy(&output.stderr) + ); + assert_access_output_has_no_credentials(&output); + if ["zero", "multi", "paged-multi"].contains(&scenario) { + assert!(String::from_utf8_lossy(&output.stderr).contains("cloud org list --json")); + } + let requests = server.received_requests().await.unwrap(); + assert_eq!( + requests.len(), + usize::from(scenario != "explicit") + usize::from(paged) + usize::from(files), + "{scenario}" + ); + } + } + + #[test] + fn access_organization_conflicts_fail_usage_before_authentication() { + let dir = tempfile::tempdir().unwrap(); + for args in [ + vec![ + "cloud", + "--org-id=first", + "access", + "request", + "--org-id=second", + ], + vec![ + "cloud", + "--org-id=first", + "access", + "request", + "--org=second", + ], + vec![ + "cloud", + "access", + "--org=first", + "request", + "--org-id=second", + ], + vec![ + "cloud", + "--org-name=first", + "access", + "request", + "--org-name=second", + ], + vec![ + "cloud", + "--org-name=same", + "access", + "request", + "--org=same", + ], + ] { + let output = Command::new(clickhousectl_binary()) + .env_clear() + .env("DO_NOT_TRACK", "1") + .env("HOME", dir.path()) + .current_dir(dir.path()) + .args(args) + .output() + .unwrap(); + assert_eq!( + output.status.code(), + Some(2), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!(output.stdout.is_empty()); + } + } +} diff --git a/docs/reference/authentication.md b/docs/reference/authentication.md index bfc02e4f..273c2663 100644 --- a/docs/reference/authentication.md +++ b/docs/reference/authentication.md @@ -37,6 +37,8 @@ The first applicable source wins: Supplying only one credential flag blocks fallback. Cloud commands skip OAuth token refresh when a higher-priority API key source wins, leaving saved OAuth tokens unchanged. When OAuth is selected, expired or near-expiry tokens are refreshed before the API request. `auth status` identifies inactive sources; `--debug` on a Cloud command prints the selected credential source and API URL to stderr. +`cloud access roles --debug` and `cloud access request --debug` report their OAuth credential source and resolved API URL on stderr, including with `--json`. These experimental commands use OAuth even when higher-priority API keys are configured; the diagnostics describe the identity actually used for access negotiation. + `cloud postgres query` accepts **OAuth only**, unlike `cloud service query`. Remove higher-priority API key sources before using that route. Clearing a saved key file alone is insufficient if exported variables or `.env` still supply a key. ## Select an organization and resource