Skip to content

test(cli): apply the one-layer rule to Cloud services wiremock tests (#1175) - #1189

Open
sdairs wants to merge 9 commits into
claude/1175-wiremock-clickpipesfrom
claude/1175-wiremock-services
Open

sdairs wants to merge 9 commits into
claude/1175-wiremock-clickpipesfrom
claude/1175-wiremock-services

Conversation

@sdairs

@sdairs sdairs commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Part of #1175.

Summary

Applies the one-layer rule (#1187) to the Cloud services tests in tests/cli_request_shape_test.rs. That covers service CRUD and lifecycle, settings, tags, private endpoints, backups, scaling, upgrade windows, service query, Query API endpoints and saved queries. The binary goes from 527 to 472 tests, about −2,400 lines.

  • Validation (25 deleted, 4 trimmed). These subprocess tests only showed clap, post-parse validation or a builder rejecting bad input. The largest was the 30-run settings inexact-number loop. Before deleting, new unit tests were added:

    • settings number boundaries and inexact numbers
    • malformed settings documents and assignments
    • unreadable settings and config files
    • saved-query file I/O

    One subprocess run is kept where only the binary shows the behaviour: settings are parsed before organization discovery, and the selector forms.

  • Body shape (14 deleted, 2 cut to one run). Each deleted field is set by a builder or passed through a destructured dispatcher binding.

  • Shared behaviour (17 deleted). These were per-command repeats of convert_error, the 401/403 → exit 4 mapping, by-id not-found and the OAuth write check. The last end-to-end copy of each still exists, for example dispatched_cloud_401_*, deleted_resource_404_* and udf_every_write_rejects_oauth_before_api.

  • Output owned by unit tests (6 deleted).

  • Repeated runs and never-fail assertions:

    • The --file alias loops and single-mode halves are cut.
    • The reset-password test now runs in an isolated environment. Agent detection could previously force JSON in both of its "modes". It now asserts exit 1 rather than !success().
    • A contains("-") table check now checks the parsed rows instead.

Verifier. An independent verifier mutation-tested 35 handler and dispatcher sites. It restored or added coverage where a deletion, or a gap that predates this PR, let a mutation survive:

  • --replace-open-api-keys with an existing endpoint, a read-modify-write. The deleted test is restored.
  • Reset-password --new-password-hash and --new-double-sha1-hash reaching the wire.
  • service settings get in human mode.
  • A new service scale wiring test. The family had none before.
  • service create --tag and --enable-endpoint, and private-endpoint create --description, now folded into existing exact-body wiring tests.

Known gaps, not addressed here:

  • list_services_filtered losing its org scope on errors is not tested; only the unfiltered list has an error test.
  • Scaling-schedule get in human mode is not tested.
  • service private-endpoint get-config has no wiring test.
  • The stored-key a_* and first_* tests each take about 1.1s and look like they hit a fixed wait. That would be a follow-up for Cut local test run time: six tests wait out real timeouts (~70s of ~2m) #1174.

Test plan

  • cargo fmt --all --check, both clippy configurations, cargo check -p clickhousectl --no-default-features
  • cargo test -p clickhousectl
  • Author and verifier mutation checks: 51 mutations in total. Each surviving mutation was either fixed or is listed above.

🤖 Generated with Claude Code


Note

Low Risk
Test-only refactor with no production code changes; risk is reduced regression coverage in edge cases called out in the PR description.

Overview
Refactors clickhousectl Cloud services coverage by moving parse/validation and builder logic out of tests/cli_request_shape_test.rs into unit tests, and trimming the wiremock suite to HTTP wiring, auth routing, and behaviors only the full binary can show (~527 → 472 tests, ~−2,400 lines).

New unit tests cover shared input paths: read_config_value (missing/malformed JSON), read_sql (unreadable file → I/O failure), and services::settings (integer boundaries, inexact numbers, malformed documents/assignments, unreadable settings files).

Integration tests removed or shortened where clap, builders, or shared error mapping already had coverage—backup bucket validation loops, backup-config edge cases, duplicate --file alias runs, query-endpoint permutations, OAuth-before-HTTP repeats, API error classification duplicates, and large settings subprocess loops. Comments point reviewers to the unit tests or clap contracts instead.

Restored or added wiremock tests where mutation review found gaps: service scale PATCH to replicaScaling, service settings get human output, reset-password hash fields on the wire, query-endpoint --replace-open-api-keys**, and richer **BYOC create** / **private-endpoint** bodies. A few subprocess checks remain (e.g. settings fail **before org discovery**, selector forms for **service query`**).

Reviewed by Cursor Bugbot for commit 2e01803. Bugbot is set up for automated code reviews on this repo. Configure here.

sdairs and others added 9 commits November 2, 2026 11:54
…1175)

Delete the Cloud service subprocess tests whose only assertion is that clap,
post-parse validation or a request builder rejects bad input before HTTP.
Each one has a unit test asserting the same rejection.

The service-settings numeric boundaries, malformed settings inputs and
unreadable settings files had no unit test, nor did read_config_value's
file errors or saved-query read_sql; add those first. One subprocess run
stays for each handler ordering that only the binary can show: settings
parse before organization discovery, and the three service query selector
forms reaching the query host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#1175)

Remove the wiremock tests that only re-check request fields asserted by the
build_*_request unit tests: AWS backup bucket external IDs, clearing the
backup start time alongside a period, query-endpoint origin and key merging,
IP allowlist descriptions, settings type and integer preservation, the
backup encryption config file, service profile query omission, and the
reset-password generation modes.

Every deleted field is set by a builder or passed through a destructured
dispatcher binding, never assembled in the handler. One wiring test per
handler still puts the builder body on the wire. Snapshot config and replica
routing update keep one exact-body run each; the routing run uses opposite
values so a swapped pass-through still fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1175)

Remove the service-family wiremock tests that re-check behaviour one shared
function owns: convert_error's message, organization scope and exit-code
classification (including the 401/403 to exit 4 mapping), the by-id
not-found refinement, and the OAuth write check. They covered backup
buckets, service get/list/delete/wake, scaling schedules, settings, upgrade
windows, Query API endpoints, saved queries, replica-aware routing and
service profiles.

The client.rs and cloud/mod.rs unit tests own these rules. One end-to-end
copy of each stays elsewhere in the file for the later consolidation:
dispatched_cloud_401/403/500, deleted_resource_404_wrong_organization_scope,
by_id_read_*, org_get_unknown_well_formed_id_reads_as_not_found,
udf_every_write_rejects_oauth_before_api and
key_update_expiry_rejects_oauth_before_http.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove wiremock tests that only pin rendering or message content already
asserted at the unit layer:
- the upgrade-window sparse view (upgrade_window_human_view tests)
- settings get and schema in human mode (generic print_human tests in
  cloud/output.rs; settings read wiring stays in the OAuth routes test)
- the Query API endpoint delete confirmation sentence
- the two query gateway-timeout cases without a native endpoint
  (query_timeout_error_falls_back_to_a_host_placeholder)

The gateway-timeout tests that read the endpoint from the service response
and check for no retry stay, in both human and JSON mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cut runs that repeat a dimension the result does not depend on:
- the agent-detected `service wake` JSON test (agent detection keeps its
  end-to-end tests in cloud_runtime_errors_* and service query)
- the --file alias loops for backup buckets and Query API endpoints (the
  aliases are pinned by assert_primary_json_input)
- start/stop human output, from 6 runs to 2
- the query-endpoint first-create origins, from 2 runs to 1
- the JSON half of settings list and the human half of snapshot reads

Fix assertions that could not fail:
- reset-password ran without an isolated environment, so agent detection
  could force JSON in both of its "modes", and it checked
  `!status.success()`. It now runs isolated and asserts exit 1 with
  empty stdout.
- the Query API endpoint human list asserted `stdout.contains("-")`, which
  table borders always satisfy. It now checks the parsed rows.
- snapshot reads asserted only non-empty human stdout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hash wiring (#1175)

Two deletions in this series lost handler coverage that no unit test
reaches:

- query_endpoint_create_key_replacement_is_explicit is the only run where
  --replace-open-api-keys meets an existing endpoint in the
  read-modify-write. Dropping the flag in the dispatcher passed every
  remaining test. Restored as it was.
- No remaining test sent --new-password-hash or --new-double-sha1-hash,
  so swapping the two in the dispatcher passed. The new test sends both,
  asserts the exact PATCH body, and checks that a hash reset succeeds
  without a password in the response.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`cloud service scale` had no subprocess test, so building the request
from default options, swapping the memory flags in the dispatcher, or
sending the service id as the organization all passed. The new test
sends every vertical-mode flag with distinct values. It asserts the
exact PATCH body on the replicaScaling route with API-key auth, and the
JSON stdout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deleting service_settings_get_preserves_json_types_and_renders_human_values
left no human-mode run of `service settings get`. A handler that printed
JSON in human mode passed every test. One human-mode run, in an isolated
environment so agent detection cannot force JSON, asserts stdout is not
JSON and that the name and value appear as rendered lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ns in wiring tests (#1175)

Three dispatcher pass-throughs had no test that sent the flag:

- `service create --tag` and `--enable-endpoint`: the dynamic BYOC create
  test's exact body now includes both. Only an enable is sent, so a
  swap with --disable-endpoint changes the body.
- `service private-endpoint create --description`: the wiring test now
  sends it and asserts the exact body rather than the id alone.

Dropping --tag, swapping the endpoint toggles, or dropping
--description in the dispatcher now fails these tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sdairs
sdairs requested a review from iskakaushik as a code owner November 2, 2026 12:40
@sdairs
sdairs added this pull request to stack #1153 November 2, 2026 12:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant