feat(onboarding): verify Test connection without a model id (#792)

[lifelmy]

Problem

Fixes #792.

In Settings → Model services, the Test connection button calls the onboarding.provider.probe RPC, whose underlying probe_llm_provider() hard-requires a model id (an empty model raises ValueError). That is a chicken-and-egg bind for a new user: you cannot know which models a provider serves until you can list them, but the model list is only fetched after the connection verifies — and the connection cannot be verified without already supplying a valid model id.

Change

When the model field is empty, connectivity is now verified through the model-list endpoint (GET /v1/models via the existing discover_provider_models()) instead of a one-token chat completion — exactly the behavior the issue proposes.

• Backend (gateway/rpc_onboarding.py): onboarding.provider.probe branches on the model field. With a model it runs the usual usage-accounted chat probe; with an empty model it calls a new _connectivity_probe_via_discovery() helper that runs discover_provider_models() and maps the listing outcome (ok / failureKind / detail) back onto the frozen probe envelope. The chat-only timing fields (latencyMs, firstResponseMs, totalMs) stay at their never-reached-the-network sentinels since no chat round-trip happened. An ok listing that simply returns nothing still counts as reachable (the credentials verified). The wire contract is unchanged.
• Frontend (useSetupCatalog.ts): for a draft primary-provider config, an empty model is no longer treated as a missing required field, so the Test connection button is enabled and probes with no model param. Stored/draft profile probes go through onboarding.llmProfile[.draft].probe, which still resolves a concrete deployment model, so those keep requiring a model.

Testing (all run locally, real execution)

• uv run ruff check src tests → all checks passed
• uv run pytest -q tests/test_gateway/test_rpc_onboarding.py tests/test_contracts tests/test_onboarding → 1196 passed, 4 skipped
  • new: test_provider_probe_without_model_verifies_via_model_list, test_provider_probe_without_model_reports_auth_failure
• uv run pytest -q tests/test_cli/test_models_probe.py → 12 passed; uv run pytest -q tests/test_onboarding/test_provider_probe.py → passed
• uv run mypy src/opensquilla/gateway/rpc_onboarding.py → no issues
• npx vitest run on useSetupCatalog.privacy.test.ts + useSetupProviderForm.test.ts (232 passed) and SetupProviderPanel.test.ts + SetupProviderCredentialCard.test.ts (123 passed); updated the privacy suite's model-less probe expectations
• npm run build (Web UI artifact verified + staged), npm run typecheck (all guards passed), uv build --wheel → built opensquilla-0.5.4-py3-none-any.whl

Notes

I use a coding assistant to help implement, and I review and take responsibility for the final change. Opened as a draft for maintainer review; happy to adjust the surface (e.g. keep the button enabled but relabel it, or gate the fallback behind a capability flag) if you prefer a different shape.

[lifelmy]

Current follow-up on head 64f14dcdfa after refreshing upstream/main (f9467234b5; branch is 0 behind / 6 ahead): the PR remains mergeable, and the implementation-specific suites pass locally.

Verification:

• .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py → 72 passed
• cd opensquilla-webui && bun run test:unit -- src/composables/setup/useSetupCatalog.privacy.test.ts → 175 passed

The remaining failed GitHub check is unrelated to this PR's onboarding diff: Windows high-risk (core) completed 6,947 tests successfully and then tests/test_process_tree.py::test_windows_gateway_crash_kills_job_and_reconcile_removes_stale_row timed out at its 15-second worker.wait() limit. CI result only mirrors that failure. I attempted gh run rerun 33724721382 --failed again, but GitHub rejected it because this contributor account lacks repository admin rights.

A maintainer/admin rerun of failed jobs for run 33724721382 is still needed to clear the transient Windows gate.

[lifelmy]

Refreshed this PR onto the latest upstream/main (3f92428e); current head is 31118a7d. The merge completed without manual conflict edits and preserved the model-id-free connection verification paths.

Verification on the refreshed head:

• .venv/bin/pytest -q tests/test_onboarding tests/test_github_issue_link_sync.py -> 1001 passed, 4 skipped
• npm --prefix opensquilla-webui run test:unit -- src/composables/setup/useSetupCatalog.privacy.test.ts -> 175 passed
• focused ruff check -> passed
• git diff --check upstream/main...HEAD -> passed

The push superseded the prior CI run containing the unrelated Windows core timeout. GitHub reports the refreshed branch as mergeable; its new CI jobs are running.

[lifelmy]

Refreshed this PR onto the latest upstream/main (4535029); new head is 33de521. The branch was 1 commit behind and merged cleanly without manual conflict edits. Verification: backend onboarding/RPC/issue-link suites passed (1073 passed, 4 skipped); WebUI privacy suite passed (175 passed); focused ruff and git diff --check passed.

[lifelmy]

Task A CI follow-up (2026-09-05): the completed Windows recovery-migration shard had one failure in test_aggregator_partial_output_idle_timeout_is_replaced_by_fixed_aggregator[tool], outside this PR's onboarding/WebUI diff; the other 4,251 tests in that phase passed. I re-ran the exact failing test 5 times locally and all 5 passed. The PR-focused backend run passed 75 tests (including onboarding and the failing ensemble test), WebUI privacy passed 175 tests, focused ruff and git diff --check passed. I attempted to rerun failed GitHub jobs, but GitHub requires repository admin rights for this run, so the failed check remains maintainer-controlled.

[lifelmy]

Task A refresh (2026-09-05): fetched latest upstream/main at 770ff387c6, merged it without conflicts, and pushed head cfad87c0d7. The two new base commits touch provider ensemble/runtime and WebUI toast-host paths; this PR's model-less Test connection behavior remains intact.

Verification on the refreshed head:

• uv run pytest -q tests/test_gateway/test_rpc_onboarding.py tests/test_onboarding/test_flow_provider_verify.py -> 96 passed
• npm --prefix opensquilla-webui run test:unit -- src/composables/setup/useSetupCatalog.privacy.test.ts -> 175 passed
• focused uv run ruff check -> passed
• git diff --check upstream/main...HEAD -> passed

No new comments/reviews or unresolved review threads.

[lifelmy]

Refreshed this branch onto latest upstream/main (1663b1a); new head is e3c1716. The one-commit base update merged without conflicts, preserving the onboarding Test connection behavior. Verification: backend onboarding suite 72 passed; WebUI privacy Vitest suite 175 passed; focused Ruff and git diff --check passed.

[lifelmy]

Task A refresh: merged latest upstream/main (846bbc6b) without conflicts; new head is d7a086ae8. The model-id-free Test connection behavior remains covered. Verification: full onboarding tests passed (984 passed, 4 skipped; one existing deprecation warning), focused Ruff passed, WebUI privacy Vitest passed (175 tests), and git diff --check upstream/main...HEAD passed.

[lifelmy]

Task A refresh: merged latest upstream/main (085bfed00d) without conflicts; new head is 7416f76f13. The upstream change consolidates settings and durable turn admission, and the model-id-free Test connection behavior remains intact. Verification: backend onboarding/provider verification tests passed (96), focused Ruff passed, WebUI privacy Vitest passed (175), and git diff --check upstream/main...HEAD passed.

[lifelmy]

Refreshed onto current upstream/main (cfa8e4b); merge completed without conflicts and preserved the onboarding change. New head: b871d7e. Verification: .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py (72 passed); npm --prefix opensquilla-webui run test:unit -- src/composables/setup/useSetupCatalog.privacy.test.ts (175 passed); git diff --check refs/task-a/upstream/main...HEAD (clean). Fresh CI is running.

[lifelmy]

CI follow-up: the Ubuntu Desktop recovery profiles job failed in an unrelated profile-consolidation Electron shutdown timeout after its onboarding flow passed. This PR does not touch desktop/electron recovery or shutdown files, and the same CI workflow passed on the freshly merged base cfa8e4b (run 34069076410). I requested a rerun of the failed job; remaining checks are still in progress.

[lifelmy]

Task A refresh: merged latest upstream/main (bf7fa0e1e) without conflicts; new head is 924b79adb. The upstream progressive-tool-disclosure change does not overlap the model-less onboarding connection flow. Verification: .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py (72 passed), WebUI privacy Vitest (175 passed), focused Ruff passed, and git diff --check upstream/main...HEAD passed.

[lifelmy]

Task A refresh: merged latest upstream/main (05a6b8619) into the branch without conflicts; new head is cb157be11ea3c2c6ff3a4f2b28b44bac64c7c79d. The branch was 1 commit behind; the merge preserved the model-id-free Test-connection onboarding change.

Verification: .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py → 72 passed; git diff --check upstream/main clean. No new review feedback; unresolved threads = 0.

[lifelmy]

Task A refresh: merged latest upstream/main (f7dd1b6f6) into the branch without conflicts; new head is a365363f0. Re-ran pytest tests/test_gateway/test_rpc_onboarding.py = 72 passed. PR remains MERGEABLE/CLEAN.

[lifelmy]

Task A refresh: PR had gone BEHIND after upstream/main advanced to e4eb36b2d. Merged upstream/main into agent-tasks/792 without conflicts; new head is bb5706d81. Upstream changes (#1589 experiment control-plane removal, #1590 chat image preview) don't overlap the onboarding RPC change. Re-ran pytest tests/test_gateway/test_rpc_onboarding.py → 72 passed. mergeable=MERGEABLE.

[lifelmy]

Task A refresh: merged latest upstream/main (fc8cce1ca) into agent-tasks/792 without conflicts; new head is 1da9c14d4 (0 behind / 22 ahead). The upstream contract/skill-management update did not alter this PR's model-id-free onboarding behavior. Verification: .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py (72 passed); npm --prefix opensquilla-webui run test:unit -- src/composables/setup/useSetupCatalog.privacy.test.ts (175 passed); focused Ruff and git diff --check upstream/main...HEAD passed.

[lifelmy]

Task A refresh: merged latest upstream/main (021063b) without conflicts; new head is 115153d (0 behind). The two upstream commits remove retired runtime steering and isolate router cards, and do not overlap this PR’s model-id-free onboarding connection flow. Verification: .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py (72 passed); focused Ruff and git diff --check passed; WebUI privacy Vitest passed (175 passed). The first push hit a transient GitHub commit_refs error; a retry succeeded and the remote SHA was rechecked.

[lifelmy]

Task A refresh: fetched and merged latest upstream/main (9486415) without conflicts; new head is 057786a (0 behind / 24 ahead). The upstream change overlaps onboarding RPC/tests but merged cleanly; this PR’s model-id-free Test connection behavior remains covered. Verification: .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py (72 passed); focused Ruff and git diff --check passed; from opensquilla-webui, npm run test:unit -- src/composables/setup/useSetupCatalog.privacy.test.ts (175 passed). Note: an initial root-level direct vitest invocation failed before test collection because it bypassed the WebUI project alias config; rerunning via the package script passed.

[lifelmy]

Task A refresh: fetched and merged current upstream/main (829e11a4a) without conflicts; new head is cc03e40ae (0 behind / 25 ahead). Upstream includes an unrelated WebUI draft-hydration fix; this PR's onboarding Test-connection behavior remains intact.

Verification:

• .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py — 72 passed
• focused Ruff — passed
• npm run test:unit -- src/composables/setup/useSetupCatalog.privacy.test.ts — 175 passed
• git diff --check — passed

[lifelmy]

Task A refresh: merged current main (75712ec) into this branch and pushed 876757f; no conflicts. Revalidated backend onboarding tests (72 passed), frontend privacy suite (175 passed), focused Ruff, and git diff --check; all passed. Fresh CI has been triggered.

[lifelmy]

Task A refresh: merged current upstream/main (ee26be137) without conflicts and pushed 02f191029; branch is now 0 behind. Revalidated backend onboarding tests (72 passed), frontend privacy tests (175 passed), focused Ruff, and git diff --check; all passed.

[lifelmy]

Task A refresh: merged current upstream/main (ff4cf2a) without conflicts and pushed 8fd74d1; branch is now 0 behind. The upstream Windows-updater/runtime-diagnostics changes do not overlap the model-id-free onboarding connection path. Verification: .venv/bin/pytest -q tests/test_gateway/test_rpc_onboarding.py (72 passed); focused Ruff passed; npm --prefix opensquilla-webui run test:unit -- src/composables/setup/useSetupCatalog.privacy.test.ts (175 passed); git diff --check passed.

[lifelmy]

Task A refresh: merged current upstream/main (b661fa9) without conflicts and pushed head 540d77f (0 behind / 29 ahead). Verification: onboarding RPC pytest -> 72 passed; WebUI privacy Vitest -> 175 passed; git diff --check passed. Fresh CI is still in progress with no failures reported so far.

[lifelmy]

Task A refresh: merged latest upstream/main (09b21762a4) without conflicts and pushed a2533a644 (0 behind). Verification: onboarding RPC tests -> 72 passed; WebUI privacy Vitest -> 175 passed; focused Ruff and git diff --check passed. Fresh GitHub CI has started.

[lifelmy]

Task A CI follow-up on current head a2533a644:

• The completed CI run has one failing shard: Windows high-risk (core). Its two failures are unrelated timeout paths outside this PR's diff: test_windows_signed_update_audit.py timed out after 30s in dispatch.ps1, and test_workspace_write_deny_effects.py timed out after 60s invoking a helper script. The shard otherwise completed 7502 passed / 97 skipped.
• This PR only changes onboarding RPC/catalog files (rpc_onboarding.py, its focused tests, and useSetupCatalog*). Fresh local verification on the current base passes: backend 72 passed, WebUI privacy suite 175 passed, focused Ruff clean, and git diff --check clean.
• I attempted to rerun only the failed GitHub Actions jobs, but GitHub rejected it because rerunning this repository's workflow requires admin rights. A maintainer rerun should distinguish the Windows timing flake without changing the PR head.

[lifelmy]

Task A refresh: merged latest upstream/main (75a7085960) without conflicts and pushed 6e3331a956 (0 behind / 34 ahead). The upstream Desktop reconnect change is adjacent to connection settings but does not alter this PR’s model-id-free onboarding test-connection contract. Verification: onboarding RPC pytest (72 passed), WebUI privacy Vitest (175 passed), focused Ruff and git diff --check passed.