Skip to content

fix(service-settings): register the ai.test_embedder and knowledge.test handlers by default - #24830

Merged
objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-24769-settings-test-handlers
Oct 20, 2026
Merged

objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-24769-settings-test-handlers

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #24769

Clause-②: no

What was wrong

Setup's AI "Test embedder" and Knowledge "Test connection" buttons both answered 400 SETTINGS_ACTION_FAILED, with "No handler registered for action 'test_embedder' in 'ai'." and "No handler registered for action 'test' in 'knowledge'.". The manifests declare both buttons, and aiTestEmbedderActionHandler and knowledgeTestActionHandler are exported beside them. But SettingsServicePlugin's default actionHandlers map registered only mail.test, sms.test, storage.test and ai.test.

Nothing registers either action later. At a43405d59, git grep "registerAction(" finds only three registrations against a settings namespace outside this package: plugin-email (mail), service-sms (sms) and service-storage (storage). @objectstack/service-knowledge, @objectstack/knowledge-memory, @objectstack/knowledge-ragflow and @objectstack/embedder-openai never reference the settings service. So the PM's mechanism assumption M2 holds on the "no override" half: in this repository the stubs are the only possible handlers. @objectstack/service-ai is edition: 'cloud' and not in this repository, so whether it overrides ai/test_embedder is NOT MEASURED.

Landing: packages/services/service-settings/src/settings-service-plugin.ts, as the card expected. The defect is in the producer of the default map, not in a consumer.

The fix

  1. Registration. The default map now carries ai: { test, test_embedder } and knowledge: { test }. Override semantics are unchanged (M3, measured): a host's own actionHandlers still replaces the whole default map (opts.actionHandlers ?? DEFAULTS), and SettingsService.registerAction (a Map.set) still lets a plugin replace any one handler later. The option's docblock used to say only "the bundled mail.test handler is registered automatically". It now names the default set and says that passing the option replaces the whole map.
  2. Answer text (M2's second half). No answer claimed outright that a connection or round trip was tested. Two answers did promise something false, though: "Mount @objectstack/embedder-openai + a knowledge adapter to exercise live calls" and "Mount @objectstack/knowledge-ragflow to exercise live calls". In this file that phrase follows the storage stub's pattern ("Mount @objectstack/service-storage to exercise live I/O"), and there it is true: service-storage overrides storage/test with a live probe. Neither embedder-openai nor knowledge-ragflow overrides anything, so mounting them never made either button live. Both phrases are removed. Every passing answer on a remote backend now says what it is: "This checks the form only — no embedding request was sent." for the embedder, and "This checks the form only — no connection was attempted." for Turso and RAGFlow. Two handler docblocks claimed a live-probe override "in @objectstack/service-knowledge". That is false, and they now state what was measured.

What each button checks after this change

Neither button touches the network. Each merges the posted form state (a nested values object or a bare key map) over the saved values, then:

AI → "Test embedder" (POST /api/settings/ai/test_embedder)

  • embedder_provider unset or none: ok: false, warning, "Embedder is disabled. Pick a provider to enable knowledge / RAG."
  • any provider except ollama with no embedder_api_key: ok: false, error.
  • custom / azure with no embedder_base_url: ok: false, error.
  • otherwise: ok: true, info, "PROVIDER embedder settings are complete (model=MODEL). This checks the form only — no embedding request was sent."

Knowledge → "Test connection" (POST /api/settings/knowledge/test)

  • memory: ok: true, warning, "In-memory adapter — no external service to probe. …" (unchanged).
  • turso with no URL: ok: true, info, the reuse-the-tenant's-connection note, plus "This checks the form only — no connection was attempted."
  • turso with a libsql:// URL and no auth token: ok: false, error.
  • turso with any other URL: ok: true, info, "Turso adapter configured (URL). This checks the form only — no connection was attempted." The existing sentence about @objectstack/knowledge-turso follows, unchanged.
  • ragflow with no Base URL or no API key: ok: false, error.
  • ragflow complete: ok: true, info, "RAGFlow adapter configured (URL). This checks the form only — no connection was attempted."
  • any other adapter: ok: false, error.

The route is unchanged: ok: true answers 200 with the result, and ok: false answers 400 SETTINGS_ACTION_FAILED with the result under error.details.

The pin, and the class it closes

src/settings-default-action-handlers.pin.test.ts reads every builtin manifest's http-kind action_button whose URL is the settings action route. It constructs the plugin with no options, runs init(), and drives each declared action through runAction on the service the plugin registered. Every handler the plugin registers is wrapped to record its own invocation. An action counts as honoured when a registered handler ran, or when runAction answered ok without one (its built-in reset, served for every namespace). The no-handler answer is neither. No message text is used as the discriminator.

The file holds a floor: the population must contain all seven current pairs. A second assertion checks that each button's route names its own namespace and id. Two HTTP-door cases replay the card's own two requests through registerSettingsRoutes. A new button declared without its handler, or a handler dropped from the default map, now goes red by name.

Red before, green after

Run with the test commit a66a74166 on the unchanged plugin (base a43405d59): Tests 3 failed | 2 passed (5).

AssertionError: expected [ Array(2) ] to deeply equal []
+   "ai/test_embedder",
+   "knowledge/test",
AssertionError: expected 400 to be 200      (POST ai/test_embedder)
AssertionError: expected 400 to be 200      (POST knowledge/test)

With the fix, on 62b892d56 (the final head, origin/main 307ce4c4b merged in): pnpm --filter @objectstack/service-settings exec vitest run --maxWorkers=2 reports Test Files 54 passed (54), Tests 899 passed (899). pnpm --filter @objectstack/service-settings typecheck exits 0. tsc --listFiles includes all three touched test files.

Ablations (one-off, run at c8dbf1395; nothing permanent was added)

All three mutations went through scripts/ablation-replace.mjs in WRAP mode. Each anchor hit once, the blob changed, and the restore was proven: blob equal to HEAD, and git diff HEAD empty. The tests resolve this package's own source through relative imports, and @objectstack/* through the vitest source aliases, so no dist/ is involved. Every ablation went red, the expected direction.

  • A: drop knowledge: { test: knowledgeTestActionHandler } from the default map. Tests 2 failed | 3 passed (5). The failures were expected [ 'knowledge/test' ] to deeply equal [] and, at the HTTP door, expected 400 to be 200. Restored blob 716f8a5cda1d equals HEAD.
  • B: append the old "Mount @objectstack/embedder-openai + a knowledge adapter to exercise live calls." to the embedder answer. Tests 1 failed | 27 passed (28): expected 'openai embedder settings are complete…' not to match /to exercise live calls/. Restored blob bb1ad034d9ad equals HEAD.
  • C: append the old "Mount @objectstack/knowledge-ragflow to exercise live calls." to the RAGFlow answer. Tests 1 failed | 24 passed (25): ragflow: expected 'RAGFlow adapter configured (http://lo…' not to match /to exercise live calls/. Restored blob 396195582b05 equals HEAD.

B and C keep the "no … was sent / attempted" sentence and add the old promise, so the negative pin fires on its own.

Gates (local subset, on 62b892d56)

  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 67 commands from the 7 changed paths, against merge base 307ce4c4b. Each was run with its exit code captured before any pipe.
  • 66 exited 0. node scripts/check-plugin-teardown-shape.mjs --self-test first exited 3 because the shallow clone could not reach its pinned fixture commit. Once that commit was present it re-ran: exit 0, "48 cases pass".
  • pnpm check:dual-build-cjs-loads exited 3, PREREQUISITE NOT MET. It reads every package's dist/, and this worktree has built only the service-settings closure. NOT MEASURED locally; CI's Lint & Repo Gates runs it.
  • --ran reconciliation: "67 derived famil(ies) accounted for — 66 run, 1 NOT-MEASURED (1 DERIVED from a recorded exit 3)", 0 UNRUN.
  • Dependency closure built with turbo run build --filter='@objectstack/service-settings...' (16 tasks, exit 0).
  • test:repo was not run locally; it belongs to CI (Test Core).

Acceptance notes

  • Nothing in this repository reads the settings these buttons check. That is the knowledge namespace and the ai.embedder_* keys: the only readers are service-settings' own manifests, tests and visibility evaluator. @objectstack/knowledge-ragflow and @objectstack/embedder-openai take their configuration from plugin constructor options. Whether a cloud runtime reads these values was not measured. Not filed: this is a read-only inference with no public-door measurement. Carrier: none.
  • aiTestActionHandler's docblock is unverified. It still says the real ai/test lives in @objectstack/service-ai and overrides the stub on kernel:ready. That package is cloud-edition, so this could not be measured here. Left untouched, because ai.test was already registered and is not this card's button.
  • Compatibility edge. init() registers each default handler through registerAction, which throws UnknownNamespaceError for an unregistered namespace. A host that passes its own manifests without knowledge, and does not pass actionHandlers, would now fail at boot. The same edge already existed for mail / sms / storage / ai. Every in-repo site that passes manifests also passes actionHandlers: {}; the sites that pass no manifests get the builtin set, which includes knowledge.
  • Two fixture comments name an incomplete handler list. config-change-audit.test.ts and settings-admission-tenancy-posture.test.ts describe the bundled handlers as mail/sms/storage/ai. The reason those comments give still holds; they are outside this card's file surface and were not edited.
  • Two contrast-control tests were replaced. The embedder test in ai.manifest.test.ts and the RAGFlow test in knowledge.manifest.test.ts pinned the two removed "Mount … to exercise live calls" sentences, kept because those packages are built here. The packages do exist, but the promise was false for these buttons. The replacements pin the new property, and the AI live-call branches are still each exercised on their own.

Risk and rollback

The change is additive registration plus answer text. Rollback is a revert of c8dbf1395: the two buttons go back to answering 400.

Changeset: .changeset/24769-settings-test-handlers.md, @objectstack/service-settings patch.


Generated by Claude Code

…n reaches a handler

Claude-Session: https://claude.ai/code/session_01UmqzD97e14iws6hmVhgZ5y
Co-authored-by: Claude <noreply@anthropic.com>
…st handlers by default

Both buttons were declared and their handlers exported, but the default
actionHandlers map never registered them, so each answered 400 'No handler
registered'. Their success messages now say they check the form only and no
request was sent: nothing in this repository replaces either with a live probe.

Claude-Session: https://claude.ai/code/session_01UmqzD97e14iws6hmVhgZ5y
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation tests tooling labels Oct 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

4 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • 1 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 87c09ebb7ac768b43d137996296d042a3cee1c97 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from befbee0352a3b4decd6c6b65fef5977247098c73 — the merge of head 62b892d56b0bb0b0927a4d999aeec84d7bb21039 into base 87c09ebb7ac768b43d137996296d042a3cee1c97, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin befbee0352a3b4decd6c6b65fef5977247098c73 && git checkout befbee0352a3b4decd6c6b65fef5977247098c73
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 87c09ebb7ac768b43d137996296d042a3cee1c97 62b892d56b0bb0b0927a4d999aeec84d7bb21039 && git checkout -B drift-repro 87c09ebb7ac768b43d137996296d042a3cee1c97 && git merge --no-ff 62b892d56b0bb0b0927a4d999aeec84d7bb21039

node scripts/docs-audit/affected-docs.mjs --json 87c09ebb7ac768b43d137996296d042a3cee1c97

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对

ACCEPT · #24769 · domain:services seat 2 · os-support-ai · session_01UmqzD97e14iws6hmVhgZ5y · 2026-10-20T08:45Z · head 62b892d56b, the review of record for claim 6285572462.

  1. Form:
    • Draft, base main. The first line is Fixes #24769; the seat's grep of the body finds no other closing keyword beside an issue number.
    • Clause-②: no at line start in the body and in the changeset. Registering a handler for an action the manifests already declare implements a declared capability. It widens no published contract.
    • One patch changeset for @objectstack/service-settings.
    • 7 files, +309/−44, all inside the claim's file surface. settings-service.ts, settings-service.types.ts, auth.manifest.ts and the translation files are untouched.
  2. Premise and fix, read by the seat:
    • The plugin hunk. The default actionHandlers map gains ai.test_embedder and knowledge.test. The opts.actionHandlers ?? defaults override semantics are unchanged (M3).
    • M2, measured by the dev. No package in this repository registers either action. So both buttons stay form checks, and the two answers that promised "Mount … to exercise live calls" now say "This checks the form only". Those answers were false for these buttons. This text change is the same defect (a declared action answered untruthfully), inside the claim's surface.
    • The class pin. It enumerates every bundled action_button routed to the settings action door and drives it through the default-composed plugin, observing which handler actually ran. It went red on the unchanged plugin naming exactly ai/test_embedder and knowledge/test. It also has two HTTP-door cases (400 before, 200 after).
    • Ablation. Dropping the knowledge registration went red. Restoring each of the two removed sentences went red on its own negative pin. Each leg was restored to the HEAD blob.
  3. Gates:
    • CI on 62b892d56b at this read: 38 check-runs, 20 success, 7 skipped, 11 in progress, 0 failed. Not yet green, so not yet ready.
    • Local on the head: 67 derived, 66 run, 1 NOT-MEASURED (check:dual-build-cjs-loads, CI's), 0 UNRUN.
    • Package suites: service-settings 54 files, 899 tests; typecheck exit 0.
  4. Accepted deviations:
    • Answer text is pinned. The new tests assert the "no embedding request was sent" / "no connection was attempted" sentences, and the absence of "to exercise live calls". Kept: the negative half guards a measured-false promise, which is this card's defect.
    • A boot edge (Acceptance notes item 3). A host that passes its own manifests without knowledge and no actionHandlers would now throw UnknownNamespaceError at boot. The same edge already holds for mail, sms, storage and ai. The seat found no such host: every non-test construction (packages/cli's one-shot plugin, packages/verify's harness) passes no manifests, and hotcrm and objectui construct none. Not filed.
  5. Out of scope: the PR's five Acceptance notes. None is a filing-gate class with a public-door reach:, so none is filed.
  6. Landing: ordinary queue landing (check-governed-merges --pr 24830: not governed, 353 lines). When every check on the current head is green, the seat sends pr_ready + automerge_enable through the relay, then confirms MERGED.

Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 20, 2026 09:06
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 20, 2026 09:06
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit 7697ddc Oct 20, 2026
42 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-24769-settings-test-handlers branch October 20, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Setup: AI "Test embedder" and Knowledge "Test connection" have no server handler — the handlers exist but are not registered

2 participants