Skip to content

Commit fd920ca

Browse files
fix(lint): skill tool references stay a whole-stack verdict; the runtime door never judges them (#19898)
Fixes #19527 Clause-②: no This carries out ruling `5791822697` on #19527 (batch 215 item 4, letter B, maintainer 「215 同意」): 「cross-item reference resolution belongs to the whole-stack rule」. `ai-skill-tool-unresolved` is reported by `os validate` / `os lint` / `os build`. The runtime publish door never emits it. `RuntimeStackContext` and `CONTEXT_STACK_KEYS` are unchanged (ruling item 3), and no rule was added (item 2). ## What was measured first, on `origin/main` @ `1f89ba0d70` (before any edit) The measurement ran from a scratch `tsx` script against `packages/lint/src`. I deleted the script afterwards and did not commit it. | probe | reading | |---|---| | `runtimeGatedTypes()` | 17 types: `action app book dashboard dataset datasource email_template flow hook mapping object page permission position report seed view`. **`skill` is not one of them** | | `runtimeAuthoringRulesFor('skill')` / `stackKeyForType('skill')` | `[]` / `null` | | the wiring guard's unmapped set, `runtimeGatedTypes().filter(t => stackKeyForType(t) === null)` | `[]` | | the real door, a `skill` write naming a stack-level action / a plugin tool / a tool that exists nowhere | `errors: [] · advisories: [] · rulesRun: []` for all three | | the card's reproduction: `validateAiToolReferences` on the door's snapshot shape (`objects permissions books datasets`, with `actions` dropped by the builder) | `ai-skill-tool-unresolved` for all three | | the same member over the whole stack (with `stack.actions`) | stack-level action: `[]` · plugin tool: `ai-skill-tool-unresolved` · missing tool: `ai-skill-tool-unresolved` | | a `flow` write, which dispatches the suite with this member on its frozen `flow` default | 0 `ai-skill-tool-unresolved`: no door snapshot carries `skills` (`WRITTEN_STACK_KEYS` has no `skills`) | ⚠️ **The door does not emit the false advisory on `main` today.** `skill` has been outside the door since #19542, and the card says so itself ("Nothing is broken today"). The advisory appears only if the type is crossed. The reproduction is the rule run on the door's snapshot shape (the LIT case), and ablation 1 below produces it through the real door. So this PR changes no runtime behaviour. It turns a hold-out that was waiting on a wider snapshot into a design boundary, and it pins both halves of the ruling. I re-ran the measurement script after the edits and its output was byte-identical to the base run. ## Which of the two shapes the wiring guard allows: `skill` stays OUTSIDE the door The guard (`authoring-rule-wiring.test.ts`, 「every runtime-gated metadata type maps to a stack key」) checks `runtimeGatedTypes()`. That set is built from rule `runtimeTypes` declarations, not listed by hand. `runRuntimeAuthoringRules` returns `empty` at `rules.length === 0`, before it looks at `TYPE_TO_STACK_KEY`. So a type with an empty runtime rule set is not a gated type, and "crossed with an empty rule set" can only be written in two ways: - declare `skill` on some entry (for example the reference-integrity suite) with no member claiming it. Then `rulesRun` names a rule that judges nothing, which is the ruling's ⛔ on inventing a rule to give the door something to run. - add a `skill: 'skills'` row with no rule declaring it. The gate never reads such a row, which is the mapping-ahead-of-its-rules shape `runtime-gate.ts` refuses (the `seed: 'data'` note). So both absences stay, and the guard holds with `skill` outside it. ## The change Comments are restated from 「held out on a measurement」 to 「by design: cross-item reference resolution belongs to the whole-stack rule」, citing the ruling. Each place also drops the old "cross it once `actions` / `tools` are carried" direction, which the ruling rejected: - `packages/lint/src/authoring-rules.ts`, the `skill` block on the `validateReferenceIntegrity` entry. - `packages/lint/src/reference-integrity-suite.ts`, the `validateAiToolReferences` member. This member held the measurement and still prescribed widening the snapshot. It now cites ADR-0109 Decision §3 as the governing text. - `packages/lint/src/runtime-gate.ts`, the `skill` note on `TYPE_TO_STACK_KEY`. - `packages/lint/src/runtime-gate.inert-type-writes.test.ts`: the file header, the `GROUP_A` docblock, the `skill` block and the report/skill fence comment. The two DARK pins keep their assertions, and their reason text now points at the ruling. - `packages/lint/src/runtime-gate.datasource-writes.test.ts`: two comments that call `skill` "held out on the measurement". This is the second place that pins both halves (`runtimeAuthoringRulesFor('skill')` is `[]` and `stackKeyForType('skill')` is null). The LIT case 「one skill, one rule, two universes」 stays as the reason pin. Its snapshot-bound assertion now says what a red means under the ruling: someone widened `RuntimeStackContext` against item 3. It no longer says `skill` may now cross. New cases in the `skill` block of `runtime-gate.inert-type-writes.test.ts`: - **(a)** `it.each` over three tools (a stack-level AI-exposed action, a plugin-registered tool, a tool that exists nowhere). A `skill` write through the real door returns `errors []`, `advisories []` and `rulesRun []`. The in-process firing control: the member the door does not dispatch **does** return `ai-skill-tool-unresolved` on the door's snapshot shape for each of the three. - **(a)** No write type can carry a skill into a door snapshot. No gated type maps to `skills`, and `skills` is not a context key. - **(b)** `runAuthoringRules` for each of `validate`, `build` and `lint` over a whole stack reports exactly `[{ path: 'skills[1].tools[0]', severity: 'warning' }]`: the tool that exists nowhere is reported, and the stack-level action is not. - **(c)** The guard invariant holds over the real tables, and a `skill` write's `rulesRun` is `[]`. ## Firing controls: ablations on the committed tree (`eb4f0cae09`) Each mutation went through `node scripts/ablation-replace.mjs` in wrap mode. The tool checked the anchor count and blob hash on disk after each write, and each restore was confirmed as `blob == HEAD` with an empty `git diff HEAD`. No build step was involved: every subject is a relative import inside `packages/lint`, so vitest reads `src`. | # | mutation | red | green | |---|---|---|---| | 1 | cross `skill` fully: suite entry `runtimeTypes` + member `runtimeTypes: ['flow','skill']` + `skill: 'skills'` row | 9: all three (a) cases, the (a) no-other-write case, (c), both DARK pins, the report/skill fence, and the datasource DARK pin. The (a) failure is the exact false advisory through the real door: `advisories: [{ rule: 'ai-skill-tool-unresolved', path: 'skills[0].tools[0]' }]` for `action_showcase_portfolio_snapshot` | 68, including the wiring guard (a full crossing is consistent) | | 2 | declare `skill` on the suite entry only | 3: the repo's own 「every runtime-gated metadata type maps to a stack key」, (c), DARK | 64 | | 3 | delete the `validateAiToolReferences` member | (b): `os validate must still report the tool that exists nowhere: expected [] …` | none (run with `-t`, only (b) selected) | | 4 | drop `addActionFamily(stack.actions)` from `collectToolUniverse` | (b) gets 2 findings instead of 1, and the LIT lit-control leg fails | none (run with `-t`, only (b) and LIT selected) | The first attempt at ablation 1 did nothing: its third anchor was a substring of its own replacement, `ablation-replace` refused with "anchor count moved 1 -> 1", and no test ran. All three files were confirmed restored. It was re-run with a non-overlapping multi-line anchor, and those are the readings above. ## Downstream readers of `ai-skill-tool-unresolved` I ran `git grep` over `packages/`, `skills/`, `content/`, `examples/` and `apps/`. None of these expects the runtime door to emit the rule id, so none needed updating: `content/docs/ai/agents.mdx:166`, `content/docs/ai/tools.mdx:92`, `packages/cli/src/commands/generate.ts:394`, `packages/spec/src/ai/skill.zod.ts:373`, `skills/objectstack-ai/SKILL.md:85`, `packages/mcp/src/skill-prompts.ts:17`, `packages/lint/src/reference-integrity-suite.test.ts:362` and `packages/lint/src/validate-ai-tool-references.test.ts`. Each one describes a whole-stack (`os validate`) verdict, which is the ruled state. The only door-facing readers were the lint tests edited above. `@objectstack/metadata-protocol` (`runtime-authoring-gate.ts`, the only source file that imports `@objectstack/lint/runtime`) has no `skill`-specific door logic. ## Changeset: `@objectstack/lint` patch The dispatch assumed this changes runtime behaviour. The measurement shows it does not (see above). The repo rule still asks for a changeset, because the published bytes change. `@objectstack/lint` ships `dist`, and its build keeps source comments: after the rebuild, the ruling id is in `dist/index.js`, `dist/index.cjs`, `dist/runtime.js` and `dist/runtime.cjs` (3 hits each, one per restated block), and 「held out on a measurement」 is in none of them. That phrase was in `authoring-rules.ts` on the base, so it shipped there. Under AGENTS.md, `skip-changeset` is only for a diff that publishes nothing from a released package, so this takes a `patch`. There is precedent for comment-only changes that ship to `dist` (for example `.changeset/15295-serve-observability-mirror-comment.md`). Clause-② reading: **no**, with no arm. No authorable key or accept set moves in either direction. The door judged no `skill` write before this PR and judges none after it, and the whole-stack verdict is unchanged. It is neither widening nor narrowing. ## Local verification, at HEAD `fd206c3164` - `pnpm --filter @objectstack/lint exec vitest run --maxWorkers=2`: **108 files / 4127 tests passed**. - `pnpm --filter @objectstack/lint typecheck` (`tsc --noEmit` and `check:test-typecheck`): exit 0. `check:test-typecheck: OK`. `tsc -p tsconfig.test.json --listFilesOnly` includes both edited test files. - `pnpm --filter @objectstack/metadata-protocol exec vitest run --maxWorkers=2` (the package that wires the runtime gate): **188 files passed, 3 skipped · 2673 tests passed, 19 skipped**. - `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands`, derived again after the changeset commit: **59 commands**, all run. `--ran` reports: 「59 derived famil(ies) accounted for — 59 run, 0 NOT-MEASURED」. The first pass got exit 3 (PREREQUISITE NOT MET) from `check:dual-build-cjs-loads`, `check:lean-entry-closure` and `check:type-check-debt`. After a full `turbo run build --filter='./packages/*' --filter='./packages/*/*'` (72/72 tasks), all three measured green at the same HEAD. - **Scoping, declared:** the repo-wide `pnpm test` and `pnpm lint` are left to CI (`Test Core` shards, `Lint & Repo Gates`). `origin/main` has moved 4 commits past the base, and none of them touches `packages/lint`, `packages/metadata-protocol` or `packages/spec/src/ai`. ## Acceptance notes - `validateAiToolReferences` keeps its frozen `flow` default, so it still runs on every `flow` write. It judges nothing there, because a flow snapshot has no `skills`. I considered an explicit empty member `runtimeTypes` to make "never" structural, but did not make that change, since it would change which members run on flow writes and the ruling did not ask for it. The new (a) case pins that "never" holds for every write type. - The pair card 19477 (written without the sigil on purpose) still answers 404, as the claim comment recorded. --- _Generated by [Claude Code](https://claude.ai/code/session_013RDBh5DqXd2xnLwvHLgLFr)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent ecf90b2 commit fd920ca

6 files changed

Lines changed: 293 additions & 116 deletions
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
---
2+
'@objectstack/lint': patch
3+
---
4+
5+
`ai-skill-tool-unresolved` is a whole-stack verdict by design: `os validate` / `os lint` / `os build` report it, and the runtime publish gate never does (#19527)
6+
7+
A skill's `tools[]` entries resolve against three sources: `stack.tools`, the
8+
platform tool registry, and the `action_<name>` tools materialised from
9+
AI-exposed actions. The runtime publish gate judges one written item against a
10+
per-write snapshot that carries neither `stack.tools` nor `stack.actions`, and
11+
no snapshot can carry a tool that a runtime plugin registers outside the
12+
registry. At that door the rule could only ever produce a false
13+
`ai-skill-tool-unresolved`: a skill naming a real stack-level action would be
14+
told the tool does not exist. The ruling on #19527 (letter B) places this
15+
check at the whole-stack rule, following ADR-0109 Decision §3, and keeps the
16+
per-write snapshot as it is.
17+
18+
`skill` was already outside the runtime gate. The earlier comments called that
19+
a temporary hold-out, pending a wider snapshot. They now describe it as the
20+
design, so no later change should add `actions` / `tools` to the snapshot in
21+
order to move this check onto the door. New tests pin both halves:
22+
23+
- a `skill` write through the runtime gate gets no tool-reference finding. This
24+
holds for a stack-level action tool, a plugin-registered tool, and a tool
25+
that exists nowhere. No other write type can put a skill into a door
26+
snapshot;
27+
- `validate`, `build` and `lint` still report a tool that exists nowhere, at
28+
warning severity, and do not report the stack-level action that resolves.
29+
30+
⛔ No behaviour changes. Before and after this change, the runtime gate runs no
31+
rule for a `skill` write, and the whole-stack commands report the same
32+
findings. No authorable key, accept set or export changes.
33+
34+
**This ships, so it carries a changeset rather than `skip-changeset`.**
35+
`@objectstack/lint` publishes `dist`, and its build keeps source comments. A
36+
rebuilt artifact shows the reworded block in `dist/index.js`,
37+
`dist/index.cjs`, `dist/runtime.js` and `dist/runtime.cjs`, and the old
38+
wording (「held out on a measurement」) in none of them. So the published JS
39+
bytes change, but behaviour and the declaration surface do not.

‎packages/lint/src/authoring-rules.ts‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -907,14 +907,24 @@ export const AUTHORING_RULES: readonly AuthoringRule[] = [
907907
// MEMBERS judge that snapshot. A `report` write reaches exactly
908908
// `validateChartBindings`. Every other member keeps its declaration.
909909
//
910-
// ⛔ `skill` is NOT here. It was crossed in an earlier revision of this
911-
// card and is held out on a measurement: `validateAiToolReferences`
912-
// resolves into `stack.tools` and `stack.actions`, neither of which the
913-
// per-write snapshot carries, so the shipped corpus's own AI-exposed
914-
// stack-level action reads as unresolved at the door and the rule ships a
915-
// false advisory into Studio. The member carries the measurement; the type
916-
// takes the ruling's group B treatment of `tool`, the same universe
917-
// obstacle read from the other side.
910+
// ⛔ `skill` is NOT here, BY DESIGN: cross-item reference resolution
911+
// belongs to the whole-stack rule (ruling `5791822697` on #19527, letter
912+
// B; ADR-0109 Decision §3). `ai-skill-tool-unresolved` is a verdict of
913+
// `os validate` / `os lint` / `os build`, and this door NEVER emits it.
914+
// `validateAiToolReferences` resolves a skill's tools against a universe of
915+
// three limbs — `stack.tools`, the platform registry and the materialised
916+
// `action_<name>` family — and a per-write snapshot carries neither
917+
// `stack.tools` nor `stack.actions`, while a tool a runtime plugin
918+
// registers outside the registry is invisible to every static universe.
919+
// So at this door the member has no truthful `unresolved` verdict to give:
920+
// the shipped corpus's own AI-exposed stack-level action reads as
921+
// unresolved here and resolves over the whole stack (the reason pin is the
922+
// LIT case in `runtime-gate.inert-type-writes.test.ts`). ⛔ Not a hold-out
923+
// awaiting a wider snapshot: the ruling rejected carrying `actions` /
924+
// `tools` in `RuntimeStackContext` (a read on every gated write that still
925+
// leaves the plugin limb false). The member keeps the frozen `flow` default
926+
// and `skill` has no `TYPE_TO_STACK_KEY` row: with no rule to run it is not
927+
// a runtime-gated type, so the wiring guard's invariant asks no row of it.
918928
//
919929
// ⛔ `action` and `hook` are deliberately NOT here, although this card
920930
// crosses both types on `validateStackExpressions` above. The suite carries

‎packages/lint/src/reference-integrity-suite.ts‎

Lines changed: 38 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -460,45 +460,50 @@ export const REFERENCE_INTEGRITY_RULES: readonly ReferenceIntegrityRule[] = [
460460
{ name: 'validateTranslatableSections', run: validateTranslatableSections },
461461
{ name: 'validateFlowTemplatePaths', run: validateFlowTemplatePaths },
462462
{ name: 'validateAiSurfaceAffinity', run: validateAiSurfaceAffinity },
463-
// [#19542] NO `runtimeTypes`, i.e. the frozen `flow` default — and `skill` is
464-
// HELD OUT of the ADR-0049 wiring deliberately, on a measurement, not
465-
// forgotten. This member is the measurement's named bridge for the type and
466-
// it does read the written skill as its subject (`recordsOf(stack.skills)`,
467-
// findings pathed `skills[si].tools[ti]`), so on paper it crosses. What
468-
// stops it is the UNIVERSE it resolves into.
463+
// [#19542, #19527] NO `runtimeTypes`, i.e. the frozen `flow` default — and
464+
// `skill` stays outside the runtime publish door BY DESIGN: cross-item
465+
// reference resolution belongs to the whole-stack rule (ruling `5791822697`
466+
// on #19527, letter B). `ai-skill-tool-unresolved` is a verdict of
467+
// `os validate` / `os lint` / `os build` and the door NEVER emits it. The
468+
// governing text is ADR-0109 Decision §3: `skill.tools[]` reference
469+
// integrity is decided by THIS member over the WHOLE stack, at warning
470+
// severity, because a runtime plugin outside the registry stays statically
471+
// invisible and the runtime deliberately tolerates unresolved names.
469472
//
473+
// This member does read the written skill as its subject
474+
// (`recordsOf(stack.skills)`, findings pathed `skills[si].tools[ti]`), so on
475+
// paper it could cross. What rules it out is the UNIVERSE it resolves into.
470476
// `collectToolUniverse` unions `PLATFORM_PROVIDED_TOOL_NAMES` ∪
471477
// `stack.tools` ∪ the materialised action family from `stack.actions` and
472478
// every object's `actions`. A per-write snapshot carries `objects` — so an
473-
// object-level `action_<name>` resolves — but NEITHER `tools` NOR `actions`.
474-
// Two of the three limbs are missing, so at this door the rule has no
475-
// truthful `unresolved` verdict at all: only its CLEAN answers are reliable.
479+
// object-level `action_<name>` resolves — but NEITHER `tools` NOR `actions`,
480+
// and no snapshot can carry a tool a runtime plugin registers. So at this
481+
// door the member has no truthful `unresolved` verdict at all: only its
482+
// CLEAN answers are reliable.
476483
//
477-
// ⚠️ Measured on the SHIPPED corpus, not synthetically. `app-showcase`
478-
// declares exactly one AI-exposed action, `showcase_portfolio_snapshot`, and
479-
// it exists at stack level only (todo's six are mirrored under
480-
// `objects[].actions`, so those resolve; crm has none). A skill naming
481-
// `action_showcase_portfolio_snapshot` is advised `ai-skill-tool-unresolved`
482-
// at the door while the same member over the whole showcase stack answers
483-
// `[]`. The advisory's own hint tells the author to declare the action and
484-
// opt it in with `ai.exposed: true` — which is exactly what the author did —
485-
// and it reaches `SaveMetaItemResponseSchema.advisories`, which Studio
486-
// renders. A user would see a warning that is not true.
484+
// ⚠️ The reading, measured on the SHIPPED corpus rather than synthetically:
485+
// `app-showcase` declares exactly one AI-exposed action,
486+
// `showcase_portfolio_snapshot`, and it exists at stack level only (todo's
487+
// six are mirrored under `objects[].actions`, so those resolve; crm has
488+
// none). A skill naming `action_showcase_portfolio_snapshot` would be
489+
// advised `ai-skill-tool-unresolved` on the door's snapshot while the same
490+
// member over the whole showcase stack answers `[]`, and the advisory would
491+
// reach `SaveMetaItemResponseSchema.advisories`, which Studio renders. That
492+
// is the reason pin — the LIT case in
493+
// `runtime-gate.inert-type-writes.test.ts` — not a wait state.
487494
//
488-
// ⛔ ADR-0109 does NOT bound this. Its 「the default path declares no tool
489-
// records」 covers tool RECORDS; the `action_<name>` family IS that default
490-
// path, so the bound argues the opposite way.
491-
//
492-
// So the type takes the ruling's OWN group B treatment of `tool` — 「the
493-
// `tools` universe rule … can only remove findings ⇒ a reading, not a
494-
// ruling」 — which is this same rule read from the other side. Crossing it
495-
// needs `actions` / `tools` carried in `RuntimeStackContext` +
496-
// `CONTEXT_STACK_KEYS`, a `CLOSURE_CONTEXT_KEY_BY_TYPE` row and two more
497-
// door gathers in `@objectstack/metadata-protocol` — a second package, a
498-
// snapshot widening on EVERY gated write, and its own card. `skill` has no
499-
// `TYPE_TO_STACK_KEY` row either, so the two absences are CONSISTENT and the
500-
// gate dispatches nothing for it; `runtime-gate.inert-type-writes.test.ts`
501-
// holds both absent and records the measurement that put them there.
495+
// ⛔ Not a hold-out awaiting a wider snapshot. The ruling rejected carrying
496+
// `actions` / `tools` in `RuntimeStackContext` + `CONTEXT_STACK_KEYS`: that
497+
// costs an indexed read on EVERY gated write, the hot `object` / `flow`
498+
// doors included, and still leaves the plugin limb false — a narrower false
499+
// advisory, not a true one. `skill` has no `TYPE_TO_STACK_KEY` row either,
500+
// so the two absences are CONSISTENT and the gate dispatches nothing for it.
501+
// With no rule to run, `skill` is not a runtime-gated type under the wiring
502+
// guard's invariant (「every runtime-gated metadata type maps to a stack
503+
// key」), so that invariant asks no row of it; a row with no declaring rule
504+
// is the inert mapping-ahead-of-its-rules shape `runtime-gate.ts` refuses,
505+
// and inventing a rule to give the door something to run is refused by the
506+
// same ruling. `runtime-gate.inert-type-writes.test.ts` holds both absent.
502507
{ name: 'validateAiToolReferences', run: validateAiToolReferences },
503508
{ name: 'validateAiAgentAuthoring', run: validateAiAgentAuthoring },
504509
// Field names WRITTEN by an L2 hook body (`ctx.input.x = …`,

‎packages/lint/src/runtime-gate.datasource-writes.test.ts‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -257,7 +257,9 @@ describe('#19568 — the hold-out test: what the door sees is what the whole sta
257257
// The `skill` measurement (#19527), asked of this type. There, the door's
258258
// snapshot lacked `stack.tools` / `stack.actions`, so one rule produced
259259
// `ai-skill-tool-unresolved` at the door and `[]` over the whole stack —
260-
// a falsehood rendered in Studio, and the reason that type is held out.
260+
// a falsehood rendered in Studio, and the reason that type stays outside
261+
// the door by design (ruling `5791822697`: cross-item reference resolution
262+
// belongs to the whole-stack rule).
261263
//
262264
// Here the two runs are the same rule over the same written item, once in
263265
// the shape a per-write snapshot has (objects + the written collection),
@@ -356,8 +358,9 @@ describe('#19568 — DARK: what this crossing did NOT widen', () => {
356358

357359
it('the types still awaiting their own readings are untouched', () => {
358360
// The standing ungated controls: group B reads `translation` / `tool`
359-
// first, group D retires `doc` / `external_catalog`, and `skill` is held
360-
// out on the measurement this file tested itself against.
361+
// first, group D retires `doc` / `external_catalog`, and `skill` stays
362+
// outside the door by design (ruling `5791822697` on #19527), for the
363+
// reason this file tested itself against.
361364
expect(runtimeAuthoringRulesFor('translation')).toEqual([]);
362365
expect(runtimeAuthoringRulesFor('tool')).toEqual([]);
363366
expect(runtimeAuthoringRulesFor('doc')).toEqual([]);

0 commit comments

Comments
 (0)