Skip to content

Commit 0dd93f2

Browse files
skills(automation): integrations are declared, not coded — rule section, per-archetype recipes, three evals (#27322)
Fixes #27223 Clause-②: no Metadata-protocol: none ## 维护者速读 - **改了什么**:`objectstack-automation` 技能包新增规则节「Integrations are declared, not coded」(厂商系统走应用内声明的 `connectors:` 条目 + `connector_action`,回调走 `api` 流;五种情形才写平台代码;加载器暂不支持的文档是「待平台卡」的缺口,不是写插件的理由),`connector_action` 节点行不再写 "pre-built";新增 `references/integration-recipes.md`(六种原型各一份配方 + 作者/管理员凭据交接 + 两条「尚未支持」)和 `evals/integrations-declared.json`(DocuSign / Stripe 读 / 无规格厂商三例,DocuSign 例对任何「写平台插件」的答案判失败)。 - **为什么改**:HotCLM 的开发 agent 读了技能后结论是平台要造 DocuSign 插件(维护者原话 「hotclm 的开发agent 就搞错了,说明 skills还是不对,要立卡完善」);技能里确有两句把它推向这个方向。 - **风险与回滚**:纯技能文本与 eval,无运行时改动;token 棘轮:SKILL.md 净 −1 token,两份新文件按落地值钉死(整包 +3370 token);回滚即 revert 本 PR。 - **先看**:① `skills/objectstack-automation/SKILL.md` 新节(约第 402 行起)——五种「写平台代码」的情形与缺口命名的措辞;② `references/integration-recipes.md` 「Holds on every recipe」第二条(`connectorConfig.input` 按原样发送,逐记录 payload 今天不可声明)——这是实测到的运行时事实,也是下方 Acceptance notes 第 1 条发现;③ `evals/integrations-declared.json` 第 1 例的 must_not_contain。 - **两分钟验证**:`node scripts/check-skills-token-ratchet.mjs && node scripts/pm/check-skill-id-lint.mjs`——预期两条均 ✓,automation 五个计价文件 headroom 全为 0。 - **需要你做的**:Tier H 技能面,由你手动落地(merge);卡号为何未写进技能正文见 What changes 末段。 ## What changes - `skills/objectstack-automation/SKILL.md` — the `connector_action` node row no longer says "pre-built integration connector"; the "reserve custom code for edge-case integrations" sentence is gone; a new section **Integrations are declared, not coded** states the rule, the five cases that justify platform code, and that a document the loader cannot take yet (YAML, Swagger 2.0, `$ref`, OAuth, request signing) is a gap to name against its platform card, never a reason to write code. The `http` row's auth-routing tail moved into the recipes. Paid for in the same file: net −1 token under the ratchet, headroom 0. - `skills/objectstack-automation/references/integration-recipes.md` (new, 150 lines) — one recipe per archetype: OpenAPI with a static key (Stripe balance, nightly), remote MCP with a static header (GitHub), plain REST (`provider: 'rest'`, `actionId: 'request'`), the vendor callback (`api` flow with `secretRef`), a hand-written OpenAPI 3.x document for a spec-less vendor, and the two **not yet** entries (OAuth-protected, signature-authenticated). Each states what the app author declares and what the customer's administrator supplies. It also carries three limits measured on this base: `connectorConfig.input` is handed to the provider as written (no CEL envelope or `{{ }}` hole is evaluated — `service-automation/src/builtin/connector-nodes.ts:108`, and `value-slot-map.ts` lists no connector slot), the `openapi` provider has no `include` key (`connector-openapi/src/openapi-provider.ts:38-52` — the trimmed document is the allowlist), and the inbound `api` door verifies only `x-objectstack-signature` in the timestamped form (`trigger-api/src/api-trigger.ts:163-178`, "has no unsigned branch"), so a vendor signing its own header cannot post today. - `skills/objectstack-automation/evals/integrations-declared.json` (new) — the DocuSign case (expected: a declarative `openapi` instance over a trimmed, packaged 3.x document converted offline from Swagger 2.0, an `api` callback flow, three named gaps, marked blocked on OAuth; `must_not_contain` refuses `registerConnector(`, `connector-esign`, `provider: 'docusign'`, `implements Plugin`, `clientSecret`, `token:`), the Stripe-read case (scheduled flow writing `crm_account` fields) and the vendor-without-spec case. - `scripts/check-skills-token-ratchet.mjs` — the SKILL.md row re-locked at its measurement (6466 → 6465); two rows pin the new files at their landed counts (1193, 2177). - `.changeset/27223-integrations-declared.md` — `@objectstack/skills` patch (the catalog ships in that package's `dist` via `sync-catalog.mjs`). **Card numbers are not in the skill text.** The card asks the recipes to name the lifting cards. `pnpm check:pm-skill-id-lint` forbids `#NNN` in every `.md` under `skills/` by maintainer ruling (2026-08-12, widened to the published catalog on 2026-10-12) — measured on this tree: a probe line `Waits on #26709 for include.` under `references/` reds the gate (exit 1, naming the file and line). The recipes therefore name each gap and say it is blocked on "its platform card"; the numbers live here: `include` on the `openapi` provider and offline normalization → #26709; `oauth2` in the declarative auth union → #26702; request signing → #26710; the administrator-filled credential object → #27237 (in flight; the recipes teach today's env-var `credentialRef` only). **Bundle growth, with its authorization.** The published bundle grows by 3,370 tokens (two new files pinned at landed counts; SKILL.md −1). The ratchet lands growth only against a maintainer ruling quoted in the PR body; this one is the card's own directive, verbatim and untranslated: 「所以应该先处理 A/B 吧,并且要有 skills或者文档指导元数据开发者。前面agent还在建议我开发 DocuSign」 and 「hotclm 的开发agent 就搞错了,说明 skills还是不对,要立卡完善」(2026-10-28). The PM-set budget — SKILL.md net tokens ≤ 0, recipes ≤ 150 lines, one eval file — is met. The claim's "one entry in `references/_index.md`" does not apply: that file is generator-owned (`gen:skill-refs` writes schema pointers only); `check:skill-refs` passes with the hand-written reference beside it, and SKILL.md links the recipes directly. ## Tests No package source changed, so no `pnpm test` / `pnpm typecheck` is owed; the edited gate script's own suite is its `--self-test` (79 cases pass, below). **Skill evals — what exists and what was run.** The root repo has no eval runner: no `pnpm` script consumes `skills/*/evals/*.json`, and the only eval gate in the tree, `ui/scripts/check-skill-eval-tokens.mjs`, scans `ui/skills` alone. The card's "`pnpm` skill evals green" therefore cannot be run as spelled. What was run instead, from a scratch replay (not committed) of that gate's whole-token rule — every `must_contain` token must occur as a whole token in the bundle's `.md` corpus — over `skills/objectstack-automation/`: <details><summary>Oracle, base bundle (a0aa8a1): the three new evals FAIL — exit 1</summary> ``` corpus: 5 .md file(s): SKILL.md, evals/approvals/test-revise-loop.md, references/_index.md, references/examples-flows.md, references/state-machines-and-approvals.md FAIL eval 1: 8 must_contain token(s), 5 untaught -> "defineConnector", "provider: 'openapi'", "type: 'api'", "Swagger 2.0", "blocked" FAIL eval 2: 7 must_contain token(s), 1 untaught -> "provider: 'openapi'" FAIL eval 3: 5 must_contain token(s), 3 untaught -> "provider: 'openapi'", "operationId", "type: 'basic'" ORACLE: 9 must_contain token(s) untaught ``` </details> <details><summary>Oracle, this branch (2993d73): the three new evals PASS — exit 0</summary> ``` corpus: 6 .md file(s): SKILL.md, evals/approvals/test-revise-loop.md, references/_index.md, references/examples-flows.md, references/integration-recipes.md, references/state-machines-and-approvals.md PASS eval 1: 8 must_contain token(s), 0 untaught PASS eval 2: 7 must_contain token(s), 0 untaught PASS eval 3: 5 must_contain token(s), 0 untaught ORACLE: every must_contain token is taught by the bundle ``` </details> NOT MEASURED: a live LLM answering arm per eval (base text vs new text) — no runner exists and a self-graded run by the authoring seat is not evidence; the DocuSign case's refusal of a plugin answer is enforced by its `must_not_contain` list, not by a measured model run. ## Acceptance notes 1. **Finding, class c (metadata-authoring trap), for the seat to file.** `connectorConfig.input` on a `connector_action` node is handed to the provider handler as written (`packages/services/service-automation/src/builtin/connector-nodes.ts:108`: `handler((cfg.input ?? {}) …)`); `builtin/value-slot-map.ts` resolves `fields`, `subflow`/`map` `input`, `script` `inputs`, `screen` `defaults`, `notify` `templateData` and `http` `body`, never connector `input`. The authoring door accepts a CEL envelope there: `FlowSchema.safeParse` of a flow whose `connectorConfig.input` is `{ path: { customer: { dialect: 'cel', source: 'record.stripe_customer_id' } } }` answers `success = true` and keeps the envelope verbatim (measured with `tsx` against `packages/spec/src/automation/flow.zod.ts` on this tree). The `openapi` handler then interpolates the path from `String(value)`, so the vendor receives an object stringified. Reach: `os validate` runs `FlowSchema`; producer: any app author's stored flow definition. Dedupe words: `connector_action input value slot CEL envelope verbatim`. The recipes state the limit; they do not work around it. 2. **The card's "`pnpm` skill evals" does not exist for the root catalog** — the eval fixtures under `skills/*/evals/` have no mechanical consumer in this repository; `ui/` has one for its own bundle. Observation for the PM (the acceptance wording), not a card from this seat. 3. **Pre-existing, observation only**: under the same whole-token rule, two existing evals in `evals/flows-triggers-approvals.json` carry untaught tokens on base as well as on this branch — eval 7 (`excludeSubmitter: true`) and eval 8 (`type: 'screen'`, `type: 'file'`, `signed_copy`). No root gate asserts the rule; carrier: none. 4. `#25747` (the DocuSign instance the card cites as open) is closed `not_planned` as of 2026-10-28 — no effect on the premise, which was re-verified on `origin/main` a0aa8a1: `pre-built integration connector` at `SKILL.md:89`; `defineConnector|provider: 'openapi'` → 0 files under `skills/*/references` and `skills/*/evals`; `Declaring a provider-bound instance` at `content/docs/automation/connectors.mdx:40`. 5. `references/examples-flows.md:49-55` (the plugin-registered `slack` connector in a `connector_action`) is outside the declared surface and was not edited; it stays a legitimate shape, and the rewritten `connector_action` row says so ("or a plugin's"). 6. The `runAs: 'system'` paragraph in SKILL.md says a system sweep must pin its organization but names no column, and the catalog's own scheduled example carries no predicate; the Stripe recipe mirrors the taught shape and adds a one-line reminder. Read-only inference; noted, not filed. 7. Five derived families exit 3 locally (`PREREQUISITE NOT MET`: they read built `dist/`, which this diff never builds) — `check:doc-formula-expressions`, `check:dts-closure`, `check:dual-build-cjs-loads`, `check:lean-entry-closure`, `check:sourcemap-no-sources-content`. They are pulled in by the `scripts/` touch, read none of the changed paths, and run in CI after the build; recorded as NOT MEASURED locally. ## Verification Head `2993d737` (tree identical to the readings below; no commit followed them). <details><summary>skills/** readings — the two required before/after counts (lines and tokens)</summary> | File | Lines before → after | Tokens before → after (ratchet convention) | |:--|:--|:--| | `skills/objectstack-automation/SKILL.md` | 451 → 451 | 6466 → 6465 (ceiling re-locked at 6465, headroom 0) | | `skills/objectstack-automation/references/integration-recipes.md` | 0 → 150 | 0 → 2177 (pinned, headroom 0) | | `skills/objectstack-automation/evals/integrations-declared.json` | 0 → 35 | 0 → 1193 (pinned, headroom 0) | | whole package (sum of every `skills/*/SKILL.md`) | 4615 → 4615 | bundle total (whole shipped tree) 215826 → 219195 | </details> <details><summary>Token ratchet and id-lint (exit codes captured before any pipe)</summary> ``` node scripts/check-skills-token-ratchet.mjs --self-test -> exit 0 ; "✓ check-skills-token-ratchet self-test: 79 cases pass." node scripts/check-skills-token-ratchet.mjs -> exit 0 ; "✓ … 84 authored bundle file(s) within their ceilings; 10 generator-owned file(s) measured, not ratcheted." skills/objectstack-automation/SKILL.md is 6465 tokens (ceiling 6465; headroom 0) skills/objectstack-automation/evals/integrations-declared.json is 1193 tokens (ceiling 1193; headroom 0) skills/objectstack-automation/references/integration-recipes.md is 2177 tokens (ceiling 2177; headroom 0) node scripts/pm/check-skill-id-lint.mjs -> exit 0 ; "✓ check-skill-id-lint: 104 file(s) clean" probe (a `#26709` line under references/, removed before commit) -> exit 1 ; "✗ … zz-probe.md: 1 issue-ID citation(s)" ``` </details> <details><summary>Derived gate families — `dispatch-gates --commands` on the actual diff, every command run, reconciled with `--ran`</summary> `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 64 commands for the 5 changed paths; each ran as `bash -c CMD > log 2>&1; code=$?` and was recorded as `CMD :: exit CODE`. `node scripts/pm/dispatch-gates.mjs --ran gates-ran-coded.txt --repo objectstack-ai/objectstack` -> exit 0: "✓ dispatch-gates --ran: 64 derived famil(ies) accounted for — 59 run, 5 NOT-MEASURED (5 DERIVED from a recorded exit 3)." Exit 0 (59), among them: `check:skill-docs`, `check:skill-refs` (`@objectstack/spec`), `check:doc-authoring`, `check:nul-bytes`, `check:pm-skill-id-lint`, `check:skill-identifier-liveness`, `check:skill-frame-sync`, `check:skill-compatibility`, `check:ratchet-remedy-authority`, `check:pm-dispatch-gates` (412 s), `check:changeset-gate-self-tests`, `check-empty-changeset --base origin/main`, `check-adr-0087-registration --base origin/main`, `check-changeset-no-major --base origin/main`, `check:published-files`, `check:merge-driver`, `check:cross-package-test-inputs`. Exit 3 (5, `PREREQUISITE NOT MET`, NOT MEASURED locally): `check:doc-formula-expressions`, `check:dts-closure`, `check:dual-build-cjs-loads`, `check:lean-entry-closure`, `check:sourcemap-no-sources-content`. Not derived, run on their own: `check:skills-token-ratchet` family is in the derived list; no `packages/spec` build was needed (no package source changed, and `check:skill-docs` / `check:skill-refs` run from source). </details> <details><summary>FlowSchema probe for Acceptance note 1</summary> ``` pnpm --filter @objectstack/spec exec tsx scratch-27223-connector-input-probe.ts (scratch file, deleted before commit) FlowSchema.safeParse success = true parsed connectorConfig.input = {"path":{"customer":{"dialect":"cel","source":"record.stripe_customer_id"}}} ``` </details> CI: the PR's check-runs are `in_progress` at report time; the PM reads their convergence. --- _Generated by [Claude Code](https://claude.ai/code/session_017xRBcVBKAFk1ntXSjvPt7U)_ --------- Co-authored-by: objectstack-dev <noreply@anthropic.com>
1 parent 4889e22 commit 0dd93f2

5 files changed

Lines changed: 308 additions & 67 deletions

File tree

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
"@objectstack/skills": patch
3+
---
4+
5+
Clause-②: no
6+
Metadata-protocol: none
7+
8+
objectstack-automation: integrations are declared, not coded. The `connector_action`
9+
node row no longer calls a connector "pre-built"; a new SKILL.md section states the
10+
declarative-connector-first rule (a vendor system over HTTP is a `connectors:` entry
11+
with `provider: 'openapi' | 'mcp' | 'rest'`, a packaged spec and `auth.credentialRef`,
12+
dispatched by `connector_action`; a vendor callback is an `api` flow), the five cases
13+
that justify platform code, and that a document the loader cannot take yet is a gap to
14+
name — never a reason to write a plugin. `references/integration-recipes.md` carries one
15+
recipe per archetype with the author / administrator credential hand-off and today's
16+
limits (`connectorConfig.input` is sent as written; no `oauth2`; no vendor-signed
17+
callbacks). `evals/integrations-declared.json` adds the DocuSign, Stripe-read and
18+
vendor-without-spec cases; the DocuSign case fails any answer that proposes a plugin.

‎scripts/check-skills-token-ratchet.mjs‎

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -441,7 +441,26 @@ export const CEILINGS = new Map([
441441
// AT the measurement. Maintainer ruling 2026-10-27 on the seat's decision
442442
// card 「skills 修复卡的教学和 eval 都撞 token 天花板,怎么付?」, tap A
443443
// verbatim and untranslated: 「A:先削重复,余额按实测提高」.
444-
['skills/objectstack-automation/SKILL.md', 6466],
444+
// 6466 -> 6465 (card #27223): the `connector_action` row no longer says
445+
// "pre-built", the "reserve custom code for edge-case integrations" line is
446+
// gone, and a new section "Integrations are declared, not coded" states the
447+
// declarative-connector-first rule, the five cases that justify platform
448+
// code and the gap-naming discipline. Paid down in the same file: the `http`
449+
// row's auth-routing tail (now in references/integration-recipes.md), the
450+
// Flow Variables example re-spelled one entry per line, a stale "(see the
451+
// approval example below)" pointer, the ADR-0032 sentence and the eval
452+
// pointer in References, and eight clause-level trims. Arithmetic in priced
453+
// bytes (stamp line excluded): 25864 -> 25860 (-4 net) -> ceil(25860 / 4) =
454+
// 6465; re-locked AT the measurement, headroom 0 tokens.
455+
// 6465 -> 6457 (same card, contract-review patch round): the five-condition
456+
// sentence gains "and then it is the platform's, filed as a card; the app
457+
// still writes none" (ADR-0019 D2), and the flow-type table's `api` row names
458+
// `secretRef` (or a literal `secret`) instead of `secret` alone. Paid down in
459+
// the same file: the CEL blockquote's "(the example apps use plain strings)",
460+
// three pointer lines' "Moved to", and five clause-level trims. Arithmetic in
461+
// priced bytes: 25860 -> 25827 (-33 net) -> ceil(25827 / 4) = 6457; re-locked
462+
// AT the measurement, headroom 0 tokens.
463+
['skills/objectstack-automation/SKILL.md', 6457],
445464
// 13783 -> 13892 (2026-08-31 app-repo-principles raise, see the block above).
446465
// The invariant-vs-transition-gate choice, placed in the field-conditional-rule
447466
// bullet list where the tool is actually picked. Both wrong picks are silent:
@@ -727,6 +746,14 @@ export const CEILINGS = new Map([
727746

728747
// objectstack-automation
729748
['skills/objectstack-automation/evals/approvals/test-revise-loop.md', 1329],
749+
// NEW FILE (card #27223): the three integration cases — DocuSign (fails any
750+
// answer proposing a platform plugin), Stripe read, vendor without a spec.
751+
// Pinned AT its landed count, no headroom. Re-measured in the same PR's
752+
// contract-review patch round (`clientSecret` -> `clientSecret:` in case 1's
753+
// must_not_contain, so a correct answer naming the absent field still passes):
754+
// 4772 -> 4773 bytes, 1193 -> 1194.
755+
// ceil($(wc -c < skills/objectstack-automation/evals/integrations-declared.json) / 4) == ceil(4773 / 4)
756+
['skills/objectstack-automation/evals/integrations-declared.json', 1194],
730757

731758
// objectstack-ui — the three authored eval files; its `references/react-blocks.md`
732759
// is generator-owned and carries no row here on purpose (see the boundary
@@ -1077,6 +1104,18 @@ export const CEILINGS = new Map([
10771104
// because a split that arrives with budget is a raise wearing a new path.
10781105
// The package deltas are the pointer lines and the four new file titles only.
10791106
['skills/objectstack-automation/references/examples-flows.md', 1436],
1107+
// NEW FILE (card #27223): one recipe per integration archetype — OpenAPI with
1108+
// a static key, remote MCP with a static header, plain REST, the vendor
1109+
// callback, a hand-written document for a spec-less vendor — plus the two
1110+
// "not yet" entries (OAuth, request signing) and the author / administrator
1111+
// credential hand-off. Pinned AT its landed count, no headroom. Re-measured
1112+
// in the same PR's contract-review patch round: recipe 1's `$ref` wording
1113+
// widened from "no `$ref` into other files" to "dereference every `$ref`
1114+
// offline, in-document ones included" (the loader skips a `$ref` parameter,
1115+
// request body or response, so the derived schema silently loses it):
1116+
// 8705 -> 8814 bytes, 2177 -> 2204.
1117+
// ceil($(wc -c < skills/objectstack-automation/references/integration-recipes.md) / 4) == ceil(8814 / 4)
1118+
['skills/objectstack-automation/references/integration-recipes.md', 2204],
10801119
// 5569 -> 5587 (card #26298, audit #26184 row A2): the approval config table
10811120
// gains `excludeSubmitter` (refused beside `onEmptyApprovers: 'auto_approve'`),
10821121
// +105 tokens. Paid down first: Best Practices 1 and 4 restated the Branching

0 commit comments

Comments
 (0)