Repository navigation
Commit 0dd93f2
skills(automation): integrations are declared, not coded — rule section, per-archetype recipes, three evals (#27322)
Fixes #27223
Clause-②: no
Metadata-protocol: none
## 维护者速读
- **改了什么**:`objectstack-automation` 技能包新增规则节「Integrations are declared,
not coded」(厂商系统走应用内声明的 `connectors:` 条目 + `connector_action`,回调走 `api`
流;五种情形才写平台代码;加载器暂不支持的文档是「待平台卡」的缺口,不是写插件的理由),`connector_action` 节点行不再写
"pre-built";新增 `references/integration-recipes.md`(六种原型各一份配方 +
作者/管理员凭据交接 + 两条「尚未支持」)和 `evals/integrations-declared.json`(DocuSign /
Stripe 读 / 无规格厂商三例,DocuSign 例对任何「写平台插件」的答案判失败)。
- **为什么改**:HotCLM 的开发 agent 读了技能后结论是平台要造 DocuSign 插件(维护者原话 「hotclm
的开发agent 就搞错了,说明 skills还是不对,要立卡完善」);技能里确有两句把它推向这个方向。
- **风险与回滚**:纯技能文本与 eval,无运行时改动;token 棘轮:SKILL.md 净 −1
token,两份新文件按落地值钉死(整包 +3370 token);回滚即 revert 本 PR。
- **先看**:① `skills/objectstack-automation/SKILL.md` 新节(约第 402
行起)——五种「写平台代码」的情形与缺口命名的措辞;② `references/integration-recipes.md` 「Holds
on every recipe」第二条(`connectorConfig.input` 按原样发送,逐记录 payload
今天不可声明)——这是实测到的运行时事实,也是下方 Acceptance notes 第 1 条发现;③
`evals/integrations-declared.json` 第 1 例的 must_not_contain。
- **两分钟验证**:`node scripts/check-skills-token-ratchet.mjs && node
scripts/pm/check-skill-id-lint.mjs`——预期两条均 ✓,automation 五个计价文件 headroom
全为 0。
- **需要你做的**:Tier H 技能面,由你手动落地(merge);卡号为何未写进技能正文见 What changes 末段。
## What changes
- `skills/objectstack-automation/SKILL.md` — the `connector_action` node
row no longer says "pre-built integration connector"; the "reserve
custom code for edge-case integrations" sentence is gone; a new section
**Integrations are declared, not coded** states the rule, the five cases
that justify platform code, and that a document the loader cannot take
yet (YAML, Swagger 2.0, `$ref`, OAuth, request signing) is a gap to name
against its platform card, never a reason to write code. The `http`
row's auth-routing tail moved into the recipes. Paid for in the same
file: net −1 token under the ratchet, headroom 0.
- `skills/objectstack-automation/references/integration-recipes.md`
(new, 150 lines) — one recipe per archetype: OpenAPI with a static key
(Stripe balance, nightly), remote MCP with a static header (GitHub),
plain REST (`provider: 'rest'`, `actionId: 'request'`), the vendor
callback (`api` flow with `secretRef`), a hand-written OpenAPI 3.x
document for a spec-less vendor, and the two **not yet** entries
(OAuth-protected, signature-authenticated). Each states what the app
author declares and what the customer's administrator supplies. It also
carries three limits measured on this base: `connectorConfig.input` is
handed to the provider as written (no CEL envelope or `{{ }}` hole is
evaluated — `service-automation/src/builtin/connector-nodes.ts:108`, and
`value-slot-map.ts` lists no connector slot), the `openapi` provider has
no `include` key (`connector-openapi/src/openapi-provider.ts:38-52` —
the trimmed document is the allowlist), and the inbound `api` door
verifies only `x-objectstack-signature` in the timestamped form
(`trigger-api/src/api-trigger.ts:163-178`, "has no unsigned branch"), so
a vendor signing its own header cannot post today.
- `skills/objectstack-automation/evals/integrations-declared.json` (new)
— the DocuSign case (expected: a declarative `openapi` instance over a
trimmed, packaged 3.x document converted offline from Swagger 2.0, an
`api` callback flow, three named gaps, marked blocked on OAuth;
`must_not_contain` refuses `registerConnector(`, `connector-esign`,
`provider: 'docusign'`, `implements Plugin`, `clientSecret`, `token:`),
the Stripe-read case (scheduled flow writing `crm_account` fields) and
the vendor-without-spec case.
- `scripts/check-skills-token-ratchet.mjs` — the SKILL.md row re-locked
at its measurement (6466 → 6465); two rows pin the new files at their
landed counts (1193, 2177).
- `.changeset/27223-integrations-declared.md` — `@objectstack/skills`
patch (the catalog ships in that package's `dist` via
`sync-catalog.mjs`).
**Card numbers are not in the skill text.** The card asks the recipes to
name the lifting cards. `pnpm check:pm-skill-id-lint` forbids `#NNN` in
every `.md` under `skills/` by maintainer ruling (2026-08-12, widened to
the published catalog on 2026-10-12) — measured on this tree: a probe
line `Waits on #26709 for include.` under `references/` reds the gate
(exit 1, naming the file and line). The recipes therefore name each gap
and say it is blocked on "its platform card"; the numbers live here:
`include` on the `openapi` provider and offline normalization → #26709;
`oauth2` in the declarative auth union → #26702; request signing →
#26710; the administrator-filled credential object → #27237 (in flight;
the recipes teach today's env-var `credentialRef` only).
**Bundle growth, with its authorization.** The published bundle grows by
3,370 tokens (two new files pinned at landed counts; SKILL.md −1). The
ratchet lands growth only against a maintainer ruling quoted in the PR
body; this one is the card's own directive, verbatim and untranslated:
「所以应该先处理 A/B 吧,并且要有 skills或者文档指导元数据开发者。前面agent还在建议我开发 DocuSign」 and
「hotclm 的开发agent 就搞错了,说明 skills还是不对,要立卡完善」(2026-10-28). The PM-set
budget — SKILL.md net tokens ≤ 0, recipes ≤ 150 lines, one eval file —
is met. The claim's "one entry in `references/_index.md`" does not
apply: that file is generator-owned (`gen:skill-refs` writes schema
pointers only); `check:skill-refs` passes with the hand-written
reference beside it, and SKILL.md links the recipes directly.
## Tests
No package source changed, so no `pnpm test` / `pnpm typecheck` is owed;
the edited gate script's own suite is its `--self-test` (79 cases pass,
below).
**Skill evals — what exists and what was run.** The root repo has no
eval runner: no `pnpm` script consumes `skills/*/evals/*.json`, and the
only eval gate in the tree, `ui/scripts/check-skill-eval-tokens.mjs`,
scans `ui/skills` alone. The card's "`pnpm` skill evals green" therefore
cannot be run as spelled. What was run instead, from a scratch replay
(not committed) of that gate's whole-token rule — every `must_contain`
token must occur as a whole token in the bundle's `.md` corpus — over
`skills/objectstack-automation/`:
<details><summary>Oracle, base bundle (a0aa8a1): the three new evals
FAIL — exit 1</summary>
```
corpus: 5 .md file(s): SKILL.md, evals/approvals/test-revise-loop.md, references/_index.md, references/examples-flows.md, references/state-machines-and-approvals.md
FAIL eval 1: 8 must_contain token(s), 5 untaught -> "defineConnector", "provider: 'openapi'", "type: 'api'", "Swagger 2.0", "blocked"
FAIL eval 2: 7 must_contain token(s), 1 untaught -> "provider: 'openapi'"
FAIL eval 3: 5 must_contain token(s), 3 untaught -> "provider: 'openapi'", "operationId", "type: 'basic'"
ORACLE: 9 must_contain token(s) untaught
```
</details>
<details><summary>Oracle, this branch (2993d73): the three new evals
PASS — exit 0</summary>
```
corpus: 6 .md file(s): SKILL.md, evals/approvals/test-revise-loop.md, references/_index.md, references/examples-flows.md, references/integration-recipes.md, references/state-machines-and-approvals.md
PASS eval 1: 8 must_contain token(s), 0 untaught
PASS eval 2: 7 must_contain token(s), 0 untaught
PASS eval 3: 5 must_contain token(s), 0 untaught
ORACLE: every must_contain token is taught by the bundle
```
</details>
NOT MEASURED: a live LLM answering arm per eval (base text vs new text)
— no runner exists and a self-graded run by the authoring seat is not
evidence; the DocuSign case's refusal of a plugin answer is enforced by
its `must_not_contain` list, not by a measured model run.
## Acceptance notes
1. **Finding, class c (metadata-authoring trap), for the seat to file.**
`connectorConfig.input` on a `connector_action` node is handed to the
provider handler as written
(`packages/services/service-automation/src/builtin/connector-nodes.ts:108`:
`handler((cfg.input ?? {}) …)`); `builtin/value-slot-map.ts` resolves
`fields`, `subflow`/`map` `input`, `script` `inputs`, `screen`
`defaults`, `notify` `templateData` and `http` `body`, never connector
`input`. The authoring door accepts a CEL envelope there:
`FlowSchema.safeParse` of a flow whose `connectorConfig.input` is `{
path: { customer: { dialect: 'cel', source: 'record.stripe_customer_id'
} } }` answers `success = true` and keeps the envelope verbatim
(measured with `tsx` against `packages/spec/src/automation/flow.zod.ts`
on this tree). The `openapi` handler then interpolates the path from
`String(value)`, so the vendor receives an object stringified. Reach:
`os validate` runs `FlowSchema`; producer: any app author's stored flow
definition. Dedupe words: `connector_action input value slot CEL
envelope verbatim`. The recipes state the limit; they do not work around
it.
2. **The card's "`pnpm` skill evals" does not exist for the root
catalog** — the eval fixtures under `skills/*/evals/` have no mechanical
consumer in this repository; `ui/` has one for its own bundle.
Observation for the PM (the acceptance wording), not a card from this
seat.
3. **Pre-existing, observation only**: under the same whole-token rule,
two existing evals in `evals/flows-triggers-approvals.json` carry
untaught tokens on base as well as on this branch — eval 7
(`excludeSubmitter: true`) and eval 8 (`type: 'screen'`, `type: 'file'`,
`signed_copy`). No root gate asserts the rule; carrier: none.
4. `#25747` (the DocuSign instance the card cites as open) is closed
`not_planned` as of 2026-10-28 — no effect on the premise, which was
re-verified on `origin/main` a0aa8a1: `pre-built integration connector`
at `SKILL.md:89`; `defineConnector|provider: 'openapi'` → 0 files under
`skills/*/references` and `skills/*/evals`; `Declaring a provider-bound
instance` at `content/docs/automation/connectors.mdx:40`.
5. `references/examples-flows.md:49-55` (the plugin-registered `slack`
connector in a `connector_action`) is outside the declared surface and
was not edited; it stays a legitimate shape, and the rewritten
`connector_action` row says so ("or a plugin's").
6. The `runAs: 'system'` paragraph in SKILL.md says a system sweep must
pin its organization but names no column, and the catalog's own
scheduled example carries no predicate; the Stripe recipe mirrors the
taught shape and adds a one-line reminder. Read-only inference; noted,
not filed.
7. Five derived families exit 3 locally (`PREREQUISITE NOT MET`: they
read built `dist/`, which this diff never builds) —
`check:doc-formula-expressions`, `check:dts-closure`,
`check:dual-build-cjs-loads`, `check:lean-entry-closure`,
`check:sourcemap-no-sources-content`. They are pulled in by the
`scripts/` touch, read none of the changed paths, and run in CI after
the build; recorded as NOT MEASURED locally.
## Verification
Head `2993d737` (tree identical to the readings below; no commit
followed them).
<details><summary>skills/** readings — the two required before/after
counts (lines and tokens)</summary>
| File | Lines before → after | Tokens before → after (ratchet
convention) |
|:--|:--|:--|
| `skills/objectstack-automation/SKILL.md` | 451 → 451 | 6466 → 6465
(ceiling re-locked at 6465, headroom 0) |
| `skills/objectstack-automation/references/integration-recipes.md` | 0
→ 150 | 0 → 2177 (pinned, headroom 0) |
| `skills/objectstack-automation/evals/integrations-declared.json` | 0 →
35 | 0 → 1193 (pinned, headroom 0) |
| whole package (sum of every `skills/*/SKILL.md`) | 4615 → 4615 |
bundle total (whole shipped tree) 215826 → 219195 |
</details>
<details><summary>Token ratchet and id-lint (exit codes captured before
any pipe)</summary>
```
node scripts/check-skills-token-ratchet.mjs --self-test -> exit 0 ; "✓ check-skills-token-ratchet self-test: 79 cases pass."
node scripts/check-skills-token-ratchet.mjs -> exit 0 ; "✓ … 84 authored bundle file(s) within their ceilings; 10 generator-owned file(s) measured, not ratcheted."
skills/objectstack-automation/SKILL.md is 6465 tokens (ceiling 6465; headroom 0)
skills/objectstack-automation/evals/integrations-declared.json is 1193 tokens (ceiling 1193; headroom 0)
skills/objectstack-automation/references/integration-recipes.md is 2177 tokens (ceiling 2177; headroom 0)
node scripts/pm/check-skill-id-lint.mjs -> exit 0 ; "✓ check-skill-id-lint: 104 file(s) clean"
probe (a `#26709` line under references/, removed before commit) -> exit 1 ; "✗ … zz-probe.md: 1 issue-ID citation(s)"
```
</details>
<details><summary>Derived gate families — `dispatch-gates --commands` on
the actual diff, every command run, reconciled with `--ran`</summary>
`node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` derived 64 commands for the 5 changed paths;
each ran as `bash -c CMD > log 2>&1; code=$?` and was recorded as `CMD
:: exit CODE`.
`node scripts/pm/dispatch-gates.mjs --ran gates-ran-coded.txt --repo
objectstack-ai/objectstack` -> exit 0: "✓ dispatch-gates --ran: 64
derived famil(ies) accounted for — 59 run, 5 NOT-MEASURED (5 DERIVED
from a recorded exit 3)."
Exit 0 (59), among them: `check:skill-docs`, `check:skill-refs`
(`@objectstack/spec`), `check:doc-authoring`, `check:nul-bytes`,
`check:pm-skill-id-lint`, `check:skill-identifier-liveness`,
`check:skill-frame-sync`, `check:skill-compatibility`,
`check:ratchet-remedy-authority`, `check:pm-dispatch-gates` (412 s),
`check:changeset-gate-self-tests`, `check-empty-changeset --base
origin/main`, `check-adr-0087-registration --base origin/main`,
`check-changeset-no-major --base origin/main`, `check:published-files`,
`check:merge-driver`, `check:cross-package-test-inputs`.
Exit 3 (5, `PREREQUISITE NOT MET`, NOT MEASURED locally):
`check:doc-formula-expressions`, `check:dts-closure`,
`check:dual-build-cjs-loads`, `check:lean-entry-closure`,
`check:sourcemap-no-sources-content`.
Not derived, run on their own: `check:skills-token-ratchet` family is in
the derived list; no `packages/spec` build was needed (no package source
changed, and `check:skill-docs` / `check:skill-refs` run from source).
</details>
<details><summary>FlowSchema probe for Acceptance note 1</summary>
```
pnpm --filter @objectstack/spec exec tsx scratch-27223-connector-input-probe.ts (scratch file, deleted before commit)
FlowSchema.safeParse success = true
parsed connectorConfig.input = {"path":{"customer":{"dialect":"cel","source":"record.stripe_customer_id"}}}
```
</details>
CI: the PR's check-runs are `in_progress` at report time; the PM reads
their convergence.
---
_Generated by [Claude
Code](https://claude.ai/code/session_017xRBcVBKAFk1ntXSjvPt7U)_
---------
Co-authored-by: objectstack-dev <noreply@anthropic.com>1 parent 4889e22 commit 0dd93f2
5 files changed
Lines changed: 308 additions & 67 deletions
File tree
- .changeset
- scripts
- skills/objectstack-automation
- evals
- references
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
441 | 441 | | |
442 | 442 | | |
443 | 443 | | |
444 | | - | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
445 | 464 | | |
446 | 465 | | |
447 | 466 | | |
| |||
727 | 746 | | |
728 | 747 | | |
729 | 748 | | |
| 749 | + | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
730 | 757 | | |
731 | 758 | | |
732 | 759 | | |
| |||
1077 | 1104 | | |
1078 | 1105 | | |
1079 | 1106 | | |
| 1107 | + | |
| 1108 | + | |
| 1109 | + | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
| 1117 | + | |
| 1118 | + | |
1080 | 1119 | | |
1081 | 1120 | | |
1082 | 1121 | | |
| |||
0 commit comments