[objectstack-fleet]

Requested by Steve · project thread

Fixes #26515

Clause-②: no
Metadata-protocol: none

维护者速读

• 改了什么:objectstack-automation 技能的评测文件里 5、7、8 三条理想答案改写到与各自的断言一致;token 棘轮行按实测抬到 2734。
• 为什么改:三条理想答案自相矛盾(5 把禁写的错误标签字面写进了答案;7 缺它要求的 type: 'position' 审批人并写了禁写的 positions 检查;8 缺它要求的 type: 'screen'),按断言评分时正确答案会被判错。
• 风险与回滚:只改评测文本、一行棘轮上限和一个 @objectstack/skills patch changeset,不碰 SKILL.md 教学与运行时;回滚即 revert 本 PR。
• 先看:skills/objectstack-automation/evals/flows-triggers-approvals.json 的三个 expected_output hunk,再看 scripts/check-skills-token-ratchet.mjs 的棘轮行注释。
• 两分钟验证:在 PR 分支跑下方 Verification 第一条一行脚本,预期输出 11 fixtures, 0 contradictions。
• 需要你做的:受管面(skills/**,Tier H),需 GOVERNED_APPROVERS 的一次 APPROVED review;之后席位落地。

What changes

Three ideal answers in skills/objectstack-automation/evals/flows-triggers-approvals.json contradicted the assertions that grade them. Each fixture was judged against the teaching on origin/main (SKILL.md § Failure routing & runAs, § Who may START a flow, § Flow Types, and references/state-machines-and-approvals.md § Approver Types); in all three the teaching and the assertions agree, and the ideal answer was the side that was wrong, so only the ideal answers move:

• eval 5 — the answer spelled the negated error-label string verbatim. It now says it in words (a bare error label routes nothing) and keeps type: 'fault' as the routing form.
• eval 7 — the answer never spelled the type: 'position' approver its assertions require and spelled the negated positions check verbatim. It now names approvers: [{ type: 'position', value: 'finance_manager' }] (the prompt's position) and says the negation in words.
• eval 8 — the answer never spelled the type: 'screen' flow kind its assertions require. It now opens with the flow kind.

No fixture was added or removed; the file keeps 124 lines (net 0 lines in skills/**). No must_not_contain string appears inside any ideal answer.

Token ratchet (scripts/check-skills-token-ratchet.mjs): the row for this file moves 2724 → 2734, pinned at the measurement, with the house-style comment. Paid down first: evals 5 and 7 were shortened by 25 and 7 bytes; the approvers line eval 7 owed costs more than that, so the net is +41 bytes (10893 → 10934), ceil(10934 / 4) = 2734. Headroom was 3 bytes, 0 tokens.

Readings (lines; tokens as the ratchet counts them, ceil(utf8 bytes / 4)):

scope	lines before → after	tokens before → after
evals/flows-triggers-approvals.json (the edited file)	124 → 124	2724 → 2734
whole package skills/objectstack-automation/** (6 tracked files)	1185 → 1185	17707 → 17717

Why

A shipped skill's evals grade a correct answer wrong when the ideal answer fails its own must_contain / must_not_contain set. Fixing the ideal answers restores the evals as a usable grader without changing what the skill teaches.

Tests

No test file changed; no forced test edits. The eval file's own consistency check is the acceptance (below).

Acceptance notes

• The card's premise was re-checked on origin/main 7c34e5d (after the same-day churn f986702 on this package): the file now holds 11 fixtures (eval 11 landed with skills(automation): the runAs 'system' guidance in objectstack-automation never says a system flow, hook or action body is refused identity-assignment writes #26502), and the parse-and-check found exactly the four contradictions the card names (5: negated string present; 7: type: 'position' missing and negated string present; 8: type: 'screen' missing). Ids 1–4, 6, 9–11 clean.
• skills/** is a Tier H governed surface: this PR stays draft until a GOVERNED_APPROVERS approval.
• Changeset: .changeset/26515-automation-eval-fixtures.md, a patch for @objectstack/skills — the catalog ships as that package (packages/skills/scripts/sync-catalog.mjs copies skills/** into its dist), so the rewritten fixtures publish; the skip-changeset label first applied at creation was removed on the seat's rework (head 72cb4e2).

Verification

Head 72cb4e28 (5c31edd = the fix; 72cb4e2 adds only the changeset). Parse-and-check of every fixture in the file (the card's acceptance one-liner):

node -e '
const f="skills/objectstack-automation/evals/flows-triggers-approvals.json";
const j=JSON.parse(require("fs").readFileSync(f,"utf8"));let bad=0;
for(const e of j.evals){const o=e.expected_output;
 for(const s of e.assertions.must_contain||[]) if(!o.includes(s)){bad++;console.log(`id ${e.id}: must_contain missing: ${s}`)}
 for(const s of e.assertions.must_not_contain||[]) if(o.includes(s)){bad++;console.log(`id ${e.id}: must_not_contain present: ${s}`)}}
console.log(`${j.evals.length} fixtures, ${bad} contradictions`)'

Output on origin/main 7c34e5d (before) and on 5c31edd (after)

# before
id 5: must_not_contain present: label: 'error'
id 7: must_contain missing: type: 'position'
id 7: must_not_contain present: current_user.positions
id 8: must_contain missing: type: 'screen'
11 fixtures, 4 contradictions

# after
11 fixtures, 0 contradictions

Skill gates (all exit 0)

pnpm --filter @objectstack/spec check:skill-docs      :: exit 0   ✅ Skill docs in sync
pnpm --filter @objectstack/spec check:skill-refs      :: exit 0   ✅ 9 generated files in sync with packages/spec
pnpm --filter @objectstack/spec check:skill-examples  :: exit 0   ✅ 266 prose examples type-check across 3 surface(s)
node scripts/check-skills-token-ratchet.mjs           :: exit 0   ✓ 81 authored bundle file(s) within their ceilings

Derived gate families (dispatch-gates --commands, 49 families) and --ran reconciliation

All 49 derived families run locally on head 5c31edd, each exit code captured before any pipe; dispatch-gates --ran reconciliation: ✓ dispatch-gates --ran: 49 derived famil(ies) accounted for — 49 run, 0 NOT-MEASURED (a DERIVED zero — all 49 recorded an exit code and none of them is 3). check:pm-dispatch-gates (fast tier, ~8 min): exit 0, ✓ dispatch-gates self-test (fast tier): 1935 cases pass; 7 slow section(s) deferred..

On 72cb4e2 the derivation grows to 60 families (the changeset adds 11: the four changeset gates with --base origin/main and their self-tests, check:published-files, check:closed-ceremony-text, check:pm-changeset-deadline-census, the two release-tooling self-tests); the 11 and the token ratchet ran on 72cb4e2, all exit 0; --ran on the combined record: ✓ dispatch-gates --ran: 60 derived famil(ies) accounted for — 60 run, 0 NOT-MEASURED (a DERIVED zero — all 60 recorded an exit code and none of them is 3).

Human Minimum

• 1. Schemas start as Zod; packages/spec holds no runtime logic. (not touched)
• 2. Every name I added uses the casing for its kind.
• 3. No formatter ran; pnpm lint passes. (CI's Lint & Repo Gates runs it; no formatter ran)
• 4. One issue, one branch off an up-to-date main, one pull request.
• 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. (.changeset/26515-automation-eval-fixtures.md, patch @objectstack/skills)
• 6. packages/spec touched: its generated artifacts are regenerated and committed. (not touched)
• 7. No hand edits to content/docs/references/, packages/*/CHANGELOG.md or content/docs/releases/.
• 8. Title, body and comments are in English.
• The Before Submitting list in CONTRIBUTING.md is done.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ADHkmT8qQiLPD3BRTdVxGH

---

Generated by Claude Code

[objectstack-fleet]

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 5c31edd8d885936e534036aade04bb3d8d176dfe
Local-runs: none
Scope: skills/objectstack-automation/evals/flows-triggers-approvals.json + scripts/check-skills-token-ratchet.mjs, skills/objectstack-automation/SKILL.md, skills/objectstack-automation/references/state-machines-and-approvals.md, packages/spec/src/automation/flow.zod.ts, packages/spec/src/automation/approval.zod.ts

Director seat (summon #52, ledger objectstack#12708), summoned by the domain:skills#1 seat for this PR. Read-only at the head above: the diff, card #26515, the dev report (6440431020), the head file as stored on the platform, the cited teaching on origin/main 03e898c1, the head's check runs. dispatch-gates --tier on the paths: skills/** clause ① ⇒ the rules face, CONTRACT_REVIEW_TIER, no one-line exit; served at CONTRACT_REVIEW_TIER. The seat records; it does not approve, ready, queue or merge.

① Derived judgments

• The side that moved is the side that was wrong. For each fixture the teaching the card names was read and agrees with the assertions, not with the old ideal answer: eval 5 — SKILL.md § Failure routing & runAs ("type: 'fault' is what routes — a label: 'error' alone does nothing"); eval 7 — § Who may START a flow (requiredPermissions on the flow, "never on current_user.positions inside it") and the approvals reference § Approver Types with approvers: [{ type: 'position', value: … }] at state-machines-and-approvals.md:98; eval 8 — § Flow Types lists screen, and flow.zod.ts:1102 enumerates 'screen' as a flow type. Only the three expected_output strings move; no assertion, prompt or teaching line changes.
• The file holds against its own grader at the head. Reading the stored head file: 11 fixtures, every must_contain string present in its own expected_output, no must_not_contain string present — 0 contradictions (the card's acceptance; the dev's one-liner reports the same). Before, on main: the four contradictions the card names, no others.
• Eval 7's value: 'finance_manager' (dev open question 1): the prompt names that position, the assertion asks only for type: 'position', and the approvals reference's own examples always carry a value. Option A is the right model of the prompt; a bare type would teach an approver with no target. No must_not_contain string is introduced by it.
• Eval 7's refusal pair stays as taught: excludeSubmitter: true beside onEmptyApprovers other than 'auto_approve', which approval.zod.ts refuses at the parse (the #25609 test spells the rule). Consistent.
• Acceptance read as every fixture in the file (dev open question 2): the card's own wording is "every fixture in the file"; eval 11 (landed with skills(automation): the runAs 'system' guidance in objectstack-automation never says a system flow, hook or action body is refused identity-assignment writes #26502) is in the set and clean. Option A.
• Ratchet row by ruling A, at the measurement. check-skills-token-ratchet.mjs row 2724 → 2734 with the arithmetic in the comment and the maintainer's tap quoted verbatim (「A:先削重复,余额按实测提高」). Checked against the stored head file: 10,934 bytes (from 10,893 on main), ceil(10934 / 4) = 2734 under the script's own ceil(utf8 bytes / 4) convention — pinned AT the measurement, headroom 0 tokens, the pay-down (evals 5 and 7 shortened) stated. Class ③ holds.

② Semver level

skills/** is not a published npm package and scripts/** is root tooling: skip-changeset, Clause-②: no, Metadata-protocol: none in the body. No level owed. Correct. (Note for the ledger: #26566 on the same file carried a @objectstack/skills patch changeset on the reading that the catalog ships in that package's dist; this PR's body reads the surface the other way. Both readings are on main now; Check Changeset is green on this head, so nothing here blocks. Which reading the lane keeps is a lane-seat question, not a must-fix on eval text.)

③ Boundary flags

• Governed path (skills/**, Tier H): GOVERNED_APPROVERS approval present — os-zhuang APPROVED at 5c31edd8 (2026-10-27T11:09Z). The PR is draft; readying and landing stay with the lane seat.
• Check runs on 5c31edd8 at the time of this record: 17 success (incl. Governed Surface Queue Guard, Check Changeset, Lint · repo & tooling gates, Lint · CI, release & engine gates, Lint · ESLint, Lint · scoped · tooling self-tests, the PR-body guards, Build Core), 16 expected skips, and 9 still in_progress (Test Core 3 shards, Lint · scoped · query-options erasure, Lint · scoped · slot lookup & corpus, Lint · docs & code guards, Lint · PM dispatch-gates, Type Check · source gates, Type Check · consumer gates). Nothing red. The lane seat's ACCEPT reads the required contexts at landing; a red among the pending ones re-opens this record.
• The dev's out-of-scope note (SKILL.md row 6358 vs plain ceil 6363): read, not a finding here — that row is untouched by this PR and the gate is green.
• Read-only shape held: the parse-and-check above is a reading of the stored head file, not a build or a test run.

④ Design judgment

• evals/flows-triggers-approvals.json: an ideal answer that spells its own must_not_contain string, or omits a must_contain one, grades every correct candidate wrong; the fix restores the three fixtures as a grader without moving what the skill teaches. The rewritten answers say the negation in words (the rule the lane's dispatch orders for skills(objectstack-automation): teach the read-only record_link screen field, with an eval prompt (#26304 follow-up, #26184) #26347 / skills(automation): teach the record-anchored approval lock read (GET /approvals/records/:object/:recordId/lock, { locked } for every reader) with one eval prompt #26362 already carry) and name the form the assertion wants. Eval 8 now opens with the flow kind, which is also the first thing a reader of the prompt would state. Holds against the cited teaching and the card's scenario.

Implemented-by: claude/project-thread-ycobo0
Reviewed-by: session_01ADHkmQr732toX135tyo5zs

VERDICT: PASS

---

Generated by Claude Code

[objectstack-fleet]

维护者速读

• 改了什么:objectstack-automation 技能的审批 eval 文件里,第 5、7、8 条的「理想答案」改成与各自断言一致;另补一个 @objectstack/skills patch changeset,并按裁决 A 把该文件的 token 上限抬到实测 2734。
• 为什么改:这三条的理想答案本身就过不了自己的断言,任何正确回答都会被判错,技能文本退化时也会被这些旧红掩盖。
• 风险与回滚:只改 eval 文本与棘轮一行,不改技能教学内容、不改任何代码或 schema;回滚 = revert 本 PR。
• 先看:skills/objectstack-automation/evals/flows-triggers-approvals.json 的三处 expected_output。
• 两分钟验证:对该文件逐条检查 must_contain 都出现、must_not_contain 都不出现 —— 11 条 0 矛盾。
• 需要你做的:无;os-zhuang 已批准,席位按授权批准落地。

席位意见:ACCEPT。

席位核对

ACCEPT · domain:skills#1 · session_01ADHkmSrRBdu9MdC6fc33Uu · head 72cb4e28 · 2026-10-27T11:33Z

1. 形态 — 3 files: the eval JSON (3 strings), the ratchet row 2724 → 2734 (ruling A, pinned at the measurement), and .changeset/26515-automation-eval-fixtures.md (@objectstack/skills patch — packages/skills/scripts/sync-catalog.mjs copies skills/** into the package dist that ships; same reading as docs(skills): a system flow is refused identity-assignment writes in objectstack-automation #26566). check-governed-merges --pr 26636: GOVERNED, tier H.
2. 前提实测 — seat parse-and-check on the head file: 11 fixtures, 0 contradictions (main: the four the card names). Teaching agrees with the assertions: approvals reference approvers: [{ type: 'position', … }], flow type 'screen' in flow.zod.ts.
3. 门禁 check-runs — head 72cb4e28: 30 success, 16 skipped, 0 failing.
4. 接受的偏差 — none. Contract review at CONTRACT_REVIEW_TIER: PASS, record 6440507235 (head 5c31edd8; the only later commit adds the changeset file).
5. 范围外去向 — the director's ledger note on the two changeset readings is settled by this PR: the catalog ships in @objectstack/skills, so skills/** edits carry a patch changeset. Skills-docs: none (fixtures only).
6. 落地条款 — Tier H: os-zhuang (GOVERNED_APPROVERS) APPROVED 5666089698; approvals do not go stale on push (ruleset dismiss_stale_reviews_on_push: false). The seat arms auto-merge and the queue lands it.

---

Generated by Claude Code