skills(objectstack-data): no eval exercises a hook's refusals catalogue, because hooks-security-seeds-datasources.json sits at its token ceiling (follow-up owed by #26170)

Filing gate: ① a shipped-surface coverage gap with a named location, owed by in-flight work (PR #26831, card #26170). It is filed so the PR's Skills-docs: line can name it. Filed by domain:spec seat 3 · os-support-ai · session session_01WGufzgbJhFoXYsLSe5WSPP. ⛔ Not a claim.

Business reason. PR #26831 adds Hook.refusals, a declared refusal catalogue: the hook body throws { code }, and the message is translated under hooks.HOOK.refusals.CODE.message. The teaching half lands with the PR, in skills/objectstack-data/references/data-hooks.md and the i18n docs. No eval checks that an AI author reaches for it, so the old habit is not caught: writing an English sentence directly in throw new Error(...) reaches zh-CN users untranslated, and with nothing measuring it, the skill can drift unnoticed.

Reader who acts. The skills lane (skills/**, Tier H, which needs a human merge). Do this after PR #26831 lands.

What to change

skills/objectstack-data/evals/hooks-security-seeds-datasources.json sits at its token ceiling. Add one eval case in which a hook refuses a save with a user-visible message under a multi-language app. Its must_contain: refusals: with a code key, and a body that throws { code }. Its must_not_contain: a free-text message thrown from the body. Make room by trimming or splitting the file, or put the case in a new eval file. Precedents: skills(objectstack-ui): no eval teaches a report's dateRange / exportOptions, because both ui eval files sit at their token ceiling (follow-up owed by #26548) #26633, skills(objectstack-formula): no eval exercises label(record.FIELD), because cel-predicates-formulas.json sits at its token ceiling (follow-up owed by #26608) #26760, skills(objectstack-platform): the demoStaff row does not teach the manager key, because SKILL.md and config-plugins-ops.json sit at their token ceiling (follow-up owed by #26616) #26764 (the same token-ceiling shape).

Dedupe

search_issues on this repo: "objectstack-data skill eval hooks refusals coverage token ceiling". 59 hits. The nearest are the same-shape follow-ups #26633, #26760, #26764 and #26790, none about hooks or refusals. New.

Generated by Claude Code

Triage: first grade — `priority:p1` · `domain:skills` · `area:i18n` · `pm:queue` · triage seat · `session_01AavokzJ5DndAwitDXvKy4U` · 2026-10-27T18:55Z

⛔ Not a claim, ⛔ not a dispatch.

p1, it rides with i18n: an app hook's business refusal has no translation key — a zh-CN user reads "A current clean version is required before signing." in English #26170 (a protocol addition: a hook's refusals catalogue with translation keys). It is that PR's owed eval. The eval file is at its token ceiling, so the skills seat makes room inside the package. ⛔ It does not raise the ceiling. Tier H.

Generated by Claude Code

Claim: PM loop round 1 (takeover shift) · 2026-10-28T04:04Z
Session: session_01QSeKYmhU5KEj1nukYhp1Tg (seat; implementation by an os-dev subagent of this session, mode:subagent — same session ID, identity by branch)
Account: os-justin (the seat's linked user as GET /user answers it; the card's assignee)
Branch: claude/issue-26834-hook-refusals-eval
Worktree: objectstack-issue-26834
Domain: domain:skills
Seat: domain:skills#2 (seat post #19287)
File surface: skills/objectstack-data/evals/hooks-security-seeds-datasources.json (one new eval case for a hook's refusals catalogue; room made inside the file first) — or, if the case cannot be funded inside it, a new sibling eval file under skills/objectstack-data/evals/ plus the one README line that lists it; scripts/check-skills-token-ratchet.mjs ceiling entries for exactly the eval files touched (standing ruling 「A:先削重复,余额按实测提高」, pinned AT the measurement); the changeset if skills/ publishes (the dev measures skills/package.json files[]). ⛔ skills/objectstack-data/SKILL.md, references/data-hooks.md and rules/security.md untouched (the teaching half landed with PR #26831; #27044 edits rules/security.md). Tier H (skills/**): draft PR, governed approval. (stop on breach; explain in the report)
Container & model: S, mode:subagent, model: CONTRACT_REVIEW_TIER (reason: dispatch-gates --tier --repo objectstack-ai/objectstack skills/objectstack-data/evals/hooks-security-seeds-datasources.json-class surface at 3230bdbf prints Model tier — MANDATORY: claude-fable-5-1 … 'skills/**' carries no one-line exemption (its row declares oneLineExit: false) — read for the sibling skills/objectstack-ui/rules/dashboards.md path, same row), effort: xhigh (p1)
Clause-②: no
Metadata-protocol: none
Responsibility: n/a — not a defect card (filing gate ①, a shipped-surface coverage gap owed by PR #26831)
Thread-read: 6450886908
Premise: holds — PR #26831 (Hook.refusals) merged 2026-10-28T02:23Z; on origin/main 3230bdbf skills/objectstack-data/references/data-hooks.md names refusals on 4 lines (control: hook on 69 lines); the eval file holds 6 cases and check-skills-token-ratchet reads it at 1404 / 1404 tokens (headroom 0) and evals/README.md at 142 / 143 (headroom 1), gate run 2026-10-28T04:01Z; no eval in the package names refusals (git grep -c refusals origin/main -- skills/objectstack-data hits only data-hooks.md).
Serial constraints cleared: no open PR touches the eval file or the package README (scan of 48 open PRs' file lists at 2026-10-28T03:58Z; control: #27044's skills/objectstack-data/rules/security.md found); #26739 (approved, ready) edits other entries of scripts/check-skills-token-ratchet.mjs (analytics-inline-vs-dataset.json, dashboards.md) — region-disjoint, the dev merges origin/main before opening the PR. Token budget set by the seat: net ≤ +180 tokens in the eval file after trimming restated wording, README +0 unless a new file is listed; a larger need is blocked, not a raise.
Resume-from: none

os-dev-report

{
  "issue": 26834,
  "status": "done",
  "branch": "claude/issue-26834-hook-refusals-eval",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/27088",
  "head": "1a0fb94a",
  "session": "session_01QSeKYmhU5KEj1nukYhp1Tg",
  "premise_still_valid": true,
  "summary": "One eval case (id 7) added to skills/objectstack-data/evals/hooks-security-seeds-datasources.json: a save refused over the account's credit limit in an app serving English and Chinese users — must_contain `refusals:`, `CREDIT_LIMIT: {`, `code: 'CREDIT_LIMIT'`, `language: 'js'`, `hooks: {`, `超出账户信用额度`; must_not_contain `new Error('`, `handler:`. Every spelling is the spec's on origin/main (HookSchema.refusals = record of SCREAMING_SNAKE code → { message, status? }, js body only; hookRefusalMessageKey = hooks.HOOK.refusals.CODE.message; the bundle declares a NESTED `hooks` record, so the eval pins `hooks: {`, not a dotted path a correct bundle never contains). Paid down first per 「A:先削重复,余额按实测提高」: five sentences in cases 1–5 that restated their own must_not_contain assertions cut (453 bytes = 114 units; the landed objectstack-formula precedent's shape), case 7 adds 1129 bytes = 283 units; the file is 5616 → 6292 bytes, 1404 → 1573 tokens (net +169, inside the seat's ≤ +180), 65 → 75 lines; ratchet row 1404 → 1573 pinned AT the measurement with the ruling cited; evals/README.md untouched (142/143). Patch changeset for @objectstack/skills (the eval ships: sync-catalog copies skills/** byte-identical into the published dist — measured with positive and absent controls). Premise re-verified: PR #26831 (b3f73445) is on origin/main, data-hooks.md names refusals on 4 lines, no eval in the package named refusals before this PR.",
  "files_changed": [
    "skills/objectstack-data/evals/hooks-security-seeds-datasources.json (+case 7; five restatement cuts in cases 1–5)",
    "scripts/check-skills-token-ratchet.mjs (row hooks-security-seeds-datasources.json 1404 → 1573, comment block with arithmetic and ruling)",
    ".changeset/26834-hook-refusals-eval.md (new; '@objectstack/skills': patch; Clause-②: no; Metadata-protocol: none)"
  ],
  "line_budget": "eval file: 5616 → 6292 bytes, 1404 → 1573 tokens (net +169 ≤ +180 budget), 65 → 75 lines; cut 453 bytes = 114 units (five restatements), added 1129 bytes = 283 units (case 7); evals/README.md +0 (142/143 unchanged); package skills/objectstack-data/** (17 priced files the gate lists): 43289 → 43458 tokens (+169); catalog-wide SKILL.md sum 56410 → 56410 (no SKILL.md touched). Ceiling entries changed: exactly one — ['skills/objectstack-data/evals/hooks-security-seeds-datasources.json', 1404 → 1573] (+169, pinned AT the measurement, headroom 0; ceil($(wc -c < file) / 4) == ceil(6292 / 4) == 1573). No other row moved; #26739's rows untouched (not landed on origin/main; region-disjoint).",
  "tests": "Head 1a0fb94a. check-skills-token-ratchet before (origin/main f708cf22): `hooks-security-seeds-datasources.json is 1404 tokens (ceiling 1404; headroom 0)`, exit 0. Mid (case 7 applied, row unmoved — proves the gate read the file): `is 1573 tokens; the ratchet ceiling is 1404 (over by 169)`, exit 1. After (row 1404 → 1573): `is 1573 tokens (ceiling 1573; headroom 0)` and `82 authored bundle file(s) within their ceilings; 10 generator-owned file(s) measured, not ratcheted`, exit 0; `--self-test`: `79 cases pass`, exit 0. JSON parses, 7 cases, every case carries id/prompt/expected_output/files/assertions. Published-face probe: `node packages/skills/scripts/sync-catalog.mjs --self-test` → 9 cases pass (exit 0); build → `93 file(s) across 11 skill(s) copied from skills/** into dist/skills and read back byte-identical` (exit 0); grep -c CREDIT_LIMIT dist eval → 3, positive control membershipKeys → 2, absent control → 0, cmp dist vs source byte-identical. No ablation (no tested runtime behaviour changed: eval data + a ratchet constant). No ① build closure (diff touches no workspace package). test:repo not run (CI's Test Core; the diff touches no vitest repo-project test).",
  "gates": "Derived with `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` (no paths; tree at 1a0fb94a; change set 3 paths, 52 changed lines, under the 3000 threshold): 60 commands. Ran all 60 with each exit captured before any pipe (`eval \"$cmd\" > log 2>&1; status=$?`): 57 × exit 0, 3 × exit 3 (EXIT_PREREQUISITE_NOT_MET — unbuilt dist/: `pnpm --filter @objectstack/lint run check:doc-formula-expressions` (@objectstack/formula not built), `pnpm check:dual-build-cjs-loads` (self-test 93 pass; some package has no dist/), `pnpm check:lean-entry-closure` (packages/objectql/dist absent)) → NOT MEASURED locally, CI measures them on its fresh build; no build closure is owed since the diff touches no package. `--ran` reconciliation: `✓ dispatch-gates --ran: 60 derived famil(ies) accounted for — 57 run, 3 NOT-MEASURED (3 DERIVED from a recorded exit 3)`, exit 0, 0 UNRUN. check:skills-token-ratchet + --self-test exit 0 (the edited script's own suite is its --self-test; no *.test.* names it). check:pm-dispatch-gates exit 0 (368 s, run last in the detached runner, read by tail --pid). check:role-word, check:doc-authoring, check:nul-bytes, check:published-files, check:pm-governed-merges all exit 0. Local-only families check:dev-prereqs / check:skill-frame-freshness not owed (no packages/spec build, no frame file). CI check-runs on the PR: in_progress at report time — not waited for; the seat reads them.",
  "dogfood_battery": "n/a — no `!` in the PR title and no (narrowing) arm",
  "mcp_calls": "0 — no MCP GitHub tool used, read or write",
  "api_writes": "3 — (1) git push ×2 to origin claude/issue-26834-hook-refusals-eval (empty probe branch, then f47ca1b2 + merge 1a0fb94a; git push is not a REST write); (2) relay pr_create via scripts/pm/fleet-write/dispatch.mjs → POST /repos/objectstack-ai/objectstack/pulls (draft forced by the relay; assignees os-justin; no labels — the dispatch named none and skip-changeset does not apply, so zero label writes; the relay refuses an empty labels array, so the key was omitted; the labels the PR now shows — documentation, size/s, tooling — were applied by the repo path labeler, another actor, not by me); (3) report comment via scripts/pm/post-stamped.mjs --comment=26834 → POST /repos/objectstack-ai/objectstack/issues/26834/comments. REST reads only otherwise: GET issues/26834, GET issues/26834/comments, GET issues/{26416,26393,26774} (labels of the precedent PRs), GET pulls/27088 (body read-back).",
  "deviations": [
    "Changeset vs the order's 'do the same as the last two merged skills-eval PRs': bb60b86f (#26416) and 7c70f91c (#26393) carried `skip-changeset` and added no changeset; this PR adds a `@objectstack/skills` patch changeset instead, because os-dev.md's published-face rule (measured: sync-catalog copies the eval byte-identical into the published dist; CREDIT_LIMIT 3 hits, positive control 2, absent control 0) and the changeset gate's own text in pr-automation.yml ('skills/** … ships in @objectstack/skills … never the label') both say the eval publishes, and the three token-ceiling precedents the card names (#26774, #26810, #26852) each added that changeset. Said in the PR body.",
    "The PR was opened on head 1a0fb94a (my commit f47ca1b2 + a clean merge of origin/main at 5cba8fc2). origin/main advanced to cbb3a1a1 (5 commits) during the gate sweep; the merge-base-anchored diff shows none of them touch scripts/check-skills-token-ratchet.mjs or skills/objectstack-data/**, so no second merge and no re-measure; AGENTS.md §10 merges main after the PR opens only on a reported conflict or a main-caused red.",
    "Harness attribution reminder names a model in the Co-Authored-By trailer; the commit carries the model-free pair required by AGENTS.md (Claude-Session URL + `Co-authored-by: Claude`), as os-dev.md says the harness reminder yields. No card trailer in the commit."
  ],
  "open_questions": [],
  "out_of_scope_findings": [
    "observation (not class a/b/c) · skills/objectstack-i18n names no `hooks` translation group: content/docs/protocol/kernel/i18n-standard.mdx lists `hooks.{hook}.refusals.{code}` (added by PR #26831) and translation.zod.ts declares the nested `hooks` record, but skills/objectstack-i18n/SKILL.md's top-level-group roster (apps, messages, globalActions, picklists, dashboards, datasets, reports, pages, flows, positions, permissions, metadataForms, settingsCommon) omits `hooks` and `git grep hooks origin/main -- skills/objectstack-i18n/` is empty — an AI author writing a zh-CN bundle from that skill alone does not learn where the refusal sentence goes. Skill coverage gap, no runtime defect; carrier: the skills seat / #26170's Skills-docs follow-up, noted in PR Acceptance notes, not filed. dedupe words: objectstack-i18n hooks refusals translation group SKILL.md",
    "observation · evals/README.md still describes the eval file without its refusal case (headroom 1 token, seat budget +0); noted, not changed. carrier: none",
    "observation · references/data-hooks.md keeps three pre-catalogue `throw new Error('…')` examples (lines 163, 780, 785) beside the refusals section — legal where no catalogue is declared, outside this card's surface. carrier: none"
  ]
}

ACCEPT → PR #27088 comment 6461136549 · domain:skills#2 · 2026-10-28T05:05Z — ⛔ not a claim. Contract review (CONTRACT_REVIEW_TIER) PASS → PR comment 6461131809 on head 1a0fb94a. Tier H: os-zhuang APPROVED, readied and armed the PR himself (2026-10-28T04:58Z); the seat follows it to MERGED and strips pm:dispatched when Fixes #26834 closes the card. Sibling gap for #26820's thread (not a card): skills/objectstack-i18n lists no hooks translation group.

Landed · domain:skills#2 · session_01QSeKYmhU5KEj1nukYhp1Tg · 2026-10-28T05:59Z

probe	reading
PR #27088	`merged: true`, merge commit `5fe0b9676` (approved, readied and armed by `os-zhuang` at 2026-10-28T04:58Z; added to the merge queue 2026-10-28T05:04Z; the seat flipped nothing)
`origin/main`	carries `5fe0b9676` (`git merge-base --is-ancestor` exit 0); the eval file holds 7 cases and the ratchet row reads 1573
records on the merged head `1a0fb94a`	contract review PASS 6461131809 (`CONTRACT_REVIEW_TIER`), seat ACCEPT 6461136549

Landing path: Tier H (skills/**) — the maintainer's hand on an authorized approval. Fixes #26834 closed the card; pm:dispatched stripped in this act. The sibling gap (skills/objectstack-i18n lists no hooks group) is on #26820's thread (6461357825) for the seat holding that card.

Navigation Menu

Repository navigation

skills(objectstack-data): no eval exercises a hook's `refusals` catalogue, because hooks-security-seeds-datasources.json sits at its token ceiling (follow-up owed by #26170) #26834

What to change

Dedupe

Triage: first grade — `priority:p1` · `domain:skills` · `area:i18n` · `pm:queue` · triage seat · `session_01AavokzJ5DndAwitDXvKy4U` · 2026-10-27T18:55Z

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

skills(objectstack-data): no eval exercises a hook's refusals catalogue, because hooks-security-seeds-datasources.json sits at its token ceiling (follow-up owed by #26170) #26834

Description

What to change

Dedupe

Activity

objectstack-fleet commented on Oct 27, 2026

Triage: first grade — priority:p1 · domain:skills · area:i18n · pm:queue · triage seat · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-27T18:55Z

objectstack-fleet commented on Oct 28, 2026

objectstack-fleet commented on Oct 28, 2026

objectstack-fleet commented on Oct 28, 2026

objectstack-fleet commented on Oct 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

skills(objectstack-data): no eval exercises a hook's `refusals` catalogue, because hooks-security-seeds-datasources.json sits at its token ceiling (follow-up owed by #26170) #26834

Triage: first grade — `priority:p1` · `domain:skills` · `area:i18n` · `pm:queue` · triage seat · `session_01AavokzJ5DndAwitDXvKy4U` · 2026-10-27T18:55Z