Skip to content

Commit 1bd7574

Browse files
docs(skills): objectstack-formula eval case 4 names the refused date forms without spelling its own must_not_contain strings (#26978)
<!-- claude-projects-attribution: {"github_login":"os-steve"} --> _Requested by **Steve** · [project thread](https://claude.ai/code/project/chan_01FdL1XeUyvVNxLreh3F4Qi9?thread=cmsg_01FdL1XeUyvVNxLreh3F4Qi9JDFo2s6eXsNxvQivW2pGpK)_ Fixes #26814 Clause-②: no Metadata-protocol: none ## 维护者速读 - 改了什么:`skills/objectstack-formula/evals/cel-predicates-formulas.json` 第 4 例的参考答案改写——不再把自己 `must_not_contain` 要拒的两个字符串(JS 的 Date 构造、`today()` 加数字)原样拼出来,改为点名其形态;七个断言、全部事实一字不少,文件字节数不变(5407 B),token 棘轮行不动;附 `@objectstack/skills` patch changeset。 - 为什么改:参考答案自己含被拒字符串,按该例标准作答反而判错(总监复审 PR #26810 发现)。 - 风险与回滚:只改一条 JSON 字符串与一个 changeset,无 schema/行为变化;回滚即 revert 本 PR。 - 先看:`skills/objectstack-formula/evals/cel-predicates-formulas.json`(第 4 例 `expected_output` 一行)。 - 两分钟验证:`node scripts/check-skills-token-ratchet.mjs`,预期看到该文件 `1352 tokens (ceiling 1352; headroom 0)` 且总判定 ✓。 - 需要你做的:批准本 PR(`skills/**` 受管面,Tier H,保持 draft 等 GOVERNED_APPROVERS 批准;席位不 ready/不入队)。 ## What changes - **Case 4 `expected_output`** of `skills/objectstack-formula/evals/cel-predicates-formulas.json` no longer spells its own `must_not_contain` strings. The two phrases that carried them are reworded; everything else in the case is byte-identical: - "never `new Date()` / `Date.now()`" → "never a JS `Date` / `Date.now()`" - "never `today() + 45` (date arithmetic faults)" → "never date arithmetic on `today()` (it faults)" - **Every assertion and fact kept**: the CEL-envelope seed `close_date: cel`daysFromNow(45)``, the determinism gate (two `os build` runs byte-identical), `daysBetween(today(), record.close_date)`, the `record.close_date - today()` duration-stored-as-`{}` trap, and the date-arithmetic fault. `must_contain` / `must_not_contain` are untouched. - **Changeset** `.changeset/26814-formula-eval-case4-self-consistent.md`, `@objectstack/skills` patch (the evals ship in the published catalog). - No forced test edits. No ratchet row moves. ## Why Case 4 refused `new Date(` and `today() + ` in an answer while its own reference answer contained both, so an answer that matched the reference was graded wrong. Found by the contract review of PR #26810 (card #26814). ## Tests Per-case cross-check over the file (each case: no `must_not_contain` string inside its own `expected_output`, every `must_contain` string present), at `origin/main` 912076f and at this head: ```text origin/main 912076f: case 4: must_not_contain "new Date(" PRESENT case 4: must_not_contain "today() + " PRESENT cases=7 failures=2 head f853185: cases=7 failures=0 ``` `JSON.parse` of the file succeeds at this head. ### Skills readings (`skills/**` diff — whole-file and whole-package, before → after) | Reading | before (912076f) | after (f853185) | |:---|---:|---:| | `evals/cel-predicates-formulas.json` lines / bytes / tokens | 75 / 5407 / 1352 | 75 / 5407 / 1352 | | whole package, all `skills/*/SKILL.md` lines / bytes | 4605 / 224961 | 4605 / 224961 | | whole package, all `skills/*/evals/*.json` lines / bytes | 1479 / 94793 | 1479 / 94793 | Token = `ceil(bytes / 4)`, the convention `scripts/check-skills-token-ratchet.mjs` declares. Ceiling row for the file stays at 1352 (headroom 0, unchanged). ## Acceptance notes - Same-byte rewrite achieved, so the standing ruling 「A:先削重复,余额按实测提高」 was not needed and the ratchet row did not move. - The cross-check script is a local probe (not committed); its two rules are the ones the card's acceptance line states. ## Verification Local gate subset at head f853185 (`node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`, no paths; 42 derived commands run serially, each exit code captured before any pipe): <details><summary>42 / 42 exit 0 · `--ran` reconciliation: 42 derived, 42 run, 0 NOT-MEASURED, 0 UNRUN</summary> ```text ✓ check-skills-token-ratchet: skills/objectstack-formula/evals/cel-predicates-formulas.json is 1352 tokens (ceiling 1352; headroom 0). ✓ check-skills-token-ratchet: 82 authored bundle file(s) within their ceilings; 10 generator-owned file(s) measured, not ratcheted. Run reconciliation — 42 derived, 42 run, 0 NOT-MEASURED, 0 UNRUN. ✓ dispatch-gates --ran: 42 derived famil(ies) accounted for — 42 run, 0 NOT-MEASURED (a DERIVED zero — all 42 recorded an exit code and none of them is 3). ``` Build closure first: `bash scripts/pm/os-verify-lock.sh -c 'pnpm build --concurrency=2'` → `VERDICT command-exit 0 · held the lock 841s (14m01s)`. </details> Every family CI runs on the PR (`Lint & Repo Gates`, `TypeScript Type Check`, `Check Changeset`, `Governed Surface Queue Guard`, …) is read from this PR's check-runs, not re-run locally. `test:repo` is not in the local subset (it is CI's; this diff edits no test). Dogfood battery: n/a (no `!` in the title, no `(narrowing)` arm). --- _Generated by [Claude Code](https://claude.ai/code/session_01ADHkmPQ8p3cVKear7Mchi2)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent b8f596c commit 1bd7574

2 files changed

Lines changed: 13 additions & 1 deletion

File tree

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
'@objectstack/skills': patch
3+
---
4+
5+
docs(skills): objectstack-formula eval case 4's reference answer names the refused date forms without spelling its own `must_not_contain` strings
6+
7+
Clause-②: no
8+
Metadata-protocol: none
9+
10+
- **Before.** Case 4 of `skills/objectstack-formula/evals/cel-predicates-formulas.json` refuses `new Date(` and `today() + ` in an answer, yet its own `expected_output` spelled both (`` `new Date()` `` and `` `today() + 45` ``), so a reference-grade answer was graded wrong.
11+
- **Now.** The reference answer says "never a JS `Date` / `Date.now()`" and "never date arithmetic on `today()` (it faults)", as case 7 names "the `label` macro". Every assertion and every fact stays: the CEL envelope seed, the determinism gate, `daysBetween(today(), record.close_date)`, the `{}` duration trap. A per-case cross-check (no `must_not_contain` string inside its own `expected_output`, every `must_contain` string present) reports 0 failures over all 7 cases; the file keeps its byte size, so its token-ratchet row does not move.
12+
- No schema, key or behaviour changes.

‎skills/objectstack-formula/evals/cel-predicates-formulas.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
{
3535
"id": 4,
3636
"prompt": "Demo opportunities should close 45 days after the package is installed, and a formula should show how many days remain until close.",
37-
"expected_output": "Seed value as a CEL envelope, `close_date: cel`daysFromNow(45)``, never `new Date()` / `Date.now()` (the determinism gate: two `os build` runs must be byte-identical). Days remaining is `daysBetween(today(), record.close_date)` — never `record.close_date - today()` (a duration stored as `{}`) and never `today() + 45` (date arithmetic faults).",
37+
"expected_output": "Seed value as a CEL envelope, `close_date: cel`daysFromNow(45)``, never a JS `Date` / `Date.now()` (the determinism gate: two `os build` runs must be byte-identical). Days remaining is `daysBetween(today(), record.close_date)` — never `record.close_date - today()` (a duration stored as `{}`) and never date arithmetic on `today()` (it faults).",
3838
"files": [],
3939
"assertions": {
4040
"must_contain": ["cel`", "daysFromNow(", "daysBetween("],

0 commit comments

Comments
 (0)