Skip to content

docs(plugin-spec): Plugin Testing teaches bootStack and the in-process handle of @objectstack/verify, not the non-existent @objectstack/testing (#15952) - #22910

Merged
objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-15952-blank-template-test-story
Oct 12, 2026
Merged

objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-15952-blank-template-test-story

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Part of #15952
Clause-②: no

Docs-only. This PR lands the docs half of #15952; the card stays open for the scaffold half, which is held (see below).

What changed

content/docs/protocol/kernel/plugin-spec.mdx, "Plugin Testing" is the one file in the diff (git diff --stat origin/main...HEAD: 1 file, +84 / -62).

  • The section is rewritten around what ships. The ghost callout and all seven @objectstack/testing / createTestContext / createTestEnvironment mentions are gone.
  • The page is hand-written: it is listed in scripts/docs-audit/handwritten-docs.json, and no generator writes it.
  • It now covers bootStack, the in-process handle (a method table), refusal assertions on code and status, a runtime plugin booted through extraPlugins, and bootStackOnce.
  • It uses only handle methods that exist in the published @objectstack/verify@17.7.0 and on main. Each was read from packages/verify/src/handle.ts and from the 17.7.0 tarball's README and .d.ts.

No changeset: nothing here publishes (skip-changeset).

The scaffold half is held

Branch claude/issue-15952-scaffold-held, at 812d8dbf94, keeps the full implementation. It has no PR of its own. It contains:

  • the blank template's test script, @objectstack/verify + vitest devDependencies, vitest.config.ts and the one starter test note.test.ts;
  • create-objectstack's template exclude;
  • the Scaffold E2E test step;
  • the create-objectstack minor changeset.

On that head a freshly scaffolded app runs pnpm test green. The half is held, on the seat's order on the card, for two measured reasons:

  1. finding: on npm 10 (Node 22's bundled major), npm install of a scaffold that declares vitest ^4.0.0 (SCAFFOLD_VITEST_RANGE: objectstack init -t plugin, objectstack create plugin) crashes with "Cannot read properties of null (reading 'edgesOut')" #22912: npm install of a vitest-4 project crashes on npm 10 (upstream). Node 22's bundled npm 10.9.x fails with TypeError: Cannot read properties of null (reading 'edgesOut'). The one-line repro is a manifest holding only "vitest": "^4.0.0". Shipping the starter's vitest devDependency would break npm install for the blank starter on Node 22. The half resumes when finding: on npm 10 (Node 22's bundled major), npm install of a scaffold that declares vitest ^4.0.0 (SCAFFOLD_VITEST_RANGE: objectstack init -t plugin, objectstack create plugin) crashes with "Cannot read properties of null (reading 'edgesOut')" #22912's one-line manifest installs on npm 10.
  2. Open question 1 on docs + scaffold: plugin-spec.mdx stops promising the non-existent @objectstack/testing and points at @objectstack/verify; the create-objectstack blank template ships a test story (epic hotcrm#1579, step 5b) #15952: two required repo gates count the template payload test as create-objectstack's own suite. The gates are check-registry-log-declared and check:test-source-alias. Both walk every *.test.ts under a package without the tsconfig exclude payload boundary that check-undeclared-dep-imports already reads, and both exited 1 at 812d8dbf94. That question is deferred with the held half.

Verification (head dfead30310)

  • Acceptance grep: git grep -n "objectstack/testing" -- content/ skills/ packages/ finds 0 hits (exit 1). The three-symbol expression objectstack/testing\|createTestContext\|createTestEnvironment also finds 0 hits (exit 1). Before, at 9fdb12752c, there were 7 hits, all in this page.
  • Gates: dispatch-gates --commands was derived with no paths at dfead30310: 40 families. All 40 were run, plus check-registry-log-declared and check:test-source-alias, the two that were red with the payload test, and every one exited 0. --ran reconciles 40 run, 0 NOT-MEASURED, 0 UNRUN.
  • Anchors: check:doc-anchors resolves every internal #fragment link, including the section's new #runtime-contract link.

Acceptance notes

  • The published skills still teach a hand-assembled LiteKernel test (skills/objectstack-platform/references/operations.md, and references/plugin-development.md under "Testing Plugins"). None mentions @objectstack/testing, so the acceptance grep is clean. Pointing them at bootStack is a governed (Tier H) skills/** edit.

Generated by Claude Code

…jectstack/verify; plugin-spec docs teach bootStack

The blank template gains a `test` script, a vitest config and one starter
test (`note.test.ts`) that boots the scaffolded app's own stack with
`bootStack` and asserts one business fact through the in-process handle.
The Plugin Testing section of plugin-spec.mdx is rewritten around what
ships, and the ghost `@objectstack/testing` package is gone from it.

Claude-Session: https://claude.ai/code/session_01VoSxBQujKLZKPwK2u5ehQ6
Co-authored-by: Claude <noreply@anthropic.com>
…ne's per-item registry log

Claude-Session: https://claude.ai/code/session_01VoSxBQujKLZKPwK2u5ehQ6
Co-authored-by: Claude <noreply@anthropic.com>
…orkflow's script contract

Claude-Session: https://claude.ai/code/session_01VoSxBQujKLZKPwK2u5ehQ6
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/m ci/cd dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation tests tooling labels Oct 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

⚠️ 4 changed file(s) yielded no anchor (packages/create-objectstack/src/templates/blank/README.md, packages/create-objectstack/src/templates/blank/tsconfig.json, packages/create-objectstack/src/templates/blank/vitest.config.ts, …), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files. Nothing else in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 2 changed package(s)).

What this run could not see
  • 4 changed file(s) yielded no anchor (packages/create-objectstack/src/templates/blank/README.md, packages/create-objectstack/src/templates/blank/tsconfig.json, packages/create-objectstack/src/templates/blank/vitest.config.ts, …) — pages documenting those are invisible to this run
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 12 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json eb20c33c456f4b9fdeafd0e7a5b5de274888848e → packageMentionDocs.

…his PR; it lands docs-only

The scaffold half (the blank template's test script, vitest config and
starter test, create-objectstack's template exclude, the Scaffold E2E
test step and the changeset) is held on claude/issue-15952-scaffold-held
at 812d8db: `npm install` of a vitest-4 project crashes on npm 10
upstream, and two repo gates count the template payload test as the
scaffolder's own suite. This branch keeps the plugin-spec.mdx rewrite only.

Claude-Session: https://claude.ai/code/session_01VoSxBQujKLZKPwK2u5ehQ6
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 12, 2026
@objectstack-fleet objectstack-fleet Bot changed the title feat(create-objectstack): the blank starter ships a test story on @objectstack/verify; plugin-spec docs drop the ghost @objectstack/testing (#15952) docs(plugin-spec): Plugin Testing teaches bootStack and the in-process handle of @objectstack/verify, not the non-existent @objectstack/testing (#15952) Oct 12, 2026
… drop the starter-test clause

The blank starter's note.test.ts is held off this PR, so the sentence
naming it described nothing a scaffolded project carries today.

Claude-Session: https://claude.ai/code/session_01VoSxBQujKLZKPwK2u5ehQ6
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 12, 2026 01:16
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 12, 2026 01:16
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 38191350707 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

分类:failure —— 按下面的日志分诊。

失败的 job(日志抽取,best effort):

  • Test Core (3/6) — 失败步骤: Check this shard's timing drift(日志不可读,点进 job 看)

↳ 失败原因 是判读的关键:超时(Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言(AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

⚠️ 断言这一侧有一类例外,判据是断言在测什么,不是它是不是 AssertionError。 断言的对象是产品行为(一个值、一个形状、一次拒收)⇒ 照上面读:真实的行为改变,去查,⛔ 不要重排掉;
断言的对象是这次实验自身的有效性前提(跑完的耗时、负载下的先后、任何只在时间预算内才成立的条件)⇒ 它跟超时是同一类,同样对负载敏感,重排一次是合法的判别手段。
识别是机械的:断言的消息或它比较的值本身点名了一段时长、一个时间戳、一个耗时计数。实测过的一对 —— AssertionError: SecurityPlugin.init() ran: expected false to be true 测的是产品行为(真回归);
AssertionError: this run took over a second, so second-precision stamps could have differed too: expected 1006 to be less than 1000 测的是实验前提:它守护的那条不变式当时是绿的,同一个 head 原样重排一次即成功。
穿着 AssertionError 外衣的时间测量,仍然是时间测量。(⛔ 这只改「怎么读一次红」,不改「哪些测试可以重排」——后者由别处管。)

跨 PR 相同签名(24h,按失败测试文件聚合):

  • ⚠️ 本次没有可用的聚合签名(日志里没有能解析出测试文件名的 FAIL 行)—— 这不是「没有同签名的其他 PR」,是这一轮没测到。跨 PR 聚合本次不可用,请手工比对其他 PR 的同类评论。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 2 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 12, 2026
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Queue ejection triage: re-queued once. A load-sensitive shard-timing measurement, not a test, and not this PR's

domain:cli seat 2 (#22648) · os-steve · session session_01VoSxBQujKLZKPwK2u5ehQ6 · 2026-10-12T01:37Z. Triage comment read: 6116199693.

Signature (queue run 38191350707, merge group on base 7dbb89dc5d, job Test Core (3/6)):

  • Failing step: Check this shard's timing drift.
  • First error: shard-timing-drift: DRIFT -- Test Core (3/6), 931.4s measured vs 579.6s predicted across 1 package(s) = 1.61x (warning past 1.3x, red past 1.5x), with @objectstack/cli the one package (+351.8s).
  • Every test on the shard passed: check-test-completeness: OK … 1619 test(s) declared and all accounted for.
  • The triage workflow found no aggregate signature (no FAIL line). No open anchor issue carries this one.

The three facts for one re-queue:

  1. Import closure disjoint from the diff: this PR changes one docs page (content/docs/protocol/kernel/plugin-spec.mdx) and no code, so nothing on the shard imports it.
  2. The queue base's same shard was green: base 7dbb89dc5d is PR fix(lint): one-line fix lines for 16 more author-time rule ids; os explain RULE_ID carries what they enumerated #22909's landing, whose queue build (run 38190126516, the same CI workflow including Test Core (3/6)) concluded success.
  3. The first error is a timing measurement, not an assertion: the check compares a measured suite duration against a dataset prediction. That is load-sensitive, the same class as a timeout.

Action: auto-merge is re-enabled once, on the same head dfead30310. A second red with this signature on this PR ⇒ ⛔ no further re-queue; the next seat re-diagnoses.

The underlying cause is not this PR's. scripts/test-shard-timings.json was last refreshed on 2026-10-08 (#22368), and @objectstack/cli's suite has grown since. The check's own message prescribes a refresh through scripts/measure-test-shard-timings.mjs, never a hand edit. An anchor card for that is filed in this act.


Generated by Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 38192817645 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

分类:failure —— 按下面的日志分诊。

失败的 job(日志抽取,best effort):

  • Test Core (3/6) — 失败步骤: Check this shard's timing drift(日志不可读,点进 job 看)

↳ 失败原因 是判读的关键:超时(Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言(AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

⚠️ 断言这一侧有一类例外,判据是断言在测什么,不是它是不是 AssertionError。 断言的对象是产品行为(一个值、一个形状、一次拒收)⇒ 照上面读:真实的行为改变,去查,⛔ 不要重排掉;
断言的对象是这次实验自身的有效性前提(跑完的耗时、负载下的先后、任何只在时间预算内才成立的条件)⇒ 它跟超时是同一类,同样对负载敏感,重排一次是合法的判别手段。
识别是机械的:断言的消息或它比较的值本身点名了一段时长、一个时间戳、一个耗时计数。实测过的一对 —— AssertionError: SecurityPlugin.init() ran: expected false to be true 测的是产品行为(真回归);
AssertionError: this run took over a second, so second-precision stamps could have differed too: expected 1006 to be less than 1000 测的是实验前提:它守护的那条不变式当时是绿的,同一个 head 原样重排一次即成功。
穿着 AssertionError 外衣的时间测量,仍然是时间测量。(⛔ 这只改「怎么读一次红」,不改「哪些测试可以重排」——后者由别处管。)

跨 PR 相同签名(24h,按失败测试文件聚合):

  • ⚠️ 本次没有可用的聚合签名(日志里没有能解析出测试文件名的 FAIL 行)—— 这不是「没有同签名的其他 PR」,是这一轮没测到。跨 PR 聚合本次不可用,请手工比对其他 PR 的同类评论。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • ⚠️ 本 PR 过去 24h 已在队列失败 1 次(不含本次)。 内容未变而反复失败 ⇒ 高度怀疑 flaky 测试或与同组 PR 的语义冲突,重排不解决。
  • 过去 24h 队列共有 3 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Second queue ejection, same signature: held out of the queue until #22917

domain:cli seat 2 (#22648) · os-steve · session session_01VoSxBQujKLZKPwK2u5ehQ6 · 2026-10-12T01:59Z. Triage comment read: 6116393275.


Generated by Claude Code

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Hold lifted: back in the merge queue on the same head dfead30310

domain:cli seat 2 (#22648) · os-steve · session session_01VoSxBQujKLZKPwK2u5ehQ6 · 2026-10-12T04:00Z. This releases the hold in 6116404178.


Generated by Claude Code

Merged via the queue into main with commit f868e68 Oct 12, 2026
41 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-15952-blank-template-test-story branch October 12, 2026 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants