Skip to content

test(interactive): M6 P0-A integrated Host acceptance and cross-process verification - #7

Closed
rockyshi1993 wants to merge 13 commits into
mainfrom
test/interactive-m6-integrated-acceptance
Closed

rockyshi1993 wants to merge 13 commits into
mainfrom
test/interactive-m6-integrated-acceptance

Conversation

@rockyshi1993

@rockyshi1993 rockyshi1993 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

M6 technical acceptance candidate — NOT final full sign-off

Baseline: M1–M5 and scoped S07 capacity fix #8 are now merged on main @ f49e2109a5de1986a8fdaf1952984d119832af82. The M6 S07 test is a positive regression; full M6 still requires S13 genuine model-host evidence.

This PR adds actual subprocess-based reference Host acceptance (real FlowKit compiler, immutable Program, SQLite WAL/FULL Checkpoint/Journal, M5 Ed25519 signed Host receipt) and a separate Node scripted worker exchanging JSON via pipes. It does not claim a real Codex/Claude/Grok provider or model, and does not expose the raw Coordinator or Host signer to the worker.

Executable evidence

  • Real original DSL source -> compileInteractive -> Host start -> durable Pending -> public-task-only IPC to a separate worker -> Host-generated signed attestation -> verifier -> Coordinator -> final State and Journal.
  • Kill Host after Pending publication and after worker completed but before receipt acceptance, restart from retained artifact without DSL source, preserve Pending/request identity, no duplicate State commit. Explicitly do not claim exactly-once external provider execution.
  • maxStagesPerCall=1 vs 128 State byte equivalence, stage/journal revision and hash continuity.
  • Condition/Agent/Terminal real Standard vs Interactive final State equivalence.
  • Source bytes, compiled Program bytes and Skill instruction drift rejection before resuming/disclosing the task; no SQLite mutation.
  • Missing Condition field durable errored(FK_R002) and repeat resume stability.
  • The former S07 2MiB saturation reproduction has been replaced with a positive assertion of pre-admission rejection and successful 64KiB receipt under the reviewed fix(interactive): S07 bounded checkpoint and durable receipt index #8 runtime; fix(interactive): S07 bounded checkpoint and durable receipt index #8 remains separate from this M6 test-only PR.
  • Existing M1–M5 negative tests retained; no root API expansion, npm release, deployment or scope creep.

CI

  • Require new tests/integration/*.test.mjs in npm run verify across Ubuntu/Windows × Node 22/24, with installed-package, types, Standard regression and docs.
  • Extend Linux OS-isolation job on Node 22/24 to run subprocess Host and unprivileged model-worker integration with Host-private key/SQLite inaccessible.

Gate / evidence honesty

Final M6 requires T01–T14, T29/T30, T34-A, S01–S07 and S13/S14 as applicable to P0-A, plus independent source review. S13 genuine model Host tool-cycle is not established by a deterministic scripted worker. If an authenticated real-model host cannot be used to obtain native model/tool logs, record that gate as BLOCKED and do not mark full M6 accepted or merge merely on green CI. This PR remains draft until every mandatory gate is supported by evidence.

Superseded by current-main replacement

Closed without merge on 2026-11-11. The M6 integration tests were ported to current main and corrected in PR #13, which is merged as test/CI/roadmap-only change. This original branch was based on stale main and is retained only as historical context. Do not merge this PR. Native Grok Build C12/S13 and M6 final acceptance remain open.

Copy link
Copy Markdown
Contributor Author

FlowKit Interactive P0-A · M6 独立验收复审记录(2026-11-10)

审查对象: db5d8eaf54816f38f188096bafc00b008ee15a6e;基线 c873e3edf0f1a78b13178966cdeb408391755170(M5 已合并)。
PR 变化: 仅 .github/workflows/ci.yml、package.json 与 tests/integration/{m6-host,m6-worker,m6-e2e.test}.mjs 五个文件。无生产源码、Standard 六个根 API、产品发布规则变动。

A. 真实 CI 验证 — PASS(针对上述精确 HEAD)

GitHub Actions 42842229092 7/7 completed/success:

  • Ubuntu 24.04 × Node 22.13.0:npm run verify + docs consumer,343/343 PASS / 0 FAIL / 0 SKIP。
  • Ubuntu 24.04 × Node 24.22.1:相同结果。
  • Windows 2025 × Node 22.13.0:相同结果。
  • Windows 2025 × Node 24.22.1:相同结果。
  • Linux Node 22 + Node 24 两项专项:低权限模型工作进程可接收公开任务,但不能读取 Host 私钥或 SQLite;随后完成真实 OS 进程间的 JSON IPC → 受信 Host 签名 → 真实 Coordinator CAS → 终态。工作进程为确定性脚本,不是实际大模型。
  • Website job:PASS。

B. 本轮新集成证据

在真实 Node Host 进程中,用原版 FlowKit compileInteractive 编译源码,加载不可变 Program,再经 SQLite WAL/FULL、M5 createTrustedInteractiveHost、独立子进程管道、Ed25519 签名回执、M4 complete 进入终态。模型工作进程不继承 Host 环境变量、无密钥/数据库能力;Host 只发送经过输入白名单的 PublicTask,不暴露私有 State。

已重验:Pending 发布后 SIGKILL 并从只保留 Program/Store 的新进程恢复、Worker 已产出结果但 Host 尚未 CAS 时的崩溃与同一 requestId 再派发、Program 源文件变动和删除、编译产物字节变动、Skill 字节变动、Host policy 变化、DB artifact 篡改、缺失 when 字段 FK_R002 稳定错误、Condition/Agent/Terminal 与 Standard 最终 State 字节哈希对照、阶段预算 1 与 128 的最终 State 等价。

可审计量化指标(Ubuntu Node24 样本): 预算1:27 次 CAS、28 条 Journal、终态 CP 972B;预算128:11 次 CAS、12 条 Journal、终态 CP 952B;两者最终 State 字节一致。该次 Host SIGKILL 后恢复耗时 139ms;其他矩阵记录分别是 Ubuntu Node22 1404ms、Windows Node22 363ms、Windows Node24 236ms。样本数据不构成生产性能 SLA。压测/写放大上界需在独立容量测试中验证。

C. P0-A 验收矩阵(当前状态)

Gate 本轮/既有证据 结论
S01 / T01:Condition→Terminal 与 FK_R002 M6 缺输入负例、两次 resume + M4 路由测试 PASS
S02 / T02–T03/T29:Agent 阶段重启、Request identity M4 crash.test(phase 前后 kill)+ M6 双进程恢复 PASS(参考 Host)
S03 / T04/T13/T14:验证器、重试、失败出口 M4 coordinator.test 正负例 PASS
S04 / T06/T09:同请求双 complete、丢响应 M4 crash.test 真实双进程 CAS + lost-response PASS
S05 / T05/T30:租户、错请求、错 attempt、非法产物 M3 store.test、M4/M5 身份拒绝、M6 SQLite 篡改 PASS
S06 / T07/T22:源码/Program/Skill/Policy 漂移 M6 参考 Host 逐一拒绝,M5 固定策略,M2 同字节 Loader PASS(当前参考 Host 的固定 Manifest 策略)
S07 / T08/T34-A:规范化 JSON、2MiB 拒绝、阶段预算与 Ledger M3/M4 JSON 负例,M6 预算/CP/CAS 度量 PARTIAL:接近 2MiB 的持续增长 Ledger 饱和路径尚未做独立故障注入/可恢复出口证明
S13 / T11/T27:真实模型宿主调用闭环 只完成真实 Node JS 进程/SQLite/脚本工作进程,没有 authenticated Codex/Claude/Grok 等真实模型的 JS tool → Pending → 模型 → JS tool 原生日志 BLOCKED
S14 / T12/T28/T33:四矩阵、Standard 回归、TS、包 完整 7/7 GitHub CI;npm run verify 包括类型/打包/Standard 根 API 回归 PASS

D. 复审裁决(必须遵守)

M6 自动化技术测试:PASS。P0-A M6 全量验收:PARTIAL / BLOCKED。
不能把脚本工作进程写成真正的 AI 模型调用,也不能凭 7/7 CI 自动视作 S13 已验收。Node/Linux 权限隔离不代表 Windows 特定 ACL 或真实 Codex/Claude/Grok 宿主的凭证/授权链已验证。
此外 S07 Ledger 接近 2MiB 上限的长期运行仍需压力/故障证明;若发现无法退出的 waiting,应作为 P1 修复并重跑矩阵。

已确认的需求 V0.3、技术设计 V0.6-R5 在更早阶段冻结,但其完整原版文件未通过本次工具环境提供;本记录不能声称逐行核对不可见文档。

依用户已授权的门禁要求,PR #7 保持 draft/open,不 squash merge;main 继续保留已通过的 M1–M5。 后续需完成 S13 原生模型宿主真实调用证据与 S07 Ledger 边界验收,才可复审后合并。不得发布 npm、部署、开源或推进 P0-B/P1/P2。

Copy link
Copy Markdown
Contributor Author

M6 independent review update — 2026-11-11 (exact HEAD d999ef8d3aa261fd1cd9820a9533a05f251329d6)

Validation: GitHub Actions 42922185790 has 7/7 successful jobs: npm run verify on Ubuntu 24.04 / Windows 2025 × Node 22.13.0 / 24.22.1 (344/344 tests per matrix cell, no failures), both Linux cross-UID security jobs, and website. This is technical test green, not M6 acceptance.

New reproducer / P1 blocker (S07 / T08, T34-A): commit d999ef8 adds tests/integration/m6-ledger-boundary.test.mjs. A legitimate waiting checkpoint can be durably stored at 2,097,077 of 2,097,152 bytes while a valid 64-KiB Agent answer will not fit the subsequent receipt-accepted checkpoint. Both complete() attempts reject with RangeError: Canonical JSON exceeds maximum bytes; original checkpoint and Journal are unchanged, and resume() returns the same waiting Pending. Atomicity PASS, liveness FAIL: a durable Pending can be issued before the Host knows the eventual receipt is unpersistable. Repeating the valid receipt does not unblock the run. This is a reproducible acceptance-blocking P1; green tests deliberately demonstrate the flaw, not its correction. Fix requires a contract-consistent capacity strategy (pre-publication admission control plus bounded receipts/State, or an audited external receipt ledger) and a new positive recovery test. Do not silently prune receipt history or weaken original acceptedRevision/digest idempotency.

S13/T11/T27 remains BLOCKED: reference Host executes real compiler/SQLite/CAS/Ed25519 and an independently spawned script worker; no authenticated Codex/Claude/Grok model performed a genuine host JS-tool → Pending → model → host-tool cycle with auditable native logs. A scripted subprocess cannot substitute for this gate.

Frozen-source cross-check limitation: confirmed original Requirements V0.3 and Technical Design V0.6-R5 documents were not found as full authoritative files in the current connected materials; prior review/PoC reports reference them, but cannot substitute for exact-line contract audit.

Decision: M6 is NOT accepted. Keep PR #7 draft/open, do not squash merge, publish, deploy, open-source, or start P0-B/P1/P2. Main stays at c873e3edf0f1a78b13178966cdeb408391755170 (M1–M5). Required next actions are (1) settle and implement the S07 receipt/store capacity contract with full regressions, and (2) execute an authorized genuine model Host integration in a properly isolated environment and retain native tool-call/provenance logs; then independently re-review and rerun exact-HEAD matrix CI.

Copy link
Copy Markdown
Contributor Author

M6 S07 验收状态更新 — 2026-11-11(替代早前的 S07 BLOCKED 结论)

S07:PASS(已修复并合并);S13:BLOCKED(真实模型宿主证据未完成);M6 总体仍未验收。

S07 修复事实

  • 独立修复 PR #8 已完成源码复审并 squash 合并到 main,合并提交 f49e2109a5de1986a8fdaf1952984d119832af82。
  • 主分支合并后 CI 42943462571 7/7 completed/success。PR fix(interactive): S07 bounded checkpoint and durable receipt index #8 合并前最终 CI 42942778808 7/7 success,四平台各 346/346 PASS、0 fail、0 skip。
  • 修复保留原 2 MiB 硬上限,为非终态保留 256 KiB 容量;容量不足使用 errored(FK_I_CHECKPOINT_CAPACITY) 的持久化 CAS/Journal 退出,不再永久 waiting。原 State 保留;没有把未接纳的回执冒充已接纳。
  • SQLite 新增 仅用于 FlowKit 执行幂等的回执索引;Checkpoint 仅保留 16 条近期摘要,全部历史回执的 digest/acceptedRevision 保留在索引中;M5 历史签名 Ack、旧记录回填、事务一致性、数据库篡改拒绝均有回归证据。
  • 经复审删除了没有依据的 128 KiB 结果限制;实际由整个 Checkpoint 的可用空间决定;256 KiB 的合法结果在空间足够时仍可成功。不包括 Context、用户业务产物、存储 Provider 或 GC 能力。

当前 M6 验证

  • 本 PR 的 tests/integration/m6-ledger-boundary.test.mjs 已从旧的缺陷复现改为新修复的正向回归:近 2 MiB 的不安全 Checkpoint 写入在 CAS 前拒绝,原 Checkpoint/Journal 不变;已有合法 64 KiB Agent 回执可继续到 succeeded,其 acceptedRevision 可从新的 SQLite 索引查询。CI 日志给出 M6_S07_LEDGER_CAPACITY_FIXED。
  • 本 PR 更新后 head e1e15a70356e53401288d393aa7527c6723fe797 的 CI 42943579505 7/7 completed/success;Ubuntu/Windows × Node22/24 各 357/357 PASS、0 fail、0 skip,两项 Linux 隔离及网站验证成功。
  • 未完成:S13 / T11/T27,缺真实 Codex/Claude/Grok 等宿主的模型原生 JS 工具调用→真实 Agent→Host 签名/回执→终态审计日志;脚本 worker 不是模型实测,不得替代。
  • S07 不再是 M6 的 P1 阻断,但 M6 PR test(interactive): M6 P0-A integrated Host acceptance and cross-process verification #7 必须继续 Draft/Open,不可因 CI 绿色而提前合并,须先获得 S13 的真实宿主证据并完成独立复审。无 npm 发布、部署或 P0-B/P1/P2 变动。

Copy link
Copy Markdown
Contributor Author

M6 S07 Host 边界补测更新(2026-11-11)

S07 的 Trusted Host 大回执边界已通过独立 PR #9 完成测试、契约澄清和复审,并 squash 合并到 main@674a3a92b128877f07e4f31590aab4bab306c698。该 PR 仅修改两份文档 + 一份 Host 端到端测试,没有修改生产执行代码、Program ABI 或状态存储模型。

精确 HEAD 70394d4e981ae0eaae50e1314c951818830d7a9c 的 CI 42946128092 7/7 成功,四矩阵各 349/349 通过、0 fail/skip。

新覆盖:未认证原始大回执 >2MiB 必须拒绝且不改 Run;真实 Host 需独立确认 Provider 超限事件,然后签现有的 小型 agent-call-failed,经 Host 验证和 Coordinator CAS 触发有界重试或 errored(FK_R004);当已签名/有界结果仍无法进入 Checkpoint 则保持 #8 的 FK_I_CHECKPOINT_CAPACITY 路径。三种错误路径不可混同。脚本测试也验证了结果引用、宿主实例重新构造、历史 Ack 和不泄露完整巨大模型输出。

M6 S13 原生 Codex/Claude/Grok 模型宿主工具链仍为 BLOCKED。 该补测的 provider 限额观察为脚本模拟,不是实际模型原生日志;本 PR #7 保持 Draft/Open,不能凭上述结果直接合并或开始 P0-B。

Copy link
Copy Markdown
Contributor Author

Roadmap Stage 2 completed — M6 gate unchanged (2026-11-11)

The previously confirmed second phase (Interactive/Standard compile config parity, binding version strategy, unused checkStep false-rejection, measured Program parsing cost) has been independently implemented, reviewed and squash merged as PR #10, main@ea2bb10893a9be2e5de3561fc9a660a6977ae15c.

Exact-head CI 42954236755 7/7 PASS and four Ubuntu/Windows × Node22/24 matrices 357/357 PASS each (0 FAIL, 0 SKIP). This phase added Standard projectRoot/Skill static config support, safe no-Agent checkStep compatibility, explicit Host-owned binding version policy documentation and mismatch regression, and a bounded Program cache that continues to re-verify exact immutable bytes and identity each phase. Diagnostic single-run sample: 16 Condition nodes, original total 147.5 ms vs cache-enabled 30.7 ms; not a production SLA.

This does not close M6 S13. PR #7 must remain Draft/Open; the real authenticated Codex/Claude/Grok Host JS tool call + model + original provider/session provenance evidence has NOT been obtained. Before M6 final acceptance, sync against the new main (S07 #8, Host oversize #9, Stage2 #10), re-run the exact-HEAD complete acceptance matrix and independent frozen-contract review. Do NOT merge #7 or initiate P0-B/P1/P2 early.

Copy link
Copy Markdown
Contributor Author

S13 ChatGPT 自主工具桥接实测(2026-11-11)

按用户建议,已在独立 PR #11 中创建真实的 Condition→Agent→Terminal FlowKit JavaScript 工作流,并由 当前 ChatGPT 模型实际读取 CI 发布的公开 Pending 后,使用 GitHub 工具提交模型生成的回答。

真实 Pending requestId=req:6bed7ea39b96e81b99100ffe69bb174ba38533d42f3bbdbefad6239cee0ac829,任务摘要 sha256:96733a8e513b08020acc42ef5dabd93413d52b452d8b982d40afcf3b81ac2876;后续验证由真实 Program、SQLite、可信 Host、Ed25519 回执和 CAS 完成到 succeeded/done,原始 acceptedRevision=6,Journal 12 条;伪造签名拒绝,重开 SQLite 恢复 Pending、重复回执保持原始 Ack。证据见 Actions 42957070193,7/7 成功,四个平台各 358/358 测试通过;PR #11 复审评论详述身份与数据边界。

重要:这个结果证明 ChatGPT 本次工具驱动真实 FlowKit 工作流可行;不证明 Codex 原生 Host/JS tool 日志,也未由 Host 独立认证模型提供商/会话来源。 两次 CI 不共用同一个数据库,Host 在第二次 CI 内完成 SQLite 关闭/重开。保持 M6 S13 BLOCKED、PR #7 Draft/Open,不以脚本、GitHub 提交身份替代真正原生模型证明。

Copy link
Copy Markdown
Contributor Author

M6 S13 — C01–C12 复杂验收补充(2026-11-11)

根据用户要求,现已冻结 C01–C12 复杂验收矩阵及逐项结果报告(PR #12),同时补齐多 Agent 真实工具桥接和大规模脚本回归。

已实际证明(不得混淆模型来源)

  • 脚本执行协议:真正的 FlowKit 编译器、不可变 JS Program、Ed25519 Trusted Host、SQLite CAS/Journal 下,C01 四串行 Agent、C02 正反/缺失条件分支、C03 非法 Schema → retry、C04 retry 耗尽 FK_R004、C10 36 Agent/255 Journal/16 CP 近期回执缓存/36 历史 Ack、C11 stageBudget=1/128 最终 State 字节相等;另复用主分支已有的 C05 SIGKILL、C06 多进程 CAS、C07 policy 漂移、C08 安全隔离、C09 容量回归。脚本模型调用数=0。
  • 真实 ChatGPT 工具桥接:五次真实 ChatGPT 输出,先后对应不同的实际 Pending:需求分析、架构设计、实施计划、独立审查受控错误类型、独立审查第二次修正;四个 Agent 节点 + 两个 Condition。真实第 4 次结果被 Schema 拒绝后生成新 requestId/attempt=2,再由模型第 5 次回答并经 Trusted Host 签名/CAS 到 succeeded/done。39 Journal。每次答案由当前 ChatGPT 在读取 GitHub Actions 实际发布的 Pending 之后通过 GitHub 工具提交,属于真实模型参与,但不是原生 Codex/Grok 单会话运行。
  • 实际模型回答提交顺序 SHA:1d515fe0, 031eb433, f0fd5700, 7363bd17, abe592e8;完整验证 Actions 42963110678 7/7 成功,四平台各 364/364 测试通过。
  • 详尽逐项覆盖、结果/限制见 C01–C12 结果报告。

必须保留的 BLOCKED

Copy link
Copy Markdown
Contributor Author

C01–C12 复杂场景验收矩阵的独立草稿 PR #12 已完成协议回归与逐项复审。精确 HEAD 33c9418fd01d9fe42f6686507477f61f659bfabc 的 Actions 42963831991 为 7/7 success,四平台各 364/364 pass、0 fail、0 skip;实测四个串行 Agent + 两个 Condition、五次依次提交的 ChatGPT 答复(含一次人为的无效 Schema 注入与下一次合法重试)、36 Agent 长链及 255 条 Journal、预算 1/128 同 State 哈希。SIGKILL/并发/签名/容量回归使用现有 main 证据,综合 source/Skill drift 仍在 M6 Draft。

分层裁决:脚本协议测试多数 PASS;ChatGPT Bridge C01/C03 仅在非原生身份认证意义下 PASS;C02/C05/C07 PARTIAL;C12 原生 Grok Build/Codex + Model Session provenance 仍 BLOCKED。 五次答复记录在 GitHub commits 中,不能据此独立验证模型提供商/会话身份;四平台 CI replay 同一 fixture 不是 20 次真实模型调用。完整审核见 PR #12 独立复审。M6 #7 保持 Draft,不因测试绿灯合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant