diff --git a/docs/interactive-chatgpt-response-evidence.md b/docs/interactive-chatgpt-response-evidence.md new file mode 100644 index 0000000..e6f884a --- /dev/null +++ b/docs/interactive-chatgpt-response-evidence.md @@ -0,0 +1,41 @@ +# ChatGPT-authored response evidence for FlowKit Interactive + +**Evidence tier: M-level recorded model response. This is not native S13.** + +## What the experiment does + +1. FlowKit compiles the Interactive workflow and durably creates a public Agent Pending in SQLite. +2. The test emits the exact `runId`, `requestId`, `taskDigest`, Program artifact hash and allowlisted task input. +3. A ChatGPT model reads that Pending evidence in the conversation and authors a structured response. +4. The response is returned through an authenticated GitHub tool action and stored as a fixture. +5. A later test run deterministically reconstructs the same Pending and rejects the fixture unless all identity fields match exactly. +6. The trusted Host validates the result, signs its own Host receipt, commits through CAS, verifies duplicate Ack and resumes the Run. + +## What it proves + +- A ChatGPT-authored response was produced after inspection of a real FlowKit Pending. +- The response is bound to the exact deterministic Pending by `runId/requestId/taskDigest`. +- FlowKit's actual Program, SQLite Store and Trusted Host process that recorded response. +- Host signature/CAS, duplicate receipt and same-phase SQLite close/reopen behavior. + +## What it does not prove + +- The ChatGPT model invoked a FlowKit-native JavaScript tool. The response is returned through a GitHub tool action, not a native FlowKit JS-tool call. +- Independent cryptographic attestation of the ChatGPT provider/session identity. +- Grok Build or Codex native tool-call provenance. +- Cross-UID / OS-level model-process isolation. +- One physical SQLite database surviving across the separate Pending-preparation and response-acceptance Actions runs. The second run reconstructs the deterministic Pending and uses its own database. + +Therefore this evidence is useful for M-level real-model participation and response-binding tests, but **must not be counted as native Host S13 / C12 PASS**. + +## Test execution modes + +Normal CI must consume the complete committed response fixture and reach Terminal. If the fixture is absent, the test fails. + +Only an explicit preparation invocation may stop at Pending: + +```bash +FLOWKIT_S13_PREPARE=1 node --test tests/interactive/s13-chatgpt-tool-bridge.test.mjs +``` + +The preparation mode is not used by the normal CI workflow. diff --git a/docs/interactive-complex-acceptance-c01-c12.md b/docs/interactive-complex-acceptance-c01-c12.md new file mode 100644 index 0000000..f2375d2 --- /dev/null +++ b/docs/interactive-complex-acceptance-c01-c12.md @@ -0,0 +1,51 @@ +# FlowKit Interactive P0-A — 复杂验收矩阵 C01–C12（冻结版） + +**范围与判断标准**：此矩阵只测试 P0-A 的 `Condition / Agent / Terminal` 和可信 Host 的 CAS、回执、恢复、安全。不得把 Standard 的 Human/Action/Verification/Loop/Parallel 实现或脚本 Worker 的结果当作 Interactive 已支持或真实模型验证。不得把 GitHub CI 的测试数量当成真实模型会话次数。 + +**证据必须分层**： +- **Protocol / Scripted（自动化）**：实际 FlowKit Compiler → JavaScript Program → SQLite Store / Trusted Host 执行，但 Agent 结果由脚本提供。证实协议，不证实模型遵循指令。 +- **Live Model（真实模型）**：真实 ChatGPT/Grok 逐次读取已生成的 Pending 后生成答案；每次的 requestId、taskDigest、输出、工具操作必须有独立证据。ChatGPT → GitHub Actions 桥接只能证明真实参与和受控处理，不代表原生宿主/会话来源认证。 +- **Native Host（S13）**：单个受控 Grok Build / Codex 原生会话模型→JS 工具→Host→回执，原生工具日志及 Host 独立验证的会话/来源证据；不能用脚本模拟、GitHub 上传身份替代。 + +## 场景、可观测预期和证据 + +| ID | 场景 | 冻结验收要求 | 自动化证据范围 | 真实模型门禁 | +|---|---|---|---|---| +| C01 | 4 个串行 Agent + 2 个 Condition | 原生 Program 顺序、后续节点只接收合法上游状态，未跳过/重复执行；最终 State 包含各节点输出 | 新增复杂链+Host 签名测试 | **待实测**：至少四次真实模型回合 | +| C02 | 动态路线与异常条件 | ready=true/false、审查通过/拒绝、缺失 when 输入分别进入正确路径；未选 Agent 不调度 | 新增多路径正反测试 | **待实测**：至少两条真实分支 | +| C03 | Schema 拒绝与修复 | 第一次输出无效不污染 State；下一 Pending requestId/attempt 更新，合法结果可以成功 | 新增接纳/重试负例 | **待实测**：真实模型两次输出 | +| C04 | 失败/重试耗尽 | retries+1 限制，最后 `FK_R004`，State 不含未认可结果 | 新增失败序列 | 至少一个可信 Host 失败路径（真实模型待实测） | +| C05 | Host 崩溃恢复 | Pending 发布、接纳前、校验及提交各切点 SIGKILL；请求 identity/State/Journal 一致 | 已有 `tests/interactive/crash.test.mjs` 三项真子进程测试；当前 main 的 tests/integration/m6-e2e.test.mjs 另有 Host+脚本 Worker（原 PR #7 的相关工作由 PR #13 纳入） | 与真实模型耦合的 kill/resume **待实测** | +| C06 | 双进程竞争、丢响应、重复回执 | CAS 只接纳一次；后续始终返回原始 `acceptedRevision`；不声明外部 Provider 恰好执行一次 | 已有 `crash.test.mjs` + `coordinator.test.mjs` | 原生模型多会话竞争待实测 | +| C07 | Program / Skill / Policy 漂移 | 变更后未发生额外调度/CAS；旧依赖恢复不误放行 | main M5 policy 测试；当前 main 的 tests/integration/m6-e2e.test.mjs S06 覆盖脚本 Host 的 source/Skill/Program drift（由 PR #13 纳入） | 原生 Host/模型待实测 | +| C08 | 权限、签名与泄漏 | 错租户/伪签/敏感字段/UID 隔离 fail closed；模型不能直接操作私有 Store | main M5 `trusted-host` + Linux 2 项 OS 用户隔离任务 | 原生 Grok Build 私钥/SQLite 不可达待实测 | +| C09 | 容量与超限 | 2MiB hard，1.75MiB active 预留；有界结果可成功；错误原子退出；未认证超大回执不能杀 Run | main S07 `s07-capacity` + `host-oversize` | 真实模型大输出路径待实测 | +| C10 | 长链与账本 | 至少 30 个串行 Agent，Checkpoint 近期摘要 ≤16，SQLite 全量历史回执可查；Journal 连续，恢复历史 Ack | 新增长链与 SQLite 回执索引测试（现有 21 节点基础） | 不要求 30 次模型计费；脚本覆盖为准 | +| C11 | 阶段预算一致性 | `maxStagesPerCall=1` 与 128 在相同 Program/结果序列下最终 State **字节相等**，Journal 合法；不要求 revision 一致 | 新增复杂链跨预算用例，当前 main 的 tests/integration/m6-e2e.test.mjs 还有脚本 Worker 验收 | 模型侧不同调度路径待实测 | +| C12 | 原生真实模型宿主 | Grok Build 等原生 JS 工具调用原始日志、真实会话身份、Host 独立来源认证、隔离、终态 | **不能用脚本或 GitHub ChatGPT 桥接替代** | **BLOCKED**：需 Dot/Grok Build 实际宿主权限与真实日志 | + +## 每个可执行测试的最小证据格式 + +```json +{ + "scenarioId": "C01", + "execution": "SCRIPTED_HOST", + "programHash": "sha256:...", + "runId": "...", + "nodeTrace": ["requirements", "design", "plan", "review"], + "checkpointStatus": "succeeded", + "journalRevisions": 0, + "assertions": ["..."], + "result": "PASS" +} +``` + +测试完成后补充**精确提交/CI URL、各环境完成状态、实际模型回合次数、失败/未覆盖限制**。即使自动化 7/7 通过，C01–C05 只应在「协议层」标记 PASS；真实模型层未取得证据必须为 PARTIAL/BLOCKED。 + +## 阶段门禁 + +1. 只开发测试与验证报告，发现真实缺陷才另外起小范围修复 PR；不擅自重构 FlowKit Core。 +2. PR #7 已关闭且未合并，其适用 M6 测试工作由 PR #13 纳入 main；不得把历史 PR #7 当作仍可合并的分支，也不得因绿灯宣称 Grok/Codex 原生模型来源验收通过。 +3. 在复杂矩阵中允许使用脚本结果注入来做 30 Agent、并发、容量、安全负例，但必须标明执行类型和模型调用次数（脚本为零）。 +4. M6 所需的冻结需求 V0.3 / 技术 V0.6-R5 合规审查仍独立存在，不能被此矩阵替代。 +5. 本轮不引入 P0-B、P1、P2、Context、通用业务产物存储或 GC。 diff --git a/docs/interactive-complex-acceptance-c13-c24-results-2026-11-11.md b/docs/interactive-complex-acceptance-c13-c24-results-2026-11-11.md new file mode 100644 index 0000000..15bca73 --- /dev/null +++ b/docs/interactive-complex-acceptance-c13-c24-results-2026-11-11.md @@ -0,0 +1,58 @@ +# FlowKit Interactive — C13–C24 补充复杂验收结果（2026-11-11） + +## 结论 + +**P0-A 复杂协议补充用例 C13–C20：8/8 PASS（脚本可信 Host）；原 C01–C12 测试继续通过。原生模型宿主 C12/S13 仍 BLOCKED，不能据此合并 M6 / 发布。** + +- 仓库：devcodex-labs/flowkit（private） +- PR：[Draft #12](https://github.com/devcodex-labs/flowkit/pull/12)，基于 main，不修改生产 Core +- 验证测试提交：**09e14e82b72277ef58b656bf129c9c4bc0b4b779** +- 完成的 CI：[42968295106 — 7/7 SUCCESS](https://github.com/devcodex-labs/flowkit/actions/runs/42968295106) +- 关联矩阵：[C01–C12 原矩阵](./interactive-complex-acceptance-c01-c12.md)、[C13–C24 增量矩阵](./interactive-complex-acceptance-extension-c13-c24.md) +- 执行类型：**SCRIPTED_TRUSTED_HOST**，本次新增用例模型调用次数 **0**；不能计入真实模型调用次数 + +### 精确跨平台测试清单 + +| GitHub Actions Job | Test | Pass | Fail | Skip | Job | +|---|---:|---:|---:|---:|---| +| Ubuntu 24.04 / Node 22.13.0 | 372 | 372 | 0 | 0 | SUCCESS | +| Ubuntu 24.04 / Node 24.22.1 | 372 | 372 | 0 | 0 | SUCCESS | +| Windows 2025 / Node 22.13.0 | 372 | 372 | 0 | 0 | SUCCESS | +| Windows 2025 / Node 24.22.1 | 372 | 372 | 0 | 0 | SUCCESS | +| Host OS-user isolation / Node 22.13.0 | — | — | — | — | SUCCESS | +| Host OS-user isolation / Node 24.22.1 | — | — | — | — | SUCCESS | +| Website/docs | — | — | — | — | SUCCESS | + +以上从 GitHub Actions 的实际 job 日志/结论核验，非 PR 描述中的主观声称。372 是每一个环境内的全部测试数量，包含现有回归；新增复杂测试函数为 8 个，不能解释成 372 个真实 Agent 回合。 + +## C13–C24 逐项裁决 + +| ID | 协议层 P | 真实模型 M / 原生宿主 N | 可观测证据 / 缺口 | +|---|---|---|---| +| C13 分支×阶段预算 | **PASS** | M 未单独测 | 预算 1/128 下四个分支结果相同，已核对两个终态 State Hash | +| C14 链中伪造/超限回执 | **PASS** | N 模型对抗攻击待测 | 5 个不同类型的未认证/错误回执被拒且 Checkpoint 和 Journal 字节保持一致 | +| C15 Host 重建后旧 Ack | **PASS** | N 未测 | 下游等待态及 Terminal 后重放历史 receipt，不额外提交 State，Ack 原 acceptedRevision | +| C16 Provider 可信失败回执 | **PASS** | N 真 Provider 失败待测 | signed agent-call-failed 使末端 Agent 重试一次，旧回执重复 Ack，合法第二回合成功 | +| C17 链中 Host policy drift | **PASS** | N 实际宿主升级待测 | 3 条不同入口拒绝政策漂移并证明无 CAS 写入，旧 Host 继续成功 | +| C18 多 Run 交错、重复启动 | **PASS** | N 多模型 Session 未测 | 两 Run 状态隔离；重复 startKey 幂等，同 key 不同 State 冲突 | +| C19 36 Agent 中段失败 | **PASS** | 不要求 36 次模型计费 | 第 18 个脚本结果无效，终态 FK_R004，余下 18 个未分发，State 无非法字段 | +| C20 双有效签名并发竞争 | **PASS（同进程 Host 对象）** | N 真进程/多模型竞争待测 | 两个不同签名内容仅一个 CAS 成功、另一个 FK_I_RECEIPT_CONFLICT；真正双进程测试已在 C06/M4 另验 | +| C21 崩溃/跨版本恢复 | **PARTIAL** | N 未测 | M4 有真 SIGKILL 与同环境恢复；同一物理 DB 跨 OS/Node 迁移未执行，是增强项不是原 P0-A 硬门禁 | +| C22 Prompt 注入/宿主权限 | **PASS（既有 M5 + OS UID）** | **N BLOCKED** | 白名单、拒签、跨 UID 隔离已有证据；Grok Build 原生模型权限分离无证据 | +| C23 真实模型异常组合 | 不可用脚本替代 | **M/N BLOCKED** | 真实 false 分支、持续错误、与 SIGKILL 耦合恢复及多模型竞争未完成 | +| C24 兼容/发行/原生来源门禁 | **PASS（CI 部分）** | **N BLOCKED** | 7 job 全绿不意味着 C12/S13 原生认证已通过 | + +## 首轮失败与复审记录 + +新增 C13–C20 首次提交 5be1abb（CI 42967554504）曾 370 PASS / 2 FAIL，失败集中在 **C16 和 C20 的测试断言**：在状态为 waiting 时，测试错误地读取了公开 Frame 中被隐藏的 State。FlowKit 此时不公开私有 State 是预期安全设计。第二次提交 ac49e3e（CI 42968021802）修复了 C20，留下 C16 一处相同的测试问题（371 PASS / 1 FAIL）。最终测试提交 09e14e8 将 C16 内部断言改为从受信 SQLite Store 读取，四环境 372/372 PASS。 + +这两轮失败**不是 FlowKit Core 缺陷的证据**；本轮修复未修改生产源码、签名器、Coordinator、SQLite Schema 或 Program ABI。PR #12 changed-files 范围仅 docs、tests、fixtures、示例工作流。 + +## 仍不得放行的明确缺口 + +1. **C12/S13：原生 Grok Build/Codex 工具调用、原生 Session/Provider 来源认证、Host 独立可信证明和模型进程权限隔离**。GitHub Actions + ChatGPT 五回合桥接不满足该条件。 +2. **真实模型负例**：C02 的 false 分支、C04 持续错误、C05 实际模型执行中 SIGKILL/恢复、C06 多原生会话竞争、C08/C09 原生输出与权限攻击。 +3. **C07 综合漂移**：原 Draft PR #7 已关闭且未合并；其适用的 source/Program/Skill drift 测试已由 PR #13 纳入当前 main 的 tests/integration/m6-e2e.test.mjs S06。该证据仍是脚本 Host 协议层，不是实时模型或原生 Host 漂移验收。 +4. 未证明的 Provider exactly-once、模型遵循任意 Skill 指令、同 UID 混合进程隔离不能假定成立；这些必须通过宿主信任设计而非脚本测试补足。 + +**裁决：C13–C20 协议 PASS，C01–C12 原先已验证结果继续有效；整体 M6 最终验收仍 BLOCKED。PR #12 保持 Draft 待更新/复审；PR #7 已关闭且未合并。当前阶段不修改 main、不发布新版本。** diff --git a/docs/interactive-complex-acceptance-extension-c13-c24.md b/docs/interactive-complex-acceptance-extension-c13-c24.md new file mode 100644 index 0000000..35940d9 --- /dev/null +++ b/docs/interactive-complex-acceptance-extension-c13-c24.md @@ -0,0 +1,51 @@ +# FlowKit Interactive P0-A — 复杂验收补充 C13–C24 + +> 2026-11-11。追加于[冻结 C01–C12](./interactive-complex-acceptance-c01-c12.md)，不修改原冻结用例和通过条件。只针对 P0-A 的 Condition / Agent / Terminal、JavaScript Program / SQLite / Trusted Host。C12/S13 原生模型宿主门禁独立有效。 + +## 0. 范围与证据等级 + +- **P — protocol/scripted**：真实编译器、生成 JS Program、SQLite WAL/FULL、Host 回执/签名/CAS、脚本 Agent 回答；证实程序约束，不证实模型行为。 +- **M — live model**：实际模型读取真实 Pending 后生成回答，有独立 requestId + taskDigest + tool trace；不等于原生宿主来源认证。 +- **N — native host**：Grok Build/Codex 等原生受控宿主，会话/工具原始日志、受信传输的独立来源证明和模型进程隔离。只有这一级可放行 C12/S13。 +- **X — out of scope**：Human、Action、Verification、Loop、Parallel、P0-B/P1/P2、Context、通用产物存储、Provider 恰好执行一次。不得用本矩阵引入未批准的功能。 + +## 1. 追加用例和可观察不变式 + +| ID | 组合场景 | 正反例和核心断言 | 执行入口 | 验收层 | +|---|---|---|---|---| +| **C13** | 分支真值表 × 阶段预算 | ready=false 不分发 Agent；ready 缺失 FK_R002；reviewApproved=false 去 declined；true 去 done。预算 1/128 的终态 State Hash 一致、Journal 合法 | complex-c01-c12.test.mjs C13 | P | +| **C14** | 第 2 个 Agent 等待时恶意回执 | 错 body、attempt、requestId、伪签、超过 2MiB 的未认证大回执全部拒绝；Checkpoint 字节、Journal、Ack 无变化；合法回执后继续成功 | 同文件 C14 | P | +| **C15** | 下游运行时旧回执重传 | Host 对象重建与终态后重传均返回原始 acceptedRevision，无额外 Journal、无状态重写 | 同文件 C15，交叉 C06/C10 | P | +| **C16** | 最后 Agent 可信 Provider 报错 | 只有签名的 Host call-failed 能触发重试；新 requestId，原状态不污染；旧失败回执仅返回旧 Ack；合法第二次响应路由成功 | 同文件 C16 | P | +| **C17** | 多 Agent 链中途策略漂移 | 修改绑定版本后 load/resume/getPublicAgentTask 均拒绝；Checkpoint/Journal 字节不变；旧 Host 可续接 | 同文件 C17 | P | +| **C18** | 两个 Run 交错与重复启动 | 同 startKey+intent 幂等，变更 State 报冲突；不同 Run 的分支、State、Journal 不串 | 同文件 C18 | P | +| **C19** | 36 Agent 链第 18 步失败 | 无效结果触发 FK_R004；余下 18 个 Agent 不再调度，State 不出现未验证字段，终态 resume 幂等 | 同文件 C19 | P | +| **C20** | 两个独立有效签名但冲突的结果 | 同一 Pending 同时提交两个不同结果，恰好一个 CAS 成功，另一个 FK_I_RECEIPT_CONFLICT；后续仍可正常完成 | 同文件 C20；真 OS 多进程由 crash.test.mjs 另验 | P | +| **C21** | SIGKILL 与跨版本恢复 | Pending、校验、接纳、Commit 切点的真正杀进程恢复；同物理 SQLite 文件在不同 Node 版本/OS 重新打开仍能保留身份、账本 | main crash.test.mjs 已有同环境 SIGKILL；跨 Node/OS 文件转移未验证 | P：部分；跨版本增强项 | +| **C22** | Prompt injection / 模型声称拥有权限 | 未签名的模型输出绝不成为 Host 命令；公开 Task 只包含白名单输入；模型无法获取私钥/私有 SQLite；真实模型中仍需攻击实测 | M5 trusted-host 与 Linux 跨 UID；原生模型进程未验证 | P PASS；N BLOCKED | +| **C23** | 真实模型异常组合 | 真实拒绝分支、真实重试耗尽、实际中途 SIGKILL+恢复、两个真实模型会话竞争都有独立原生执行记录；不能复用脚本 fixture | 原 C02/C04/C05/C06 的 M/N 缺口 | M/N BLOCKED | +| **C24** | 安装兼容、证据与发布门禁 | Program ABI、SDK exports/types、CLI、Node 22/24 × Windows/Linux、docs/site/package、宿主原始工具调用来源；只有全部适用硬门禁通过才可宣称 M6 成功 | CI 7 jobs + C12/S13 | CI P 已历史通过；N BLOCKED | + +**裁决**：C13–C20 全部通过只代表复杂协议层 PASS，不等于 24 个真实模型场景全部通过。C21 跨版本迁移是增强项，不擅自升格为 P0-A 原硬门禁；C22–C24 的 N 缺口仍属于原 M6 S13 门禁。 + +## 2. 端到端风险族覆盖索引 + +| 风险族 | 关键验收 | 测试位置 | 判定要点 | +|---|---|---|---| +| 编译/SDK | 真正 Analyzer、TS/JS、固定 Stage/Graph、Program ABI、hash、打包/types/docs | compiler-front, ir, emitter, stage2-integration, stage2-program-cache, test:package, test:types, docs:examples | 不得将非法图降级通过，不能以模型代码冒充受信 Program | +| 条件与数据流 | 公开字段、Condition 真/假/缺失、串行 Agent、长链早停、预算等价 | C01/C02/C10/C11/C13/C19 + coordinator | 比较 State/Trace，不只比较 terminal status | +| 失败/重试 | Schema 错误、validator throw、Provider failure、retries+1、容量失败退出 | C03/C04/C09/C16/C19 + coordinator, host-oversize, s07-capacity | 无假 Ack、无无限 waiting、无错误数据合并 | +| 身份/幂等 | runId/requestId/attempt/startKey、重复/冲突回执、并发 | C06/C10/C15/C18/C20 + crash, store | State commit 至多一次，不保证外部模型执行恰好一次 | +| 原子性/恢复 | WAL/FULL、真 SIGKILL、Journal hash 链、回执索引、2 MiB hard、active 余量 | C05/C09/C10/C19/C21 + store, crash, s07-capacity | 崩溃可恢复、损坏 fail-closed | +| 安全/漂移 | Program/Skill/Policy hash、签名与租户、秘密披露、跨 UID | C07/C08/C14/C17/C22 + trusted-host, stage2, host-isolation | Host-only 密钥/DB、不可泄漏、不可被 Prompt 自报权限 | +| 模型/宿主 | 多回合、真实异常路径、原生来源及会话日志 | C01–C06/C12/C22/C23/C24 | P/M/N 分层，禁止用 CI 重放充当模型调用 | + +## 3. 判停和证据 + +每个测试必须记录场景编号、执行类别、模型调用次数、实际 Program 身份、Run/Request 身份（有时）、终态、Revision、Journal/Checkpoint 不变式和断言。敏感值、Host 私钥、数据库内容不得写入公开任务或日志。 + +测试通过标准：Ubuntu 24.04 / Windows 2025 × Node 22.13.0 / 24.22.1 的精确 HEAD 回归，另有 Ubuntu Node 22/24 OS 用户隔离、网站、打包、类型与 docs 消费。新提交取消的旧 CI 不得作为新 HEAD 证据。 + +阻断（不得合并 M6）：伪造或越权 Ack、双 CAS 成功、State/Journal 不一致、未披露字段泄漏、合法输入无限 waiting、真实宿主来源证明缺失。故障属于 Core 时另开最小修复 PR，不得在验收分支悄然改动生产逻辑。 + +历史 C01–C12 结果：[2026-11-11 原报告](./interactive-complex-acceptance-results-2026-11-11.md)。M6 独立整体门禁继续按当前主线路线图与验收台账执行；历史 PR #7 已关闭且未合并，其适用测试由 PR #13 纳入 main。C13–C20 精确结果以本分支最新 GitHub Actions 为准，不继承 364/364 的历史测试计数。 diff --git a/docs/interactive-complex-acceptance-results-2026-11-11.md b/docs/interactive-complex-acceptance-results-2026-11-11.md new file mode 100644 index 0000000..5f64b4c --- /dev/null +++ b/docs/interactive-complex-acceptance-results-2026-11-11.md @@ -0,0 +1,86 @@ +# FlowKit Interactive P0-A 复杂验收 C01–C12 — 实测结果与缺口 + +**日期：2026-11-11。仓库：devcodex-labs/flowkit。** +**被测试分支：test/interactive-complex-c01-c12-acceptance。** +**历史测试基线 main：ea2bb10893a9be2e5de3561fc9a660a6977ae15c。** +**代码+真实 ChatGPT 结果 HEAD：abe592e888a4ea77357e31bd5d26ab14b4dac1dd。** + +**结论：复杂 P0-A 自动化回归通过，真实 ChatGPT 4 Agent / 5 回合的非原生工具桥接通过，Grok Build 等原生宿主 C12/S13 仍 BLOCKED。不是“C01–C12 全部完成”。** + +## 1. 精确提交 / CI 证据 + +- [GitHub Actions 42963110678](https://github.com/devcodex-labs/flowkit/actions/runs/42963110678)：7/7 jobs completed / success，Windows 2025 与 Ubuntu 24.04 × Node 22.13.0 / 24.22.1 四组合各 **364 tests / 364 pass / 0 fail / 0 skip**，并且四个组合均输出了 C01_CHATGPT_RECORDED_RESPONSES_ACCEPTED 及 C01/C02/C03/C04/C10/C11 的 FLOWKIT_COMPLEX_ACCEPTANCE 结构化记录。 +- CI 同时覆盖打包、类型、Standard Regression、文档消费、网站构建以及两项 Linux Host 跨 UID 权限隔离。 +- 完成前的多轮 GitHub Actions 因 PR 新提交被取消，**不能**把它们作为精确最终 HEAD 的 7/7 通过证据；每个模型回合的原始 Pending / SHA 仍可在对应 Job 日志核查。 +- Program 代码来源：真正的 FlowKit Analyzer/Interactive Lowerer/Emitter，运行时使用实际生成的 JavaScript Program、默认 SQLite Store 与 Ed25519 Trusted Host；除真实模型桥接外，Agent 回答由脚本确定性提供，**模型调用数为零**。 + +## 2. 每项单独裁决 + +| ID | 协议/脚本验收 | 当前 ChatGPT 桥接 | 原生 Grok Build/Codex Host | 主要证据与限制 | +|---|---|---|---|---| +| C01 串行 4 Agent + 2 Conditions | **PASS** | **PASS（5 次真实模型输出）** | BLOCKED | 脚本 4 个签名回执、35 Journal；真实桥接 4 个不同 Agent / 5 回合，39 Journal，正常 succeeded | +| C02 动态条件与缺失值 | **PASS** | **PARTIAL** | BLOCKED | 脚本分别覆盖 ready=false 直接 declined、ready 缺失 FK_R002、reviewApproved=false 走 declined；真实桥接只覆盖 ready=true + reviewApproved=true | +| C03 Schema 拒绝与修复 | **PASS** | **PASS（受控故障注入）** | BLOCKED | 第一次 reviewApproved 为字符串，被校验拒绝；新 requestId、attempt=2、reasons 中出现类型错误；第二次真正布尔 true 成功，未污染已确认 State | +| C04 重试耗尽 | **PASS** | BLOCKED | BLOCKED | 两次错误结构导致 FK_R004；真实模型只执行了一次拒绝然后修正，没有执行真实持续失败 | +| C05 Host SIGKILL 恢复 | **PASS（既有真子进程）** | PARTIAL | BLOCKED | main tests/interactive/crash.test.mjs 覆盖 Pending 发布、接纳、校验、State 提交前后 SIGKILL；真实 ChatGPT 只在单次 CI 内重新打开 SQLite，并非模型执行中途 SIGKILL | +| C06 并发与重复回执 | **PASS（既有真子进程）** | PARTIAL | BLOCKED | main crash.test.mjs 双进程 CAS 竞争及 coordinator.test.mjs 丢响应恢复；真实 ChatGPT 连续提交不是并发模型会话 | +| C07 Program/Skill/Policy 漂移 | **PASS（脚本协议范围）** | PARTIAL | BLOCKED | main Stage2 policyHash 版本漂移、M5 Program 字节/策略篡改拒绝已测试；PR #13 已将 M6 S06 source/Program/Skill drift 脚本 Host 用例纳入 main。真实模型升级漂移与原生 Host 认证仍未测试 | +| C08 签名、隔离与泄漏 | **PASS（已有指定边界）** | PARTIAL | BLOCKED | main TrustedHost 错签、错租户、输入披露白名单与 Linux 跨 UID 密钥/SQLite 读取拒绝；未证明 Windows ACL 或原生 Grok Build 模型进程隔离 | +| C09 Checkpoint 容量 | **PASS（脚本）** | BLOCKED | BLOCKED | main s07-capacity / host-oversize：2 MiB hard、1.75 MiB active、可接受的 256 KiB 结果、不可接纳的合法回执原子退出；未认证 >2 MiB 拒绝；无真实模型大输出事件 | +| C10 36 Agent 长链 | **PASS（脚本）** | 非必需 | BLOCKED | SQLite Journal 255 条，近期回执缓存 16，全部 36 个 acceptedRevision 可恢复查询，最早回执重放 Ack 不变 | +| C11 budget=1 与 128 | **PASS（脚本）** | 非必需 | BLOCKED | 最终 State 哈希 sha256:e69d3bc210a864cab2801e1c479c5cbc6db7bf492e65256078bafc53be1f7c25，两个预算 Journal 分别 85 与 35 条，符合阶段预算造成不同迁移记录的预期 | +| C12 原生真实模型宿主 | **不适合脚本替代** | ChatGPT GitHub 桥接 PASS，但不满足此项 | **BLOCKED** | 缺 Grok Build / Codex 原生单会话模型→JS 工具→Pending→真实模型→Host 原生工具日志、独立来源认证与宿主隔离证据 | + +**判定用语说明**：PASS 只适用于该列所声明的执行类型，PARTIAL 为只覆盖部分冻结预期，BLOCKED 为缺真实环境或证据。不存在“364/364 = 364 个复杂真实模型用例”的推断。 + +## 3. 真实 ChatGPT 多节点工具桥接的确切边界 + +使用 [C01 真实工作流定义](../examples/interactive/c01-live-chatgpt.workflow.js)，生成的不可变 Program 哈希： + +sha256:339425694337fdd24ff005b2d8eab3109d4026ce6bc176f7a8c130421a7cabb5 + +本次当前 ChatGPT 模型**按顺序**从 GitHub Actions 的 C01_LIVE_CHATGPT_PENDING 日志读取公开任务，提交到 tests/fixtures/c01-live-chatgpt-turns.json 的五次模型结果： + +| 回合 | 模型实际操作 | 提交 SHA | 条件/结果 | +|---|---|---|---| +| 1 | Requirements Analysis，写 requirementSummary | 1d515fe0fe9530e1a866a566e0bc122e6b834690 | 真实 Pending #1；成功 | +| 2 | Architecture Design，使用公开 requirementSummary 写 designSummary | 031eb4337a292eebcacafcf3cef27e48c2757e12 | 真实 Pending #2；成功 | +| 3 | Implementation Plan，使用公开 designSummary 写 executionPlan | f0fd5700d1856e05d437e54184b324f5c9d89286 | 真实 Pending #3；成功 | +| 4 | Independent Review，故意给错误类型的 reviewApproved | 7363bd1742fc94a99e05d7140d2e545d57145e04 | **受控错误注入**；校验拒绝，仅接纳回执身份，不写入业务 State，产生 attempt=2 | +| 5 | Independent Review，根据新的 reasons 修正 reviewApproved 布尔值 | abe592e888a4ea77357e31bd5d26ab14b4dac1dd | 新 requestId，成功，Condition 决定 succeeded/done | + +最终结构化日志字段：C01_LIVE_CHATGPT_COMPLETED，recordedModelResponses=5，status=succeeded，journalRevisions=39，retriesObserved=1，terminal node=done，模型公开任务未泄露 Host 私有 canary。每个 CI Job 在各回执间确实关闭/重开了自己的 SQLite 数据库，证明同一 Job 内物理数据库恢复及 Host 重建。 + +**必须防止三种误称：** + +1. 首轮 Actions 发布公开任务 → ChatGPT 通过 GitHub 工具提交 → 下一轮 CI **重新创建相同逻辑 Run**。两个 Actions 运行**没有**共用一个数据库，也不是单个原生模型 Agent Runtime 自主调用下一次模型。该模型输出只是从真实公开 Pending 派生，GH 提交身份不能代替模型原生会话证明。 +2. 第 4 回合是为了验证校验/重试特意注入错误类型，**不是**证明模型在自然任务中会稳定自行纠错。 +3. 四个 OS/Node 组合都运行同一份已保存的模型回答，不等于真实模型发起了 20 次独立调用。模型真实回答数是 **5**，CI 重放总次数另行统计。 + +## 4. 原有复用的代码证据 + +- C05：main [tests/interactive/crash.test.mjs](https://github.com/devcodex-labs/flowkit/blob/ea2bb10893a9be2e5de3561fc9a660a6977ae15c/tests/interactive/crash.test.mjs)。 +- C06：同一 crash.test.mjs 与 [coordinator.test.mjs](https://github.com/devcodex-labs/flowkit/blob/ea2bb10893a9be2e5de3561fc9a660a6977ae15c/tests/interactive/coordinator.test.mjs)。 +- C07：main Stage2 / M5 tests，以及 PR #13 纳入 main 的 tests/integration/m6-e2e.test.mjs S06 综合 source/Program/Skill drift 脚本测试。 +- C08：main trusted-host.test.mjs + .github/workflows/ci.yml Linux Host OS-user isolation。 +- C09：main s07-capacity.test.mjs 与 host-oversize.test.mjs。 +- C01–C04/C10/C11：本 PR 的 [complex-c01-c12.test.mjs](../tests/interactive/complex-c01-c12.test.mjs) 与 CI JSON 事件。 + +## 5. 独立验收裁决与下一步 + +**代码结果：** 尚未发现新的 P0-A Core 语义故障。C01–C04/C10/C11 新增协议脚本测试和真实 ChatGPT 五回合完整通过，且未提前实现未授权的 P0-B/P1/P2。 + +**不足：** C07 的脚本 Host 综合漂移已在 main，但真实模型 false 分支、C04 真实持续失败、C05 中途实际 SIGKILL + 模型、C06 并发模型会话以及 C08–C09 原生模型场景尚未覆盖。**C12/S13 必须继续 BLOCKED。** + +**建议：**将此验收分支继续保持 Draft 作为历史证据，不把固定五回合 fixture 作为可永久依赖的生产回归测试。下一步使用用户可授权的 Dot/Grok Build 原生宿主，在真实模型会话和明确 OS 可信边界下优先复验 C01/C03/C05/C08/C12（尤其原生模型来源证明）；不因本次 CI 绿灯宣称 M6 已收口、发布软件或转入 P0-B。 + +完整的冻结验收目标详见 [C01–C12 矩阵](./interactive-complex-acceptance-c01-c12.md)。 + + +## Current-main follow-up — 2026-11-12 + +- Current main is 606d055d5f02451fea0b30294889d740ef370d49. The 2026-11-11 C01–C12 live-model and C13–C20 protocol evidence above remains historical evidence on the old baseline; it is not exact-current-main CI. +- PR #7 is closed without merge. Its applicable M6 source/Program/Skill drift coverage is present on current main through PR #13, including tests/integration/m6-e2e.test.mjs S06. +- PR #12 remains Draft and is six commits behind current main. After this branch is updated, rerun all seven CI jobs on the exact new PR HEAD before any merge decision. +- Current exact-main GitHub Actions is still blocked at scheduling: main run 43161214780 was cancelled; PR #20 exact-head run 43170215982 has seven queued jobs with no runner start. This is not a test failure and does not change the historical PASS results above. +- C12/S13 native Grok Build source/session attestation and model-process isolation remain BLOCKED. diff --git a/docs/interactive-full-delivery-roadmap-2026-11-11.md b/docs/interactive-full-delivery-roadmap-2026-11-11.md index 20de9e3..274087d 100644 --- a/docs/interactive-full-delivery-roadmap-2026-11-11.md +++ b/docs/interactive-full-delivery-roadmap-2026-11-11.md @@ -1,9 +1,11 @@ # FlowKit Interactive — Full Delivery Roadmap & Closure Ledger -**Baseline date:** 2026-11-11 (UTC+8) +**Baseline date:** 2026-11-12 (UTC+8) **Repository:** `devcodex-labs/flowkit` **Historical Phase 0 baseline:** `ea2bb10893a9be2e5de3561fc9a660a6977ae15c` -**Current main HEAD after PR #14:** `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` +**Historical main HEAD after PR #14:** `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` +**Current main snapshot:** `81c7839e9f0abe6f4db20d0fc242253b0baff0ea` +**Phase 0 refresh:** Draft PR [#20](https://github.com/devcodex-labs/flowkit/pull/20) records the current baseline and 31-task crosswalk. **Phase 2 design record:** merged via PR #14 (docs-only; formal contract conformance remains conditional) > This document records the approved end-to-end route from the current M6/P0-A state through P0-B, P1, P2 and production delivery. It is a roadmap and traceability ledger, not evidence that deferred features have been implemented or passed. @@ -12,14 +14,19 @@ ### 1.1 Main and CI -- Current main HEAD after PR #14: `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd`. -- Post-merge main CI: [43132513659](https://github.com/devcodex-labs/flowkit/actions/runs/43132513659), **7/7 jobs successful** (Ubuntu/Windows × Node 22/24, Linux Host OS-user isolation × Node 22/24, website). +- Current main snapshot: `81c7839e9f0abe6f4db20d0fc242253b0baff0ea`. +- Historical main CI [43146363166](https://github.com/devcodex-labs/flowkit/actions/runs/43146363166) passed 7/7 on `13d573c5ddc4a0ed3502401589c581799c96c5d2`; it is **not** the current main SHA. +- The three rapid main pushes after PR #16/#19/#18 produced runs 43161068943, 43161142187 and 43161214780; all now show `completed/cancelled`. The workflow has `cancel-in-progress: true`, so earlier same-ref runs are superseded. The latest run's final cancellation cause is not established by accessible metadata. +- PR #12 updated-head CI [43181841181](https://github.com/devcodex-labs/flowkit/actions/runs/43181841181) and PR #11 updated-head CI [43182753787](https://github.com/devcodex-labs/flowkit/actions/runs/43182753787) both passed 7/7. Main run [43183321344](https://github.com/devcodex-labs/flowkit/actions/runs/43183321344) passed 7/7 on the current `81c7839` commit. +- PR #20 refreshed exact-head run [43185112963](https://github.com/devcodex-labs/flowkit/actions/runs/43185112963) passed 7/7 on the current `81c7839` baseline. +- Five old zero-job runs remain queued on the obsolete design branch. New current PR/main runs are scheduling normally; do not treat the stale records as a current global outage. - PR #14 merge commit: `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd`; the merge adds only the reviewed Trusted Host V2 design/acceptance/review documentation. - M1–M5 implementation surfaces and S07 capacity work exist on main. Their presence is not a blanket claim that every M6/native-host gate is passed. - PR #8, #9 and #10 are merged. - PR #7 is **closed without merge** as superseded. PR #13 is **merged** into main with current-main M6 tests, correct project-root discovery and successful exact-HEAD CI. -- PR #12 remains open Draft, head `c6b42a36050d885a37831cbdf0839428b2052c85`, based on `ea2bb10893a9be2e5de3561fc9a660a6977ae15c`. Its scripted C13–C20 tests and documentation CI passed; this does not pass C12/S13 native-host gates. +- PR #12 (C01–C24 acceptance assets) and PR #11 (ChatGPT Pending response evidence) are merged. They contribute P/M-level test/evidence assets only; neither passes native Grok C12/S13 or Windows Secure Host gates. - PR #14 is **merged** as a docs-only conditional security design record. Merge does not authorize Phase 3 or claim Windows/Linux Secure Host, Grok native provenance, C12/S13 or M6 acceptance. +- PR #16 (formal audit R1/R2), #18 (R04 input snapshot fix) and #19 (local verification report/evidence) are merged. PR #17 is closed without merge because its two regression tests are included in #18. ### 1.2 Evidence classes @@ -34,7 +41,7 @@ Never count P evidence as M or N. Never count a CI replay of recorded model turn | Phase | Scope | Required exit condition | Current state | |---|---|---|---| -| 0 | Baseline freeze and traceability | Immutable baseline, full requirement/Finding/test ledger and evidence index | **Complete** — established in merged PR #13 | +| 0 | Baseline freeze and traceability | Immutable baseline, full requirement/Finding/test ledger and evidence index | **Historical freeze complete** — PR #13; current-main re-baseline, 31-task crosswalk and CI fallback are tracked in PR #20 (final review pending) | | 1 | M6 current-main test compatibility | PR #7 tests run against frozen current main; no Core semantic change hidden in test repair | **PASS, merged** — PR #13; main CI 7/7; four matrix cells 369/369, Linux isolation/M6 e2e 11/11 on Node 22/24; no production Core change | | 2 | Trusted Host security design and formal contract conformance | Approved threat model, OS identity, IPC, key custody, Store ownership, provenance and confirmed V0.3/V0.6-R5 conformance | **设计记录已合并**（PR #14；main CI 43132513659 7/7 PASS）；F-01/F-R2-01 **CLOSED**；F-02/F-04 仍 PARTIAL。V0.3 Requirements 与 V0.6-R5 Technical Design 均已恢复并完成完整 SHA-256/包内清单校验；V0.6 Confirmed Handoff 9/9、Core POC Evidence 28/28 文件条目 PASS。现有核心 PoC 为 **22/22 测试 PASS、9/9 进程级崩溃点恢复**，但仅为 Linux/Node 22.16 独立原型；POC-01/05/06/09 PARTIAL，POC-07 Human BLOCKED，POC-08 完整矩阵 BLOCKED。F-07 的原件缺失问题已关闭，正式契约符合性审查进入 **IN-PROGRESS**；在完成 T01–T34、R01–R22、S01–S14 及当前实现映射前，Phase 2 不宣称 conformance PASS。参见 [正式契约原件登记](./interactive-phase2-formal-source-register.md) 和 [V0.3/V0.6 符合性审查计划](./interactive-v03-v06-conformance-audit-plan.md) | | 3 | Trusted Host security implementation | Windows and Linux isolation, access-denial tests and secure Host operations pass | Pending | @@ -96,7 +103,7 @@ Freeze a Host threat model before implementation: - Provider-signed evidence may be used if genuinely available, but is not assumed to be the only possible design. Any alternative Host-owned transport must state its exact assurance level and limits. - Human approval remains unavailable/blocked unless a model-unwritable trusted human decision channel is independently verified. -Deliverables: threat model, trust topology, candidate-ingress security contract (with optional authenticated listening IPC only when needed), key custody, provider/session evidence design, Windows/Linux profiles, attack matrix and acceptance gates. Independent review must close all **applicable design-blocking** findings before Phase 3. Missing formal V0.3/V0.6-R5 originals remain an explicit conformance BLOCKED item, not evidence that they never existed; OS capability PoCs can be performed separately, but no final conformance claim can be made without them. +Deliverables: threat model, trust topology, candidate-ingress security contract (with optional authenticated listening IPC only when needed), key custody, provider/session evidence design, Windows/Linux profiles, attack matrix and acceptance gates. Independent review must close all **applicable design-blocking** findings before Phase 3. The V0.3/V0.6-R5 originals have been recovered and their full SHA-256/archive manifests verified (see source register). The source-provenance portion of F-07 is CLOSED; clause-by-clause conformance remains IN-PROGRESS until T01–T34, §§1–18, R01–R22 and S01–S14 are mapped and independently reviewed. ## 6. Phase 3 — Trusted Host security implementation @@ -154,7 +161,7 @@ Exit: every C item is PASS, FAIL, PARTIAL, BLOCKED or explicitly DEFERRED with o 3. Run Linux cross-UID and Windows ACL isolation jobs independently. 4. Run build, npm verify, package/tarball, types, CLI, docs examples, website and Standard regressions. 5. Resolve every M6 blocker and rerun the relevant scenario plus full regression. -6. PR #12 may be merged only as the scripted matrix/docs contribution after its own review; its merge does not pass C12/S13. +6. PR #12 is merged as the scripted matrix/docs contribution; its merge does not pass C12/S13. 7. PR #7 was closed without merging and its current-main replacement PR #13 was merged after independent code review and CI; do not merge it again. Future M6 native-host security evidence remains in PR #14 and follow-up scoped implementation PRs. 8. Do not release a version until P0-A release criteria are explicitly met. @@ -369,10 +376,10 @@ A phase cannot pass by deleting a test, weakening an assertion, changing the evi 1. Phase 0/1 **complete**: PR #13 merged to main `9bac8f86347a405efe068f96423ae3e501005061`; old PR #7 closed unmerged. 2. Main CI [42998521531](https://github.com/devcodex-labs/flowkit/actions/runs/42998521531): 7/7 successful; Ubuntu/Windows × Node 22/24 each 369/369; Linux OS isolation + M6 11/11 on both Node versions. -3. Phase 2 security design record (still Draft/Conditional): [Trusted Host V2](./interactive-trusted-host-security-v2-design.md), [threat model](./interactive-trusted-host-security-v2-threat-model.md), [acceptance matrix](./interactive-trusted-host-security-v2-acceptance.md), [review ledger](./interactive-trusted-host-security-v2-review.md). Third-round Grok review closed F-01/F-R2-01; F-02/F-04 remain PARTIAL and F-07 remains BLOCKED. -4. Three rounds of Grok Build independent review have been completed, including the final targeted review of F-01/F-R2-01. Phase 2 is **not yet frozen**: F-02 native tool binding, F-04 cross-principal handle isolation and F-07 formal contract provenance remain open/BLOCKED. Draft/CI success is not a substitute for native Host security acceptance or a professional third-party security certification. -5. Then implement OS isolation, trusted bridge, direct Grok Build tool loop, real-model edge cases and Phase 6 M6 closure. PR #12 remains Draft until its own scoped review. +3. Phase 2 security design record (still Draft/Conditional): [Trusted Host V2](./interactive-trusted-host-security-v2-design.md), [threat model](./interactive-trusted-host-security-v2-threat-model.md), [acceptance matrix](./interactive-trusted-host-security-v2-acceptance.md), [review ledger](./interactive-trusted-host-security-v2-review.md). Third-round Grok review closed F-01/F-R2-01; F-02/F-04 remain PARTIAL. F-07 source recovery is CLOSED; formal clause-by-clause conformance remains PENDING REVIEW. +4. Three rounds of Grok Build independent review have been completed, including the final targeted review of F-01/F-R2-01. Phase 2 is **not yet frozen**: F-02 native tool binding and F-04 cross-principal handle isolation remain PARTIAL/BLOCKED; formal conformance is still pending. Draft/CI success is not a substitute for native Host security acceptance or a professional third-party security certification. +5. Current Phase 0 re-baseline is tracked in [PR #20](https://github.com/devcodex-labs/flowkit/pull/20): current snapshot, CI fallback and the approved 31-task P0-A crosswalk. Then continue the exact dependency order: formal contract review and P0-A gap closure; H3-01/H3-05 OS-only work may proceed independently under the approved security design; H3-02 requires its capability probe; H3-03 follows H3-02; H3-04 follows H3-03 + H3-05; Phase 4 native Grok remains gated by Phase 3. -**Formal contract source caveat:** Historical FlowKit requirement V0.3 and technical design V0.6-R5 are cited in previous handoffs, but their full immutable bodies/SHA-256 are not presently available to this PR's independent reviewer. Conformance to those documents remains **BLOCKED** until the exact originals can be independently read and hashed; roadmap IDs cannot replace official text. +**Formal contract source status:** Requirements V0.3 and Technical Design V0.6-R5 originals are recovered and hash-verified (see [formal source register](./interactive-phase2-formal-source-register.md)). Do not mark overall conformance PASS until the complete clause-by-clause matrix and independent review are finished. **No direct main modification or release is authorized. Merge only through a reviewed PR after exact-HEAD CI. This test/roadmap PR does not authorize production semantic changes or declare M6 complete.** diff --git a/docs/interactive-p0a-31-task-crosswalk-20261112.md b/docs/interactive-p0a-31-task-crosswalk-20261112.md new file mode 100644 index 0000000..da3bf62 --- /dev/null +++ b/docs/interactive-p0a-31-task-crosswalk-20261112.md @@ -0,0 +1,128 @@ +# FlowKit P0-A V0.1 — 31 Task Current-Main Crosswalk + +**Snapshot main:** \`81c7839e9f0abe6f4db20d0fc242253b0baff0ea\` +**Approved plan:** \`01-P0A-IMPLEMENTATION-PLAN-V0.1-CONFIRMED.md\`, confirmed 2026-11-10. +**Purpose:** Map all 31 approved file-level tasks to current source, tests and evidence. This is a source-presence and gap inventory, **not** the final acceptance report. A source file/test existing does not automatically mean its full task is PASS. + +## 1. Status meanings + +- **CODE-PRESENT** — implementation/test surface exists and maps to the task; acceptance still requires exact contract and runtime evidence. +- **PARTIAL** — some behavior exists, but a named part of the approved task or its evidence remains open. +- **GAP** — required implementation surface/behavior is not present on current main. +- **CI-BLOCKED** — local evidence exists but current exact-main GitHub Actions run has not completed. +- **NOT-IN-SCOPE** — excluded by confirmed P0-A plan; not counted as a defect. + +No item in this table is a final P0-A acceptance PASS. + +## 2. M1 — Standard compiler reuse / Feature Gate / Resumable IR + +| Task | Current main implementation | Tests/evidence | Status and remaining closure | +|---|---|---|---| +| A-M1-01 Standard analyzer reuse | \`src/interactive/compile.mjs\` calls \`analyzeSource\`; original analyzer remains in \`src/compiler/analyze.mjs\` | \`tests/interactive/compiler-front.test.mjs\`: real Standard analyzer for JS/TS and diagnostic preservation; \`ir.test.mjs\` compares original Graph/Dataflow/when AST | **CODE-PRESENT**. Finish T01–T34 mapping and exact CI matrix. | +| A-M1-02 P0-A feature gate | \`src/compiler/interactive/feature-gate.mjs\` | \`compiler-front.test.mjs\`: unsupported Action/Verification, Agent verify, featureStage and pure Condition/Terminal | **CODE-PRESENT**. Keep negative cases in full release matrix. | +| A-M1-03 Resumable IR lowering | \`src/compiler/interactive/lower.mjs\` | \`ir.test.mjs\`: stable IR, source identity, original when AST semantics | **CODE-PRESENT**. Complete independent comparison against V0.6 §2/§5 and edge-case ledger. | +| A-M1-04 IR validation | \`src/compiler/interactive/validate-ir.mjs\` | \`ir.test.mjs\`: injected edges/events, duplicate blocks, missing phases | **CODE-PRESENT**. Verify all T/R/S negative cases, not only current unit fixtures. | +| A-M1-05 M1 tests | \`tests/interactive/compiler-front.test.mjs\`, \`ir.test.mjs\` | 6 + 5 named tests; exact current-main tree is included in updated PR #12/#11 merge-tree CI runs 43181841181 and 43182753787, both 7/7 | **CODE-PRESENT / CI-PASS via exact merge-tree matrix** (Ubuntu 24.04 + Windows 2025 × Node 22.13.0/24.22.1). Current main push run 43183321344 passed 7/7. | + +## 3. M2 — Executable Program / ABI / loader + +| Task | Current main implementation | Tests/evidence | Status and remaining closure | +|---|---|---|---| +| A-M2-01 JS emitter | \`src/compiler/interactive/emit.mjs\` emits executable \`advance()\`, ProgramTable and identity | \`tests/interactive/emitter.test.mjs\`: deterministic bytes/hash, zero imports, Condition/Terminal and Agent frame/Pending/retry decisions | **CODE-PRESENT**. Complete V0.6 §5/§10 contract crosswalk. | +| A-M2-02 compile API / diagnostics | \`src/interactive/compile.mjs\` returns diagnostics and compiled bytes/identity | \`compiler-front.test.mjs\`, \`emitter.test.mjs\` | **CODE-PRESENT; module layout consolidated**. Approved plan named \`diagnostics.mjs\`, but separate file is not itself a semantic requirement; verify diagnostic fields and stable codes. | +| A-M2-03 same-byte loader | \`src/interactive/load.mjs\` exports \`loadInteractiveProgram\` | \`emitter.test.mjs\`: wrong digest, dynamic import and ABI mismatch rejection; \`stage2-program-cache.test.mjs\`: re-read exact Store bytes/identity/policy | **CODE-PRESENT** under \`load.mjs\` rather than planned \`loader.mjs\`. Finish exact corrupted-byte and same-byte load audit. | +| A-M2-04 Checkpoint / proposal runtime validation | Relevant checks are distributed across \`validate-ir.mjs\`, \`coordinator.mjs\` and \`sqlite-store.mjs\`; no standalone \`src/interactive/checkpoint.mjs\` | \`ir.test.mjs\`, \`coordinator.test.mjs\`, \`store.test.mjs\` | **PARTIAL**. Build an explicit requirement-to-function-to-negative-test map for every CheckpointView, event, proposal, stack effect and immutable identity invariant. | +| A-M2-05 emitter/loader/ABI tests | \`emitter.test.mjs\`, \`ir.test.mjs\`, \`compiler-front.test.mjs\` | Emitter suite includes deterministic standalone output, exact hash, zero imports, routing and loader rejects | **CODE-PRESENT**. Dedicated test file names differ from plan; verify the complete required golden/ABI vector coverage. | + +## 4. M3 — Canonical JSON / Registry / SQLite Store / Journal + +| Task | Current main implementation | Tests/evidence | Status and remaining closure | +|---|---|---|---| +| A-M3-01 canonical JSON and identity | \`src/interactive/canonical.mjs\`: bounded canonical bytes, snapshot, hash and decoding; Store implements artifact identity | \`store.test.mjs\`: key order, prototype-as-data, accessors, Proxy, cycle, \`-0\`, non-finite and limits | **PARTIAL**. Confirm the exact V0.6 canonicalization requirement and add/verify full RFC 8785/JCS golden vectors (POC-F04); current tests show a constrained canonical JSON contract, not by themselves complete JCS conformance. | +| A-M3-02 SQLite CheckpointStore | \`src/interactive/sqlite-store.mjs\` implements WAL/FULL, Genesis, transactions, CAS, Journal and receipt index | \`store.test.mjs\`: atomic CAS/Journal, duplicate transition rollback, connection restart and two OS processes | **PARTIAL**. Current public Store interface is synchronous while R1/R2 CF-06 records an async five-operation V0.6 contract discrepancy. Resolve against exact V0.6 §5/§8 and approval record; preserve atomic semantics. | +| A-M3-03 Program Registry / Run retain | fk_i_artifacts stores immutable Program bytes/identity; fk_i_runs(namespace, artifact_hash) has a composite FK to the artifact row. createRun() inserts the Run reference and Genesis Journal in one write transaction; deleteArtifact() refuses while any Run references the artifact. | store.test.mjs: immutable bytes, tenant namespace, deletion refusal, atomic Run/Journal; m6-e2e.test.mjs: retained Program and tamper rejection | **CODE-PRESENT for atomic Run-retain + Genesis**. Important distinction: putArtifact() pre-registers immutable bytes in a preceding transaction, but the durable Run→artifact retain association is the FK-bearing Run row committed atomically with Genesis. A crash before createRun() can leave an unreferenced artifact row, not a Run without its Program retain. Finish the exact V0.6 §6.2 lifecycle/cleanup crosswalk; do not call this an established atomic-retain defect. | +| A-M3-04 Store/Journal/Receipt tests | \`tests/interactive/store.test.mjs\`, \`s07-capacity.test.mjs\`, \`coordinator.test.mjs\` | Tests include CAS, Journal chain, receipt history, duplicate transition rollback, tenant isolation and cross-process competition | **CODE-PRESENT / PARTIAL**. Finish full historical Ack, unknown-CAS-response and all approved failure-matrix cases with raw evidence. | +| A-M3-05 crash workers | \`tests/interactive/crash.test.mjs\`, \`tests/integration/m6-worker.mjs\`, \`m6-host.mjs\` | Real SIGKILL before/after Agent publish, receipt acceptance, validation and State commit; two-process receipt race | **PARTIAL**. Current tests cover representative cut points, not proof that all F01–F20 and all applicable Windows/Linux equivalents have passed. | + +## 5. M4 — Coordinator / Agent / API + +| Task | Current main implementation | Tests/evidence | Status and remaining closure | +|---|---|---|---| +| A-M4-01 Coordinator | \`src/interactive/coordinator.mjs\` | \`coordinator.test.mjs\`: Condition/Terminal, Pending, retry, commit, resume, budget and concurrency | **CODE-PRESENT**. Full T01–T14 and S01–S07 closure remains part of M6. | +| A-M4-02 policy preflight | Policy snapshot/hash and runtime-binding checks are integrated in \`coordinator.mjs\`; no separate \`policy.mjs\` | \`coordinator.test.mjs\`: unsupported checkStep and missing bindings reject before Store writes; \`stage2-integration.test.mjs\`: binding-version drift | **CODE-PRESENT; module consolidated**. Verify all approved policy/skill/program identity drift cases. | +| A-M4-03 Agent execution semantics | Agent frame, Skill resolution, retries, validation and candidate commit are integrated in \`coordinator.mjs\`; no separate \`agent.mjs\` | \`coordinator.test.mjs\`: retries+1, FK_R004, pure validator contract, State isolation and candidate commit | **CODE-PRESENT; module consolidated**. Verify complete Standard parity and every T02/T04/T13/T14 case. | +| A-M4-04 receipt lifecycle | Receipt acceptance, duplicate identity, conflict, CAS and durable index are integrated in Coordinator/Store; no separate \`receipts.mjs\` | \`coordinator.test.mjs\`, \`store.test.mjs\`, \`crash.test.mjs\` | **CODE-PRESENT; module consolidated**. Finish full receipt replay and lost-response matrix. | +| A-M4-05 public SDK entry | \`src/interactive/index.mjs\` is exported as package subpath \`./interactive\`; there is no standalone \`src/interactive.mjs\` facade | \`tests/package/consumer.mjs\` imports \`@devcodex-labs/flowkit/interactive\` and checks its exports/real run | **CODE-PRESENT; file layout/API shape differs from plan**. CF-02 still requires V0.6 §4 and approval-record review before changing public exports. | +| A-M4-06 Coordinator/Agent/recovery tests | \`coordinator.test.mjs\`, \`crash.test.mjs\`, \`store.test.mjs\` | 23 Coordinator tests (including two R04 tests), 3 crash tests and 10 Store tests | **CODE-PRESENT / PARTIAL**. Test files are consolidated; map each T01–T14/T29 acceptance case to exact assertions and raw execution evidence. | + +## 6. M5 — Trusted Host / projections / admin / bridge + +| Task | Current main implementation | Tests/evidence | Status and remaining closure | +|---|---|---|---| +| A-M5-01 PublicTask / trusted result | \`src/interactive/trusted-host.mjs\` provides allowlisted \`getPublicAgentTask()\`; \`coordinator.mjs\` public frame currently includes terminal State; no separate \`readTrustedTerminalResult\` capability is exposed | \`trusted-host.test.mjs\`: public task excludes State/Host secrets and enforces disclosure; CF-05 source discrepancy remains | **PARTIAL**. Need an explicit public-vs-trusted result API/permission boundary and model-facing adapter test; do not infer a real leak without showing the model receives the Host \`load()\` result. | +| A-M5-02 admin controller / adminTerminate | No \`admin-controller.mjs\`, \`adminTerminate\` export or corresponding trusted admin facade found in current source/types | R2 CF-01; current coordinator/trusted-host exports list only start/complete/resume/load/accept/task | **GAP — P0-A blocker**. Design the independent Host Admin principal, tombstone, audit, CAS race, Program-missing escape path and stale-receipt rejection against frozen V0.6/approved M5 plan before implementation. | +| A-M5-03 reference Host bridge CLI | Current main has \`examples/interactive/s13-chatgpt.workflow.js\` and \`examples/interactive/c01-live-chatgpt.workflow.js\` from PR #11/#12; these are evidence-specific workflows, not a reusable trusted Host bridge CLI or general consumer example | PR #11/#12 merged test/evidence workflows; M6 integration Host fixtures remain under \`tests/integration/\` | **GAP / NOT PROVIDED AS A REUSABLE TRUSTED-HOST EXAMPLE**. Build the minimal short-lived trusted Host bridge after M5 security boundary is resolved; no new npm bin. | +| A-M5-04 tenant/auth/projection/admin tests | \`tests/interactive/trusted-host.test.mjs\`, \`tests/security/host-isolation.mjs\`, \`tests/integration/m6-e2e.test.mjs\` | 16 Trusted Host tests; Linux separate-UID negative test; M6 worker integration | **PARTIAL**. Admin tests cannot pass before A-M5-02; Windows separate-account/ACL/handle-inheritance negatives remain BLOCKED. | + +## 7. M6 — CI / package / types / crash / docs / performance + +| Task | Current main implementation | Tests/evidence | Status and remaining closure | +|---|---|---|---| +| A-M6-01 package exports/manifest | \`package.json\`, \`scripts/package-files.mjs\`, \`check-package.mjs\`, \`test-package.mjs\`; exports include \`./interactive\`, no extra npm bin | \`tests/package/package.test.mjs\`; updated PR #12/#11 merge-tree CI runs 43181841181 and 43182753787 each passed 7/7 | **CODE-PRESENT / CI-PASS via exact merge-tree matrix**; current-main direct push run 43183321344 passed 7/7. | +| A-M6-02 TS declarations/consumer | \`types/interactive.d.ts\`, \`tests/types/interactive.ts\`; package consumer imports Interactive runtime subpath | \`scripts/test-types.mjs\`, \`scripts/test-package.mjs\`, \`tests/package/consumer.mjs\`; updated PR #12/#11 merge-tree CI runs passed 7/7 | **CODE-PRESENT / CI-PASS via exact merge-tree matrix**. Verify the TS 5.0.4 minimum remains part of the final release gate. | +| A-M6-03 CI integration | package.json test/verify includes Interactive and integration tests; .github/workflows/ci.yml has 4 OS/Node verify jobs + 2 Linux Host jobs + website | PR #12 merge-tree run 43181841181 passed 7/7; PR #11 merge-tree run 43182753787 passed 7/7; current main run 43183321344 passed 7/7; refreshed PR #20 exact-head run 43185112963 passed 7/7. | **CODE-PRESENT / CURRENT MAIN AND PR CI-PASS**. This does not close M6 native-host/fault/performance gates. | +| A-M6-04 fault/concurrency matrix | \`crash.test.mjs\`, \`m6-e2e.test.mjs\`, \`m6-ledger-boundary.test.mjs\`, \`store.test.mjs\` | Representative process kills, two-process races, M6 host restart and capacity tests | **PARTIAL**. Map full F01–F20 and platform-equivalent cuts; current coverage is not a full matrix PASS. | +| A-M6-05 docs/examples | Interactive maintainer docs exist: docs/interactive.md, docs/interactive-stage2-contract.md, docs/interactive-host-security.md and docs/interactive-s07-capacity.md. PR #11/#12 also add two test-specific workflows under examples/interactive; neither is a reusable trusted Host consumer bridge. No examples/interactive/README.md exists. | docs:examples verifies existing Standard documentation examples; current Interactive material is maintainer guidance plus evidence-specific workflows, not a complete packaged consumer example suite. | **PARTIAL**. Remaining work is a maintained Interactive README, reusable Trusted Host example, consumer-facing docs integration, supported Node/OS/security limits and a consolidated M6 evidence guide. | +| A-M6-06 budget/performance | \`scripts/bench-interactive-load.mjs\` is explicitly diagnostic-only; S07 capacity tests cover checkpoint/receipt limits | \`s07-capacity.test.mjs\` 13 tests; benchmark records load/cache timings and Journal revisions | **PARTIAL**. No dedicated \`budget-perf.test.mjs\` or full write-amplification/concurrency/WAL/FULL benchmark evidence; add bounded metrics without inventing SLA. | + +## 7.1 PR #11 / #12 final evidence review (merged) + +Both PRs are now merged into current main. Their unique contributions are documentation, fixtures, example workflows and tests; neither changes production Core source, Program ABI or Store schema. + +### PR #12 — C01–C24 acceptance assets + +- Merge commit: \`8ccb53055f640539fd86c524f079867fbdb12713\`. +- Exact updated PR-head Actions run [43181841181](https://github.com/devcodex-labs/flowkit/actions/runs/43181841181): **7/7 jobs passed**. The four Ubuntu/Windows × Node 22/24 matrix jobs passed; Node 22 cells reported 386/386 tests. +- Scripted protocol layer: \`complex-c01-c12.test.mjs\` contains 14 tests covering C01–C04, C10–C11 and C13–C20. C13–C20 are scripted Trusted Host tests with zero model calls. +- Recorded-model-response layer: \`c01-live-chatgpt.test.mjs\` replays five ChatGPT-authored responses against five exact Pending identities (four dependent Agent nodes plus a deliberate invalid-schema review response and corrected retry). The CI log records \`execution=CHATGPT_AUTHORED_FIXTURE_REPLAY\`, five recorded responses, five request IDs, Program hash, final State hash, terminal success and 39 Journal revisions. This is M-level recorded response evidence, not live model API execution inside CI or native provider/session attestation. +- Normal CI requires all five response fixtures. Only explicit \`FLOWKIT_C01_PREPARE=1\` allows an incomplete preparation run. A missing fixture cannot silently produce a green normal CI test. +- The test creates a fresh temporary SQLite DB per CI run and tests close/reopen within that run; it does not share one physical DB across separate model turns/Actions runs. +- C21 remains PARTIAL; C22 has protocol/OS-UID evidence but native Host remains BLOCKED; C23 remains M/N BLOCKED; C24 remains CI/compatibility partial with N BLOCKED. C12/S13 remains BLOCKED. +- C04's dedicated fixture uses retries=1 (two attempts); default retries=2 / three attempts is covered separately by main Coordinator tests. + +### PR #11 — ChatGPT-authored response bound to FlowKit Pending (M-level) + +- Merge commit: \`81c7839e9f0abe6f4db20d0fc242253b0baff0ea\`. +- Exact updated PR-head Actions run [43182753787](https://github.com/devcodex-labs/flowkit/actions/runs/43182753787): **7/7 jobs passed**. The four Ubuntu/Windows × Node 22/24 matrix jobs passed; Node 22 cells reported 387/387 tests. +- The test publishes an actual FlowKit Public Pending, then binds a ChatGPT-authored response delivered through an authenticated GitHub tool action to the exact \`runId/requestId/taskDigest\`; the trusted Host subsequently verifies, signs its Host receipt, commits by CAS, checks duplicate Ack and performs same-phase SQLite close/reopen. +- **This is not a FlowKit native JavaScript-tool invocation and not native S13.** The model response returns through a GitHub tool action; there is no independent provider/session attestation, Grok Build/Codex native tool provenance or cross-UID model isolation. See \`docs/interactive-chatgpt-response-evidence.md\`. +- Phase 1 and Phase 2 are separate Actions invocations with separate temporary SQLite databases. Phase 2 reconstructs the deterministic Pending and verifies exact identity; it does not continue the physical DB from Phase 1. +- Normal CI requires the complete response fixture. Only explicit \`FLOWKIT_S13_PREPARE=1\` permits a preparation-only Pending run. + +**Combined decision:** retain both PR contributions in main as P/M-level evidence assets. Neither merge passes native Grok C12/S13/T27, Windows Secure Host, or overall P0-A acceptance. + + +## 8. Cross-cutting findings and corrected interpretation + +| Finding | Current conclusion | +|---|---| +| CF-01 | P0-A admin termination is absent; this is a real implementation gap. | +| CF-02 | Factory API is present; whether top-level signatures are required is a formal V0.6 §4/approval decision, not a simple missing capability. | +| CF-05 | Terminal State is present in Coordinator/Host load frame; PublicTask itself is allowlisted. The separate trusted result-read permission and model-facing adapter boundary remain unresolved. Do not claim a proven model leak without a path showing the model receives Host load output. | +| CF-06 | Current Store interface is synchronous; compare exact frozen async Store contract and approved implementation decisions before changing it. | +| CF-07 | Fixed in merged #18; keep both R04 tests. Updated PR #12/#11 merge-tree CI runs 43181841181 and 43182753787 passed all four Ubuntu/Windows × Node 22/24 verify cells, including the R04 regressions; direct current-main push run 43183321344 passed 7/7. | +| CF-08 | Reproduced in local probe; not fixed and not shown to bypass P0-A feature gate. Add an isolated regression and minimal fix. | +| F-02 | Grok native MCP/tool_call start-input binding and truncation behavior remain PARTIAL/BLOCKED. | +| F-04 | Cross-principal Windows inherited-handle / service-token tests remain BLOCKED; Linux UID test alone is insufficient. | +| F-07 | Formal source recovery is closed; full T01–T34 and V0.6 §§1–18 conformance decision remains pending. | + +## 9. Phase 0 conclusion and next actions + +1. The 31-task inventory shows M1/M2/M4 have broad implementation surfaces, but source presence is not final acceptance. M3's Run→Program retain and Genesis association is atomic through the Run FK + Genesis transaction; the async Store contract discrepancy remains open. M5 has material P0-A gaps (admin termination and reference bridge) plus result-projection closure. M6 still lacks exact-current-main CI, complete fault/performance evidence and native-host acceptance. +2. Do not repeat M1–M5 wholesale. Work only on proven gaps and incomplete evidence. +3. Complete the formal V0.3/V0.6-R5 conformance matrix before changing CF-02/05/06 public contracts. +4. Resolve CF-01 and the M5 result projection boundary before declaring P0-A complete. +5. Run H3-01 and H3-05 under their approved independent security scope; H3-02 requires its tool-capability probe; H3-03 follows H3-02; H3-04 follows H3-03 + H3-05. +6. Keep local verification and GitHub Actions statuses separate. PR #12/#11 runs 43181841181 and 43182753787, current main run 43183321344 and refreshed PR #20 run 43185112963 all completed 7/7. Five stale queued runs remain on the old design branch with zero jobs; they are not evidence that current PR/main jobs are stuck. + +**This crosswalk does not modify the approved plan or authorize changing frozen semantics. All task statuses remain subject to independent review and raw evidence.** diff --git a/docs/interactive-phase0-current-baseline-ci-fallback-20261112.md b/docs/interactive-phase0-current-baseline-ci-fallback-20261112.md new file mode 100644 index 0000000..a22be4c --- /dev/null +++ b/docs/interactive-phase0-current-baseline-ci-fallback-20261112.md @@ -0,0 +1,171 @@ +# FlowKit Phase 0 — Current Baseline & CI Fallback + +**Snapshot:** 2026-11-12 (UTC+8) +**Repository:** \`devcodex-labs/flowkit\` +**Snapshot base:** \`81c7839e9f0abe6f4db20d0fc242253b0baff0ea\` +**Purpose:** Establish one current status record and a reproducible local verification path while GitHub Actions behavior is investigated. This document does not modify the approved V0.3/V0.6-R5 contract, P0-A implementation plan, Program ABI, Store schema, or product semantics. + +## 1. Authority and status vocabulary + +Authority order remains: + +1. Requirements V0.3 (confirmed; T01–T34). +2. Technical Design V0.6-R5 (confirmed; §§1–18). +3. P0-A Implementation Plan V0.1 (confirmed 2026-11-10; M1–M6, 31 file-level tasks). +4. Trusted Host V2 (PR #14; security design supplement, not a replacement for V0.3/V0.6-R5). +5. Current \`main\` and reproducible tests. +6. Roadmaps, audit reports and local verification reports as evidence indexes, never as substitutes for the confirmed source documents. + +Use only: \`PASS\`, \`FAIL\`, \`PARTIAL\`, \`BLOCKED\`, \`NOT-EXECUTED\`, \`NOT-IN-SCOPE\`, \`DEFERRED\`. A local PASS is not a GitHub Actions PASS; a scripted host fixture is not a native-host PASS. + +## 2. Current repository and PR baseline + +### 2.1 Main + +- Current main at snapshot: \`81c7839e9f0abe6f4db20d0fc242253b0baff0ea\`. +- M1–M5 implementation surfaces are present on main. Their presence is not a blanket completion claim for every task or security gate. +- PR #16 (R1/R2 audit docs), PR #18 (R04 fix + regression tests), PR #19 (local verification evidence), PR #12 (C01–C24 acceptance assets) and PR #11 (ChatGPT Pending response evidence) are merged. +- PR #17 and PR #7 are closed without merge; their applicable test work was superseded by #18 and #13 respectively. +- PR #20 is the remaining open Draft; its five documentation files are refreshed against current main 81c7839, and exact-head run 43185112963 passed 7/7. + +### 2.2 Remaining open PRs + +| PR | Branch | HEAD | Current diff against main | Decision | +|---|---|---|---|---| +| [#20 Phase 0 baseline and CI fallback](https://github.com/devcodex-labs/flowkit/pull/20) | \`docs/phase0-current-baseline-ci-fallback-20261112\` | \`c6bff96de2d666dd483499bd4f09009392cd40c3\` (pre-refresh) | Five documentation files; branch is two commits behind current main after #12/#11 merges | Refresh baseline, correct stale PR/CI statuses, rerun exact-head CI, then final review/merge | + + +## 3. Verified local evidence already available + +PR #19 records the following local Windows evidence: + +- Windows NT 10.0.26200.0. +- Node v24.22.2: \`npm run verify\` — 371/371 tests PASS; type, package check and package consumer steps exit 0. +- Node v22.22.2: \`npm run verify\` — 371/371 tests PASS. +- Documentation examples, docs check (after LF normalization), docs build and selected impact/security probes PASS. +- Before the fix, the two R04 tests failed reproducibly on Node 22.22.2 and 24.22.2; after #18, both passed. +- CF-08 was separately reproduced as a non-deterministic compile-options rejection and remains unfixed. + +Limits: these Node versions are not the exact CI pins (22.13.0 and 24.22.1); Ubuntu was not run; Windows ACL / separate OS identities, native Grok tool events, C12/S13/T27 and complete Host Outbox recovery were not proven. Do not upgrade these items to cross-platform or native-host PASS. + +## 4. GitHub Actions diagnosis at this snapshot + +### 4.1 Workflow shape + +\`.github/workflows/ci.yml\` currently has seven jobs: + +- 4 \`verify\` matrix jobs: Ubuntu 24.04 / Windows 2025 × Node 22.13.0 / 24.22.1. +- 2 Linux Host OS-user isolation + M6 integration jobs: Node 22.13.0 / 24.22.1. +- 1 website build/check job. + +The workflow uses: + +\`\`\`yaml +concurrency: + group: ci-\${{ github.workflow }}-\${{ github.ref }} + cancel-in-progress: true +\`\`\` + +New pushes to the same ref can cancel older runs. This is intentional stale-run suppression, but multiple rapid pushes to \`main\` can cancel earlier main runs before their seven jobs finish. + +### 4.2 Observed run facts + +- Main run [43182546605](https://github.com/devcodex-labs/flowkit/actions/runs/43182546605) passed 7/7 on commit \`8ccb53055f640539fd86c524f079867fbdb12713\` after PR #12 merged. +- PR #11 exact updated-head Actions run [43182753787](https://github.com/devcodex-labs/flowkit/actions/runs/43182753787) passed 7/7 on merge tree \`c9b6033c8302cbb205f251f68aba2f45222d7191\`; PR #12 exact updated-head run [43181841181](https://github.com/devcodex-labs/flowkit/actions/runs/43181841181) passed 7/7. Both include the approved current-main implementation and all four Ubuntu/Windows × Node 22/24 verify jobs. +- Current main \`81c7839e9f0abe6f4db20d0fc242253b0baff0ea\` run [43183321344](https://github.com/devcodex-labs/flowkit/actions/runs/43183321344) completed 7/7. Refreshed PR #20 exact-head run [43185112963](https://github.com/devcodex-labs/flowkit/actions/runs/43185112963) also completed 7/7. +- PR #20's prior exact-head run [43174072647](https://github.com/devcodex-labs/flowkit/actions/runs/43174072647) passed 7/7 on the older \`606d055\` baseline. PR #20 is being refreshed to \`81c7839\`; its new exact-head run is the final Phase 0 CI gate. +- Five old runs (43095884874, 43095901291, 43095937558, 43096034832, 43096045021) remain queued on the obsolete design branch with zero jobs and no open PR. Treat them as stale events, not as active current-main work. +- New PR/main runs are now being scheduled and completing; the earlier repository-specific queue symptom has recovered for current work. The five old zero-job records are separate cleanup noise. + +GitHub Status currently reports all systems operational. The Nov 11 incident affecting Actions/Pages/Copilot was marked resolved at 21:21 UTC, with GitHub stating that the Actions backlog had been fully processed. Therefore, do **not** assert a current global GitHub outage without new evidence. A fresh isolated workflow run is required to test current scheduling on this repository. + +Official status: https://www.githubstatus.com/ +Official Actions billing/usage: https://docs.github.com/en/actions/concepts/billing-and-usage + +The repository is now public. GitHub documents standard GitHub-hosted runner usage as free for public repositories. This makes private-repository included-minute exhaustion an unlikely explanation for current standard hosted-runner scheduling; repository/org policy and stale events still require evidence. Do not infer a billing cause. + +### 4.3 PR #11 / #12 final evidence review + +**PR #12 — C01–C24 acceptance assets (merged):** +- Merge commit \`8ccb53055f640539fd86c524f079867fbdb12713\`; exact updated-head run [43181841181](https://github.com/devcodex-labs/flowkit/actions/runs/43181841181) passed 7/7. +- The scripted suite covers C01–C04, C10–C11 and C13–C20. The five ChatGPT-authored responses are recorded fixtures replayed through FlowKit; they are not live model API calls inside CI. +- Normal CI requires all five fixtures; only explicit \`FLOWKIT_C01_PREPARE=1\` permits preparation-only execution. C12/S13, C22/C23 native-host gates remain BLOCKED. + +**PR #11 — ChatGPT Pending response evidence (merged):** +- Merge commit \`81c7839e9f0abe6f4db20d0fc242253b0baff0ea\`; exact updated-head run [43182753787](https://github.com/devcodex-labs/flowkit/actions/runs/43182753787) passed 7/7. +- The experiment publishes a real FlowKit Pending, binds a ChatGPT-authored response delivered via a GitHub tool action to exact \`runId/requestId/taskDigest\`, and then exercises trusted Host signing/CAS, duplicate Ack and same-phase SQLite restart. +- This is M-level recorded response evidence, **not** a FlowKit native JS-tool invocation or native S13. Phase 1/2 use separate temporary DBs; there is no independent provider/session attestation. +- Normal CI requires the complete response fixture; only explicit \`FLOWKIT_S13_PREPARE=1\` permits preparation-only execution. + +Both PRs are merged as test/evidence contributions only. They do not close native Grok C12/S13/T27 or overall P0-A acceptance. + +## 5. CI outage / queue fallback policy + +### 5.1 Continue development without pretending CI passed + +When Actions is unavailable or a run is cancelled/queued: + +1. Continue source review, requirement mapping, implementation and unit-level work in a scoped branch. +2. Run the same repository scripts locally on the exact PR HEAD. +3. Save raw logs and a machine-readable evidence summary. +4. Have a separate reviewer inspect the exact diff and evidence. +5. Keep the GitHub Actions state explicitly \`BLOCKED\` or \`NOT-EXECUTED\`; never relabel local results as CI PASS. +6. Do not repeatedly create meaningless commits to chase a runner. Trigger one clean isolated run after the branch is ready, then inspect its run/job metadata. + +### 5.2 Standard local verification commands + +Run from a clean worktree at the exact PR HEAD: + +\`\`\`bash +node --version +npm --version +git rev-parse HEAD +git status --short +npm ci --ignore-scripts --no-audit --no-fund +npm run verify +npm run docs:examples +npm --prefix website ci --ignore-scripts --no-audit --no-fund +npm run docs:check +npm run docs:build +\`\`\` + +Use the exact CI Node pins where possible: 22.13.0 and 24.22.1. Record the actual versions; do not claim exact-matrix coverage if using 22.22.2/24.22.2. + +### 5.3 OS coverage + +- Windows tests must run on Windows for Windows-specific path, process, ACL, token and handle behavior. +- Linux tests may run locally in Ubuntu 24.04 / WSL2 with distinct OS users for early verification. Label this \`LOCAL-LINUX\`, not GitHub-hosted Ubuntu CI. If the test depends on a hosted-runner-specific behavior, keep it BLOCKED until tested there. +- Do not use a same-UID shell or same-user child process as proof of cross-principal isolation. +- For each OS/Node combination, record: exact commit SHA, OS build, Node/npm version, install command, test command, exit code, TAP summary, log path/hash, and reviewer. + +### 5.4 Merge and release gates + +- Docs-only PRs can be reviewed and merged using local docs/test evidence if the diff is isolated and no repository rules require a successful Actions check. This does not pass product M6. +- Core/runtime/security PRs require local Windows + Linux evidence appropriate to the changed behavior, raw logs, independent diff review and explicit residual-risk record before any interim merge decision. +- Missing online CI remains \`CI-BLOCKED\`; missing OS profile or native-host proof remains \`BLOCKED\`. Neither can be silently waived to declare P0-A complete. +- P0-A/M6 final closure and release still require the approved four-matrix CI and all applicable security/native-host evidence. Do not bypass branch protection or required checks. + +## 6. Current technical findings carried forward + +| Finding | Current status | Next action | +|---|---|---| +| CF-01 \`adminTerminate\` | GAP / high-confidence P0-A implementation gap | Reconcile exact V0.6-R5 + approved M5 task, then implement trusted admin controller/tombstone/CAS/audit and race tests | +| CF-02 SDK API shape | OPEN / contract decision required | Compare V0.6-R5 §4 and approval record; do not add public exports based only on illustrative V0.3 examples | +| CF-05 terminal State projection | OPEN / source-confirmed discrepancy | Separate public model projection from trusted result read; test actual model-facing adapter | +| CF-06 Store sync vs async contract | OPEN / source-confirmed discrepancy | Compare V0.6-R5 §5/§8 and approved M3 design; preserve atomic CAS/receipt semantics | +| CF-07 start input snapshot | FIXED in merged #18 | Retain both regression tests; updated PR #12/#11 merge-tree CI and current-main run 43183321344 passed all four Ubuntu/Windows × Node 22/24 verify cells | +| CF-08 compile options snapshot | REPRODUCED / not fixed | Add isolated regression, then minimal fix and re-run compiler/feature-gate tests | +| F-02 native Grok tool binding | PARTIAL/BLOCKED | H3-02 capability probe before adapter implementation | +| F-04 cross-principal handle isolation | PARTIAL/BLOCKED | H3-05 Windows/Linux OS negative tests | +| F-07 formal contract provenance | Source recovery CLOSED; conformance review PENDING | Complete T01–T34 / §§1–18 traceability; do not mark overall conformance PASS yet | + +## 7. Phase 0 work order + +1. Use this document as the current snapshot; keep historical audit reports immutable. +2. Build the 31-row A-M1-01 through A-M6-06 task crosswalk against current source files, tests, package exports, workflow and raw evidence. Distinguish implementation presence from acceptance PASS. +3. Update the roadmap, formal source register and conformance plan to the current main SHA, preserving their historical baseline notes. +4. PR #11/#12 evidence assets have been independently reviewed and merged; preserve their M/P evidence labels and native-host BLOCKED gates. +5. Continue formal V0.3/V0.6-R5 contract conformance and CF-01/02/05/06/08 closure. +6. Only after Phase 0 inventory is complete, proceed through the corrected Phase 1/2 sequence; H3-01 and H3-05 may proceed independently of Grok tool PoC, while H3-02/03/04 and Phase 4 retain their stated dependencies. + +**No production semantics, ABI, Store schema, confirmed requirements, npm publication or deployment is changed by this baseline record.** diff --git a/docs/interactive-phase2-formal-source-register.md b/docs/interactive-phase2-formal-source-register.md index b41f1e7..59fc172 100644 --- a/docs/interactive-phase2-formal-source-register.md +++ b/docs/interactive-phase2-formal-source-register.md @@ -2,7 +2,9 @@ **状态：SRC-01 / V0.3 与 SRC-02 / V0.6-R5 均已找回并完成 SHA-256/包内清单验证；SRC-03、SRC-04 也已在资料空间目录中定位。** **登记日期：2026-11-12（UTC+8）** -**当前 main：** `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` +**历史源码核验基线：** `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` +**当前 main 快照：** `81c7839e9f0abe6f4db20d0fc242253b0baff0ea`（2026-11-12） +**关联当前状态：** [Phase 0 基线/CI 回退记录](./interactive-phase0-current-baseline-ci-fallback-20261112.md) · [P0-A 31 项交叉核对](./interactive-p0a-31-task-crosswalk-20261112.md) **关联设计：** [Trusted Host V2](./interactive-trusted-host-security-v2-design.md) · [完整路线图](./interactive-full-delivery-roadmap-2026-11-11.md) ## 1. 目标原件 @@ -73,6 +75,14 @@ SRC-01 和 SRC-02 的原件字节、SHA-256 与包内清单已验证；SRC-03 - SRC-04 `FlowKit-Interactive-Final-Convergence-Review-V0.6-R5.zip` 已在 Library `/FlowKit/` 定位，大小 171,006 bytes；本轮任务不依赖该 ZIP 才能开始符合性审查。若后续需要逐条核对历史 R5 复审过程，再单独读取该归档。 - 不能将 V0.3-R2 Draft 或任何摘要当作 V0.6-R5 正式设计；本登记引用的 V0.6-R5 正式文件 SHA-256 为 `09f3a5698e238d10b80efb8e26d1c41e51b663ca620efa70815dd7b52f2106ba`。 + +## 6. Current main and PR snapshot (2026-11-12) + +- Current main: `81c7839e9f0abe6f4db20d0fc242253b0baff0ea`. The original recovery/source-hash evidence above is unchanged; `ae5fc5a` remains a historical source-review baseline only. +- PR #16/#18/#19/#12/#11 are merged; PR #17/#7 are closed without merge. PR #20 remains the sole open Phase 0 documentation PR. +- PR #12 exact-head Actions run 43181841181 and PR #11 exact-head run 43182753787 both passed 7/7. PR #20 refreshed exact-head run 43185112963 passed 7/7 on current main 81c7839. +- Formal source recovery is complete; formal conformance review remains pending. + --- **当前结论：** SRC-01（V0.3）与 SRC-02（V0.6-R5）正式原件均已找回并完成 SHA-256/清单验证；SRC-02 位于本轮上传的 Confirmed Handoff ZIP，9/9 条目通过。SRC-03 在资料空间已定位且有已验证内容副本；SRC-04 在资料空间已定位，但其 ZIP 原始字节尚未读取。原件恢复门禁已解除，下一步应直接开展 V0.3/V0.6-R5 逐条符合性核验，不再要求用户重新上传这两份文件；H3-01/H3-05 的独立 OS 安全工作可以继续，但不得越界修改冻结功能契约。 diff --git a/docs/interactive-v03-v06-conformance-audit-plan.md b/docs/interactive-v03-v06-conformance-audit-plan.md index 2c4e221..018712a 100644 --- a/docs/interactive-v03-v06-conformance-audit-plan.md +++ b/docs/interactive-v03-v06-conformance-audit-plan.md @@ -2,7 +2,9 @@ **状态：已批准进入审查；尚未形成符合性结论。** **基线日期：2026-11-12（UTC+8）** -**当前 main：** `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` +**历史源码核验基线：** `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` +**当前 main 快照：** `81c7839e9f0abe6f4db20d0fc242253b0baff0ea`（2026-11-12） +**当前基线证据：** [Phase 0 current baseline](./interactive-phase0-current-baseline-ci-fallback-20261112.md) · [P0-A 31-task crosswalk](./interactive-p0a-31-task-crosswalk-20261112.md) **需求基线：** Requirements V0.3（已确认） **技术基线：** Technical Design V0.6-R5（已确认） **安全补充：** PR #14 Trusted Host V2（已合并的安全设计记录，不覆盖 V0.3/V0.6-R5） @@ -37,7 +39,8 @@ | Core POC Evidence ZIP | SHA-256 `a6b5201e8327463b57700174297bf986fbc4902c6ef6493710e8cac021f178` | 内部 manifest 28/28 文件条目 PASS；最终 TAP 22/22；9 个 SIGKILL 断点 | | V0.3 Grok Development Handoff ZIP | SHA-256 `8a2f679a796234383b5108d58db16c1016a35c6110301bc4eb7e091ee2ed5826` | 仅含 V0.3-R2 technical draft，不是 V0.6 Confirmed Handoff | | PR #14 | Merge commit `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` | Trusted Host V2 安全设计记录已合并；不等于安全实施通过 | -| 当前 main | `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` | 本次代码符合性核验的当前基线 | +| 历史 main（原 R1/R2 代码基线） | `ae5fc5a42c67eb6e2c229d72ff4d9ac8e3aab3cd` | 保留为历史审查基线，不覆盖 | +| 当前 main（2026-11-12） | `606d055d5f02451fea0b30294889d740ef370d49` | 本轮实现符合性核验对象 | POC 报告的限定结论： - POC-01 PARTIAL：使用受限 JSON Fixture，未接入真实 FlowKit AST/Fields/Graph/Dataflow。 @@ -74,7 +77,8 @@ POC 报告的限定结论： - [x] V0.6 Confirmed Handoff 内部 9/9 SHA256SUMS 校验。 - [x] Core POC Evidence 内部 28/28 文件条目校验。 - [x] 两个独立归档中的 V0.3/V0.6 基线文件哈希一致。 -- [x] 当前 main、PR #14 merge commit 和 CI 记录已登记。 +- [x] 原 R1/R2 历史 main、PR #14 merge commit 和当时 CI 记录已登记。 +- [x] 2026-11-12 当前 main `81c7839`、PR #16/#18/#19/#12/#11 合并、PR #17/#7 关闭状态已登记；PR #12/#11 更新后 exact-head CI 均 7/7 通过；PR #20 基线刷新和最终复审仍待完成。 - [ ] 将原始文件字节、归档哈希、当前 main SHA 和工具版本写入最终审查报告的不可变 Evidence Index。 ### Step 2 — V0.3 需求逐条提取与验收语义锁定 @@ -243,6 +247,15 @@ T25 的赢家选择、未完成分支、迟到副作用、下游状态可见性 - C12/S13 必须有真实原生工具、来源绑定和 Host 安全证据。 - 发布/部署另行授权，不因 P0-A/P0-B 完成自动进行。 +## 3.1 Current-main re-baseline update (2026-11-12) + +- Keep `ae5fc5a` as the historical R1/R2 source-code comparison point; do not rewrite historical evidence. +- Current implementation target is `81c7839e9f0abe6f4db20d0fc242253b0baff0ea`. +- PR #18 fixes CF-07; PR #19 preserves local Windows verification evidence; PR #16 archives the R1/R2 audit; PR #12/#11 evidence assets are now merged. PR #17 is closed as superseded. +- CF-08 is locally reproduced but not fixed. CF-01/02/05/06 remain open with the boundaries recorded in the 31-task crosswalk. +- PR #12 and #11 updated-head CI runs 43181841181 and 43182753787 both passed 7/7. Native Grok C12/S13 remains BLOCKED; do not equate recorded ChatGPT responses with native Host attestation. +- Current main run 43183321344 and refreshed PR #20 run 43185112963 both passed 7/7; PR #20 is ready for final review/merge. + ## 4. 统一状态和停机规则 每条矩阵只能使用： diff --git a/examples/interactive/c01-live-chatgpt.workflow.js b/examples/interactive/c01-live-chatgpt.workflow.js new file mode 100644 index 0000000..fdfe6b2 --- /dev/null +++ b/examples/interactive/c01-live-chatgpt.workflow.js @@ -0,0 +1,62 @@ +import {defineWorkflow,condition,node,terminal} from '@devcodex-labs/flowkit/dsl'; + +// P0-A only. Actual ChatGPT turn outputs come from successive published +// Pending tasks, not prerecorded answers from a simulated LLM. +export default defineWorkflow({ + id:'c01-live-chatgpt-complex', + version:'1.0.0', + name:'C01/C03 live complex model acceptance', + nodes:[ + condition('gate',{ + name:'Verify start permission', + branches:[ + {when:'ready',next:'requirements'}, + {when:'true',next:'declined'}, + ], + }), + node('requirements',{ + name:'Requirements analysis', + agent:'requirements', + inputs:['request'], + output:{type:'object',properties:{requirementSummary:{type:'string'}}}, + retries:0,next:'architecture', + }), + node('architecture',{ + name:'Architecture design', + agent:'architecture', + inputs:['requirementSummary'], + output:{type:'object',properties:{designSummary:{type:'string'}}}, + retries:0,next:'implementation', + }), + node('implementation',{ + name:'Implementation plan', + agent:'implementation', + inputs:['designSummary'], + output:{type:'object',properties:{executionPlan:{type:'string'}}}, + retries:0,next:'review', + }), + node('review',{ + name:'Independent review', + agent:'review', + inputs:['executionPlan'], + output:{ + type:'object', + properties:{ + reviewApproved:{type:'boolean'}, + reviewNotes:{type:'string'}, + }, + }, + retries:1, + next:'decision', + }), + condition('decision',{ + name:'Review decision', + branches:[ + {when:'reviewApproved',next:'done'}, + {when:'true',next:'declined'}, + ], + }), + terminal('done',{name:'Accepted',status:'succeeded'}), + terminal('declined',{name:'Declined',status:'failed'}), + ], +}); diff --git a/examples/interactive/s13-chatgpt.workflow.js b/examples/interactive/s13-chatgpt.workflow.js new file mode 100644 index 0000000..bb56ba5 --- /dev/null +++ b/examples/interactive/s13-chatgpt.workflow.js @@ -0,0 +1,33 @@ +import {defineWorkflow,condition,node,terminal} from '@devcodex-labs/flowkit/dsl'; + +export default defineWorkflow({ + id:'s13-chatgpt-tool-bridge', + version:'0.1.0', + name:'ChatGPT 实际工具链 S13 验证', + nodes:[ + condition('gate',{ + name:'确定进入审核节点', + branches:[ + {when:'ready',next:'review'}, + {when:'true',next:'rejected'}, + ], + }), + node('review',{ + name:'分析 FlowKit 运行边界', + agent:'reviewer', + inputs:['request','constraints'], + output:{ + type:'object', + properties:{ + summary:{type:'string'}, + nextAction:{type:'string'}, + flowkitBoundary:{type:'string'}, + }, + }, + retries:0, + next:'done', + }), + terminal('done',{name:'工作流成功',status:'succeeded'}), + terminal('rejected',{name:'未满足进入条件',status:'failed'}), + ], +}); diff --git a/examples/interactive/skills/reviewer.md b/examples/interactive/skills/reviewer.md new file mode 100644 index 0000000..a9e01a8 --- /dev/null +++ b/examples/interactive/skills/reviewer.md @@ -0,0 +1,11 @@ +# FlowKit Host 公开 Agent 任务 + +你是本次 FlowKit S13 验证中实际参与执行的 ChatGPT 模型。 + +请根据公开的 `request` 和 `constraints`，返回一个 JSON 对象： +- `summary`：简短说明为什么 FlowKit 的工作流应该由 JavaScript Program 约束执行。 +- `nextAction`：一项可以直接验证的下一步动作。 +- `flowkitBoundary`：说明 FlowKit 与 Host/Context 的职责边界。 + +禁止请求或输出 Host 的私钥、SQLite 数据库、完整 State 或未公开内容。 +这份说明只是公开任务指令，不构成 Host 对模型来源的独立认证。 diff --git a/tests/fixtures/c01-live-chatgpt-turns.json b/tests/fixtures/c01-live-chatgpt-turns.json new file mode 100644 index 0000000..4615004 --- /dev/null +++ b/tests/fixtures/c01-live-chatgpt-turns.json @@ -0,0 +1,49 @@ +[ + { + "index": 0, + "nodeId": "requirements", + "requestId": "req:21438c34374576a97626dca57822f8818b330b5390251afebac979fe45ee6373", + "taskDigest": "sha256:b48e9fe5d58be4db39b122704761d396e38c0c8859aad010aecf5aa8a57bb138", + "result": { + "requirementSummary": "目标是在不扩大 FlowKit 职责的前提下，对 Interactive P0-A 建立可审计的复杂验收。必须覆盖四个以上串行 Agent、两个以上条件路由、上下游结构化数据传递、Schema 校验失败后的有界重试、Host 崩溃恢复、SQLite CAS 幂等、签名安全隔离、容量边界与三十个以上节点的长链。每个场景分别记录 Program 哈希、真实节点顺序、请求身份、原始回执版本、Journal 连续性和预期终态。脚本协议测试与真实模型测试必须独立标注，未获得 Grok Build 原生会话来源认证时不能宣称 S13 验收通过；业务文件存储及复杂上下文关联仍属于用户配置的 Host 或可选 Context。" + } + }, + { + "index": 1, + "nodeId": "architecture", + "requestId": "req:360b12eb62cdc3e6df997415de82522505a96b36a2cf7ab73b4d118007beb5e3", + "taskDigest": "sha256:2889244e0714d3fc2f845a74733a6c81523fce5a5eb44e8f13ee127aaae5bdac", + "result": { + "designSummary": "验收架构采用控制、执行与上下文三条明确边界：FlowKit 编译现有 DSL 为不可变的 JavaScript Program，以条件路由、Agent Pending、重试次数、Checkpoint 与 SQLite 原子 CAS 控制状态迁移；可信 Host 负责实际模型或工具调用、Ed25519 回执签名、私有凭证隔离、结果校验与业务产物的用户自定义存储；可选 Context 仅按需提供已有资料和关联引用，不参与控制下一节点。测试层划分为可复现的脚本协议回归、真实 ChatGPT 逐节点回复，以及 Grok Build 原生会话和来源认证三类证据。所有场景分别检查执行顺序、输入最小披露、错误终态、可恢复性和 Journal，不以测试数量或脚本执行成功冒充真实模型 S13 门禁。" + } + }, + { + "index": 2, + "nodeId": "implementation", + "requestId": "req:78258c27df23723def94c945aaab4d0dc2854f2fe3280c5f3fb3a4c6af131084", + "taskDigest": "sha256:341c06092fa85738c9f4295be450932ec48d530e758a80cd4e043b04f945d900", + "result": { + "executionPlan": "实施分为三组证据。第一组使用真正编译的 P0-A JavaScript Program、SQLite 和可信 Host，构造四个串行 Agent 与两个条件节点，逐项断言输入只来自前序已确认的 State，分别执行通过、拒绝和缺失条件三条路由；向审查节点注入错误 Schema，确认不写入 State、生成新的 requestId 后再次提交合法回执，重试耗尽须持久化 FK_R004。第二组运行三十六 Agent 的脚本长链，检查每条 Journal、CAS 版本和历史回执查询；分别注入 SIGKILL、双进程竞争、Program 或 Skill 漂移、签名伪造和容量超限，验证恢复与失败封闭。第三组让真实 ChatGPT 逐次读取公开 Pending、独立产出五个模型回合，并留存原始任务摘要、提交记录和终态证据。真正的 Grok Build 模型会话及独立来源认证另行验证，无法提供原生日志时持续标记 BLOCKED，不因 CI 全绿宣称完成。" + } + }, + { + "index": 3, + "nodeId": "review", + "requestId": "req:43585b79c64e2980f1d8b0de4e1fc12b1925dc6aaca4b3a936c4548754d90a55", + "taskDigest": "sha256:da7a90e4b6e758dc2c73587f4760af28135ef53685aea7af35f6476d5b28ace5", + "result": { + "reviewApproved": "true", + "reviewNotes": "独立评审认为方案具备可执行的分层证据路线，但还应要求 C05/C06 直接引用真实 SIGKILL 与双进程 CAS 日志，不能只凭文字宣称恢复成功；C07 要明确 Skill 字节漂移与 Host 绑定版本的区别，C12 必须保留 Grok Build 原生会话来源认证门禁。本轮为受控负面测试，故意把 Schema 要求的布尔型 reviewApproved 写成字符串，预期 FlowKit 只能接纳回执身份、拒绝结果写入 State，并发布新的 review Pending，不能错误进入成功终态。" + } + }, + { + "index": 4, + "nodeId": "review", + "requestId": "req:5005e5df646511a14b1c70b87bc3416ff2399c8cc70d39b2f04bba812f24c092", + "taskDigest": "sha256:decd43a929d6d758aaaa629cc1efdf832f6020cf0662728d278ad35284191490", + "result": { + "reviewApproved": true, + "reviewNotes": "针对本次实际发布的第二次审查 Pending，前一轮失败原因明确是 reviewApproved 类型错误，必须改为布尔值而不是字符串。复审认为上游计划可以作为 FlowKit P0-A 复杂验收的执行基线：它区分了四 Agent 串行与两次条件路由、Schema 失败重试、36 Agent 长链、SQLite CAS 与 Journal、一致性预算、SIGKILL 和多进程竞争，并要求保存 Program 哈希与请求身份。建议先将脚本协议结果和当前五次真实 ChatGPT 模型回合分别列为不同证据层，再补验真实 Grok Build 原生 JS 工具、模型会话来源及独立权限隔离。这里批准的是测试方案可继续实施，不代表 C12 或整个 M6 S13 已经验收通过，也不能因此合并 M6。" + } + } +] diff --git a/tests/fixtures/s13-chatgpt-reply.json b/tests/fixtures/s13-chatgpt-reply.json new file mode 100644 index 0000000..0800cbe --- /dev/null +++ b/tests/fixtures/s13-chatgpt-reply.json @@ -0,0 +1,10 @@ +{ + "runId": "s13-chatgpt-bridge-run-20261111", + "requestId": "req:6bed7ea39b96e81b99100ffe69bb174ba38533d42f3bbdbefad6239cee0ac829", + "taskDigest": "sha256:96733a8e513b08020acc42ef5dabd93413d52b452d8b982d40afcf3b81ac2876", + "result": { + "summary": "FlowKit 将执行顺序、条件分支、失败路径和终态编译为可执行的 JavaScript Program，由代码而不是模型临时决定下一节点；Agent 只提交当前节点的结果，可信 Host 依据 Program、CAS 与 Journal 验证每次推进。", + "nextAction": "在可信 Host 运行本次 Condition→Agent→Terminal 工作流，核对已持久化 Pending 的 requestId 与任务摘要；接纳模型结构化输出后核验签名回执、最终 SQLite State、Journal 连续性、宿主重启恢复及重复回执的原始 Ack。", + "flowkitBoundary": "FlowKit 负责工作流编译、程序化路由、Checkpoint、回执幂等和恢复；Host 负责模型与 JS 工具调用、来源认证、安全隔离以及用户配置的产物存储；可选 Context 模块负责上下文引用和复杂数据关联，不能替代 Program 决定流程。" + } +} diff --git a/tests/interactive/c01-live-chatgpt.test.mjs b/tests/interactive/c01-live-chatgpt.test.mjs new file mode 100644 index 0000000..ce457e5 --- /dev/null +++ b/tests/interactive/c01-live-chatgpt.test.mjs @@ -0,0 +1,218 @@ +/** + * Recorded ChatGPT-response evidence: four Agent nodes + deliberate schema retry, + * two Conditions. Responses are authored externally after inspecting the exact + * Pending in prior Actions logs, then replayed through FlowKit Program/SQLite/ + * Trusted Host. This test does not call a model API or native FlowKit JS tool. + * + * Normal CI requires all five recorded responses. Only explicit preparation + * mode (FLOWKIT_C01_PREPARE=1) may stop at a missing Pending for the next turn. + * SQLite is reopened within each CI run; separate Actions runs do NOT share a + * physical DB. GitHub tool identity is NOT provider-native session attestation. + */ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import {readFile,mkdtemp,rm} from 'node:fs/promises'; +import {fileURLToPath} from 'node:url'; +import {tmpdir} from 'node:os'; +import path from 'node:path'; +import {generateKeyPairSync,randomBytes,sign} from 'node:crypto'; +import {compileInteractive} from '../../src/interactive/compile.mjs'; +import {createSqliteCheckpointStore} from '../../src/interactive/sqlite-store.mjs'; +import {createTrustedInteractiveHost} from '../../src/interactive/trusted-host.mjs'; +import {createAgentReceiptStatement,encodeAgentReceiptStatement} + from '../../src/interactive/host-attestation.mjs'; +import {canonicalBytes,sha256Bytes} from '../../src/interactive/canonical.mjs'; + +const workflowFile=fileURLToPath(new URL( + '../../examples/interactive/c01-live-chatgpt.workflow.js',import.meta.url)); +const repliesFile=fileURLToPath(new URL( + '../fixtures/c01-live-chatgpt-turns.json',import.meta.url)); +const NOW=1794400000000; +const runId='c01-live-multi-agent-20261111'; +const namespace='complex-live-chatgpt'; +const expectedPath=['requirements','architecture','implementation','review','review']; +const instructions={ + requirements:"分析用户的 FlowKit 复杂验收要求，给出严谨、可操作的需求说明。只能回答一个 JSON 对象 {requirementSummary:string}，不少于 60 个汉字，不包含未授权的内部信息。", + architecture:"基于公开的 requirementSummary 设计精简的 FlowKit 复杂验收架构，分清 FlowKit JS Program、Host、Context 的职责。只能回答 JSON {designSummary:string}，不少于 60 个汉字。", + implementation:"基于上个节点公开的 designSummary，提出可复现的 C01-C12 复杂测试实施步骤，包含真实模型场景和故障/恢复验证。只能回答 JSON {executionPlan:string}，不少于 60 个汉字。", + review:"独立评审公开的 executionPlan 的可执行性、证据质量和宿主安全边界。只回答 JSON {reviewApproved:boolean,reviewNotes:string}，reviewNotes 不少于 60 个汉字；必须依据实际上游数据，不得凭空宣称 Grok Build 已执行或 S13 已通过。", +}; +const outputShape={ + requirements:{requirementSummary:'string'}, + architecture:{designSummary:'string'}, + implementation:{executionPlan:'string'}, + review:{reviewApproved:'boolean',reviewNotes:'string'}, +}; +function validate(schema,output){ + if(!output||typeof output!=='object'||Array.isArray(output))return ['Agent result must be object']; + const allowed=Object.keys(schema.properties??{}).sort(); + if(allowed.join('|')!==Object.keys(output).sort().join('|'))return ['Agent output keys differ from Schema']; + for(const field of allowed){ + if(typeof output[field]!==schema.properties[field].type)return [field+' must be '+schema.properties[field].type]; + if(typeof output[field]==='string'&&output[field].length<70){ + return [field+' must be at least 70 characters']; + } + } + return []; +} +const originalRequest='建立 FlowKit Interactive P0-A 的 C01–C12 复杂验收矩阵，验证多 Agent 顺序、条件路由、Schema 重试、SQLite 恢复、签名、安全与长链；不得越界实现 Context 或通用产物存储。'; + +test('C01/C03 recorded ChatGPT responses bind to exact Pending and replay through Program/Host',async()=>{ + const root=await mkdtemp(path.join(tmpdir(),'fk-c01-live-')); + const file=path.join(root,'run.sqlite'); + let store; + try{ + const compiled=await compileInteractive(workflowFile,{ + output:'memory',featureStage:'P0-A',skillCheck:false, + }); + assert.equal(compiled.diagnostics.some(x=>x.severity==='error'),false, + JSON.stringify(compiled.diagnostics)); + assert.ok(compiled.code&&compiled.programIdentity); + const keys=generateKeyPairSync('ed25519'); + const privateCanary=randomBytes(24).toString('hex'); + const runtime={ + skillResolver:async d=>{ + assert.equal(typeof instructions[d.nodeId],'string'); + return instructions[d.nodeId]; + }, + validateResult:validate, + }; + const hostFor=s=>createTrustedInteractiveHost({ + store:s,runtime,policy:{deploymentId:'c01-live-20261111', + bindingVersions:{skill:'live-c01-v1',validator:'live-c01-v1'}}, + trustedPublicKeys:{'complex-live-host':keys.publicKey}, + trustedArtifacts:[compiled.programIdentity.artifactHash], + allowedInputKeys:['request','requirementSummary','designSummary','executionPlan'], + disclosurePolicyId:'allow-public-c01-inputs', + authorizeDisclosure:async({task})=> + task?.runId===runId&&expectedPath.includes(task?.nodeId), + now:()=>NOW, + }); + store=createSqliteCheckpointStore(file,{namespace}); + let host=hostFor(store); + let frame=await host.start({ + programBytes:Buffer.from(compiled.code,'utf8'), + programIdentity:compiled.programIdentity, + runId,startKey:'complex-live-c01-start', + state:{ready:true,request:originalRequest,hostSecret:privateCanary}, + }); + assert.equal(frame.status,'waiting'); + let replies=[]; + try{ + replies=JSON.parse(await readFile(repliesFile,'utf8')); + }catch(error){ + if(error?.code!=='ENOENT')throw error; + } + const prepareMode=process.env.FLOWKIT_C01_PREPARE==='1'; + assert.ok(Array.isArray(replies)&&replies.length<=expectedPath.length); + assert.ok(prepareMode||replies.length===expectedPath.length, + 'Normal CI requires all five recorded ChatGPT responses; use FLOWKIT_C01_PREPARE=1 only for explicit Pending preparation'); + const resultDigests=[]; + const requestIds=[]; + let rejectedReviewAttempts=0; + for(let index=0;index0); + console.log('C01_CHATGPT_RECORDED_RESPONSES_ACCEPTED '+JSON.stringify({ + scenarioIds:['C01','C03'],execution:'CHATGPT_AUTHORED_FIXTURE_REPLAY', + nativeHostAttested:false,recordedModelResponses:replies.length, + resultDigests,requestIds, + programHash:compiled.programIdentity.artifactHash, + finalStateHash:sha256Bytes(canonicalBytes(frame.state)), + terminal:frame.status,nodeId:frame.terminal.nodeId, + journalRevisions:journal.length,restartedSQLiteWithinEachCI:true, + retriesObserved:rejectedReviewAttempts, + outputLength:last.result.reviewNotes.length, + note:'Five ChatGPT-authored responses replayed against exact Pending identities; GitHub tool identity is not native session attestation and each CI run recreates its DB', + })); + }finally{ + try{store?.close()}finally{await rm(root,{recursive:true,force:true})} + } +}); diff --git a/tests/interactive/complex-c01-c12.test.mjs b/tests/interactive/complex-c01-c12.test.mjs new file mode 100644 index 0000000..807e64e --- /dev/null +++ b/tests/interactive/complex-c01-c12.test.mjs @@ -0,0 +1,605 @@ +/** + * C01/C02/C03/C04/C10/C11: actual FlowKit Program, trusted Host and SQLite, + * but Agent results are SCRIPTED, never claim a real model session here. + */ +import test,{after} from 'node:test'; +import assert from 'node:assert/strict'; +import {mkdtemp,rm} from 'node:fs/promises'; +import {tmpdir} from 'node:os'; +import path from 'node:path'; +import {generateKeyPairSync,sign} from 'node:crypto'; +import {analyzeSource} from '../../src/compiler/analyze.mjs'; +import {lowerInteractive} from '../../src/compiler/interactive/lower.mjs'; +import {emitInteractive} from '../../src/compiler/interactive/emit.mjs'; +import {createSqliteCheckpointStore} from '../../src/interactive/sqlite-store.mjs'; +import {createTrustedInteractiveHost} from '../../src/interactive/trusted-host.mjs'; +import {createAgentReceiptStatement,encodeAgentReceiptStatement} + from '../../src/interactive/host-attestation.mjs'; +import {canonicalBytes,sha256Bytes} from '../../src/interactive/canonical.mjs'; + +const roots=[]; +after(async()=>{for(const dir of roots) await rm(dir,{recursive:true,force:true})}); +const NOW=1794400000000; +const DSL="import {defineWorkflow,condition,node,terminal} from '@devcodex-labs/flowkit/dsl';\n"; +const schema=(field,type='string')=> + "{type:'object',properties:{"+field+":{type:'"+type+"'}}}"; +function complexSource(retries=1) { + return DSL+"export default defineWorkflow({id:'c01-complex',version:'1.0.0',name:'C01 Complex',nodes:["+ + "condition('gate',{name:'Gate',branches:[{when:'ready',next:'requirements'},{when:'true',next:'declined'}]}),"+ + "node('requirements',{name:'Requirements',agent:'analyst',inputs:['request'],output:"+ + schema('requirementId')+",retries:0,next:'design'}),"+ + "node('design',{name:'Design',agent:'designer',inputs:['requirementId'],output:"+ + schema('designId')+",retries:0,next:'plan'}),"+ + "node('plan',{name:'Plan',agent:'planner',inputs:['designId'],output:"+ + schema('planId')+",retries:0,next:'review'}),"+ + "node('review',{name:'Review',agent:'reviewer',inputs:['planId'],output:"+ + schema('reviewApproved','boolean')+",retries:"+retries+",next:'decision'}),"+ + "condition('decision',{name:'Decision',branches:[{when:'reviewApproved',next:'done'},{when:'true',next:'declined'}]}),"+ + "terminal('done',{name:'Done',status:'succeeded'}),"+ + "terminal('declined',{name:'Declined',status:'failed'})]});"; +} +function longSource(count=36){ + const nodes=[]; + for(let i=0;ix.severity==='error'),false, + JSON.stringify(a.diagnostics)); + const ir=lowerInteractive(a); + assert.ok(ir.ir,JSON.stringify(ir.diagnostics)); + const e=emitInteractive(ir.ir); + assert.ok(e.code,JSON.stringify(e.diagnostics)); + return {programBytes:Buffer.from(e.code,'utf8'),programIdentity:e.programIdentity}; +} +function validator(schema,output){ + if(!output||typeof output!=='object'||Array.isArray(output))return ['not an object']; + const properties=schema.properties??{}; + const keys=Object.keys(properties).sort(); + if(Object.keys(output).sort().join('|')!==keys.join('|'))return ['wrong fields']; + for(const k of keys)if(typeof output[k]!==properties[k].type)return ['wrong type: '+k]; + return []; +} +async function fixture(t,{ + source=complexSource(), + maxStagesPerCall=128, + allowedInputKeys=['request','requirementId','designId','planId'], +}={}){ + const root=await mkdtemp(path.join(tmpdir(),'fk-complex-')); + roots.push(root); + const store=createSqliteCheckpointStore(path.join(root,'run.sqlite'),{ + namespace:'complex-acceptance', + }); + t.after(()=>store.close()); + const compiled=await compile(source); + const keys=generateKeyPairSync('ed25519'); + const runtime={ + skillResolver:async({nodeId})=>'Perform declared step '+nodeId, + validateResult:validator, + }; + const makeHost=()=>createTrustedInteractiveHost({ + store,runtime,policy:{deploymentId:'complex-c01-c12',bindingVersions:{ + resolver:'v1',validator:'v1', + }}, + trustedPublicKeys:{'trusted-host':keys.publicKey}, + trustedArtifacts:[compiled.programIdentity.artifactHash], + allowedInputKeys, + disclosurePolicyId:'complex-test-public', + authorizeDisclosure:async()=>true,now:()=>NOW,maxStagesPerCall, + }); + return {root,store,keys,compiled,makeHost,host:makeHost()}; +} +function audit(f,runId){ + const cp=f.store.load(runId),journal=f.store.listJournal(runId); + assert.equal(journal.length,cp.revision+1); + for(let i=0;i{ + const f=await fixture(t); + const first=await start(f,{runId:'C01-four',state:{ + ready:true,request:'software task',secret:'HOST-PRIVATE-DO-NOT-SHARE', + }}); + const {frame,trace}=await fourSteps(f,first); + assert.equal(frame.status,'succeeded'); + assert.equal(frame.terminal.nodeId,'done'); + assert.deepEqual(frame.state,{ + ready:true,request:'software task',secret:'HOST-PRIVATE-DO-NOT-SHARE', + requirementId:'REQ-001',designId:'DES-001',planId:'PLAN-001',reviewApproved:true, + }); + assert.equal(new Set(trace.map(x=>x.requestId)).size,4); + evidence('C01',f,frame,{nodes:trace.map(x=>x.nodeId),signedReceipts:4}); +}); + +test('C02 false/default/unknown Condition routes do not dispatch unreachable Agent',async t=>{ + const f=await fixture(t); + const early=await start(f,{runId:'C02-early',state:{ + ready:false,request:'software task', + }}); + assert.equal(early.status,'failed'); + assert.equal(early.terminal.nodeId,'declined'); + assert.equal(f.store.load('C02-early').receipts.length,0); + audit(f,'C02-early'); + const missing=await start(f,{runId:'C02-missing',state:{ + request:'software task', + }}); + assert.equal(missing.status,'errored'); + assert.equal(missing.terminal.causeCode,'FK_R002'); + assert.equal(f.store.load('C02-missing').receipts.length,0); + audit(f,'C02-missing'); + const pending=await start(f,{runId:'C02-late'}); + const {frame,trace}=await fourSteps(f,pending,{ + reviewResult:{reviewApproved:false}, + }); + assert.equal(frame.status,'failed'); + assert.equal(frame.terminal.nodeId,'declined'); + assert.equal(trace.length,4); + evidence('C02',f,frame,{ + routes:['ready:false -> declined','ready:missing -> FK_R002', + 'reviewApproved:false -> declined'], + earlyDispatches:0, + }); +}); + +test('C03 invalid review schema does not mutate State, retries with new requestId',async t=>{ + const f=await fixture(t); + let frame=await start(f,{runId:'C03-retry'}); + for(const id of ['requirements','design','plan']){ + frame=(await respond(f,frame,answers[id])).frame; + } + const prior=await task(f,frame); + const first=await respond(f,frame,{reviewApproved:'wrong-type'}); + frame=first.frame; + assert.equal(frame.status,'waiting'); + assert.equal(frame.pending.nodeId,'review'); + assert.equal(frame.pending.attempt,2); + assert.notEqual(frame.pending.requestId,prior.requestId); + assert.equal(Object.hasOwn(f.store.load(frame.runId).state,'reviewApproved'),false); + const dup=await f.host.accept({ + receipt:first.receipt,attestation:first.attestation, + }); + assert.equal(dup.duplicate,true); + assert.equal(dup.acceptedRevision,first.acceptedRevision); + assert.equal(dup.pending.requestId,frame.pending.requestId); + const second=await respond(f,frame,{reviewApproved:true}); + assert.equal(second.frame.status,'succeeded'); + assert.equal(second.frame.state.reviewApproved,true); + evidence('C03',f,second.frame,{attempts:[1,2],invalidStateMerged:false, + firstAck:first.acceptedRevision,secondAck:second.acceptedRevision}); +}); + +test('C04 repeated invalid review schema consumes exactly retries+1 and FK_R004',async t=>{ + const f=await fixture(t); + let frame=await start(f,{runId:'C04-error'}); + for(const id of ['requirements','design','plan']){ + frame=(await respond(f,frame,answers[id])).frame; + } + const attempts=[]; + for(let n=0;n<2;n++){ + const p=await task(f,frame); + assert.equal(p.nodeId,'review'); + attempts.push(p.attempt); + frame=(await respond(f,frame,{invalid:true})).frame; + } + assert.deepEqual(attempts,[1,2]); + assert.equal(frame.status,'errored'); + assert.equal(frame.terminal.causeCode,'FK_R004'); + assert.equal(Object.hasOwn(frame.state,'reviewApproved'),false); + evidence('C04',f,frame,{attempts,causeCode:'FK_R004'}); +}); + +test('C10 36 Agent chain keeps 16 recent receipt cache but all 36 Ack identities',async t=>{ + const count=36; + const f=await fixture(t,{ + source:longSource(count), + allowedInputKeys:['request',...Array.from({length:count},(_,i)=>'step'+i)], + }); + let frame=await start(f,{runId:'C10-long'}); + const accepted=[]; + for(let i=0;i{ + const compared=[]; + for(const budget of [1,128]){ + const f=await fixture(t,{maxStagesPerCall:budget}); + const first=await start(f,{runId:'C11-budget-'+budget}); + const trace=[]; + const {frame}=await fourSteps(f,first,{trace}); + assert.equal(frame.status,'succeeded'); + const {journal}=audit(f,frame.runId); + compared.push({ + budget,programHash:f.compiled.programIdentity.artifactHash, + state:frame.state,stateHash:sha256Bytes(canonicalBytes(frame.state)), + trace:trace.map(x=>({id:x.nodeId,attempt:x.attempt})), + revision:frame.revision,journalRevisions:journal.length, + }); + } + assert.equal(compared[0].programHash,compared[1].programHash); + assert.equal(compared[0].stateHash,compared[1].stateHash); + assert.deepEqual(compared[0].trace,compared[1].trace); + assert.ok(compared[0].revision>compared[1].revision); + console.log('FLOWKIT_COMPLEX_ACCEPTANCE '+JSON.stringify({ + scenarioId:'C11',execution:'SCRIPTED_TRUSTED_HOST',modelCalls:0, + programHash:compared[0].programHash, + stateHash:compared[0].stateHash, + runs:compared.map(x=>({budget:x.budget,revision:x.revision, + journalRevisions:x.journalRevisions})), + result:'PASS', + })); +}); + + +// C13-C20 extend the FROZEN C01-C12 set with compound failure/interleaving +// conditions. All model outputs below are SCRIPTED; zero real model calls. +test('C13 cross-budget branch product: true/false/missing and late rejection',async t=>{ + const observations=[]; + for(const budget of [1,128]){ + const f=await fixture(t,{maxStagesPerCall:budget}); + const declined=await start(f,{runId:'C13-early-'+budget, + state:{ready:false,request:'software task'}}); + assert.equal(declined.status,'failed'); + assert.equal(declined.terminal.nodeId,'declined'); + assert.equal(f.store.load(declined.runId).receipts.length,0); + + const missing=await start(f,{runId:'C13-missing-'+budget, + state:{request:'software task'}}); + assert.equal(missing.status,'errored'); + assert.equal(missing.terminal.causeCode,'FK_R002'); + assert.equal(f.store.load(missing.runId).receipts.length,0); + + const falseFirst=await start(f,{runId:'C13-late-false-'+budget}); + const rejected=await fourSteps(f,falseFirst,{ + reviewResult:{reviewApproved:false}}); + assert.equal(rejected.frame.status,'failed'); + assert.equal(rejected.frame.terminal.nodeId,'declined'); + assert.equal(rejected.trace.length,4); + + const trueFirst=await start(f,{runId:'C13-late-true-'+budget}); + const approved=await fourSteps(f,trueFirst); + assert.equal(approved.frame.status,'succeeded'); + assert.equal(approved.frame.terminal.nodeId,'done'); + const stateHash=sha256Bytes(canonicalBytes(approved.frame.state)); + const rejectedStateHash=sha256Bytes(canonicalBytes(rejected.frame.state)); + for(const runId of [declined.runId,missing.runId, + rejected.frame.runId,approved.frame.runId])audit(f,runId); + observations.push({budget,stateHash,rejectedStateHash, + statuses:[declined.status,missing.status,rejected.frame.status, + approved.frame.status]}); + } + assert.equal(observations[0].stateHash,observations[1].stateHash); + assert.equal(observations[0].rejectedStateHash, + observations[1].rejectedStateHash); + assert.deepEqual(observations.map(x=>x.statuses),[ + ['failed','errored','failed','succeeded'], + ['failed','errored','failed','succeeded']]); + console.log('FLOWKIT_COMPLEX_ACCEPTANCE '+JSON.stringify({ + scenarioId:'C13',execution:'SCRIPTED_TRUSTED_HOST',modelCalls:0, + branchCases:4,stageBudgets:[1,128],observations,result:'PASS', + })); +}); + +test('C14 tampered/oversized signed-looking replies halfway through chain are no-ops',async t=>{ + const f=await fixture(t); + let frame=await start(f,{runId:'C14-forged'}); + frame=(await respond(f,frame,answers.requirements)).frame; + const pending=await task(f,frame); + assert.equal(pending.nodeId,'design'); + const valid=receiptFor(pending,{designId:'DES-001'}); + const attestation=attest(f,valid); + const cp=canonicalBytes(f.store.load(frame.runId)); + const journal=f.store.listJournal(frame.runId); + const rejected=[ + {receipt:{...valid,result:{designId:'swapped'}},attestation}, + {receipt:{...valid,attempt:pending.attempt+1},attestation}, + {receipt:{...valid,requestId:'forged-request'},attestation}, + {receipt:{...valid,result:{designId:'x'.repeat(2_100_000)}}, + attestation}, + {receipt:valid,attestation:{...attestation, + signature:'A'.repeat(86)}}, + ]; + for(const entry of rejected){ + await assert.rejects(f.host.accept(entry), + {code:'FK_I_PROVENANCE_REJECTED'}); + assert.deepEqual(canonicalBytes(f.store.load(frame.runId)),cp); + assert.deepEqual(f.store.listJournal(frame.runId),journal); + } + const accepted=await f.host.accept({receipt:valid,attestation}); + frame=await settle(f.host,accepted); + assert.equal(frame.pending.nodeId,'plan'); + frame=(await respond(f,frame,answers.plan)).frame; + frame=(await respond(f,frame,answers.review)).frame; + assert.equal(frame.status,'succeeded'); + assert.equal(frame.state.designId,'DES-001'); + evidence('C14',f,frame,{rejectedBodies:rejected.length, + noMutationBeforeValidReceipt:true}); +}); + +test('C15 lost Ack replay after Host rebuild and after downstream Terminal is immutable',async t=>{ + const f=await fixture(t); + let frame=await start(f,{runId:'C15-lost-ack'}); + const first=await respond(f,frame,answers.requirements); + frame=first.frame; + assert.equal(frame.pending.nodeId,'design'); + const cp=canonicalBytes(f.store.load(frame.runId)); + const before=f.store.listJournal(frame.runId); + const rebuilt=f.makeHost(); + const duplicate=await rebuilt.accept({ + receipt:first.receipt,attestation:first.attestation}); + assert.equal(duplicate.duplicate,true); + assert.equal(duplicate.acceptedRevision,first.acceptedRevision); + assert.deepEqual(canonicalBytes(f.store.load(frame.runId)),cp); + assert.deepEqual(f.store.listJournal(frame.runId),before); + for(const id of ['design','plan','review']){ + frame=(await respond(f,frame,answers[id])).frame; + } + assert.equal(frame.status,'succeeded'); + const terminalRevision=f.store.load(frame.runId).revision; + const after=await f.makeHost().accept({ + receipt:first.receipt,attestation:first.attestation}); + assert.equal(after.duplicate,true); + assert.equal(after.acceptedRevision,first.acceptedRevision); + assert.equal(f.store.load(frame.runId).revision,terminalRevision); + evidence('C15',f,frame,{lostAckRecovered:true, + replayAfterTerminal:true}); +}); + +test('C16 signed Host call-failed in late Agent retries safely and does not skip route',async t=>{ + const f=await fixture(t); + let frame=await start(f,{runId:'C16-provider-failure'}); + for(const id of ['requirements','design','plan']){ + frame=(await respond(f,frame,answers[id])).frame; + } + const p=await task(f,frame); + const failure={runId:p.runId,requestId:p.requestId,nodeId:p.nodeId, + attempt:p.attempt,kind:'agent-call-failed', + failure:{message:'provider unavailable'}}; + const proof=attest(f,failure); + const failed=await f.host.accept({receipt:failure,attestation:proof}); + frame=await settle(f.host,failed); + assert.equal(frame.status,'waiting'); + assert.equal(frame.pending.nodeId,'review'); + assert.equal(frame.pending.attempt,2); + assert.notEqual(frame.pending.requestId,p.requestId); + assert.equal(Object.hasOwn(f.store.load(frame.runId).state,'reviewApproved'),false); + const duplicate=await f.host.accept({receipt:failure,attestation:proof}); + assert.equal(duplicate.duplicate,true); + assert.equal(duplicate.acceptedRevision,failed.acceptedRevision); + frame=(await respond(f,frame,answers.review)).frame; + assert.equal(frame.status,'succeeded'); + evidence('C16',f,frame,{failureReceiptAccepted:true, + attempt2:true,staleAckImmutable:true}); +}); + +test('C17 Host policy drift at intermediate Pending rejects before mutation; old Host completes',async t=>{ + const f=await fixture(t); + let frame=await start(f,{runId:'C17-policy'}); + for(const id of ['requirements','design']){ + frame=(await respond(f,frame,answers[id])).frame; + } + assert.equal(frame.pending.nodeId,'plan'); + const before=canonicalBytes(f.store.load(frame.runId)); + const history=f.store.listJournal(frame.runId); + const drifted=createTrustedInteractiveHost({ + store:f.store,runtime:{skillResolver:async({nodeId})=>'Perform declared step '+nodeId, + validateResult:validator}, + policy:{deploymentId:'complex-c01-c12', + bindingVersions:{resolver:'v2',validator:'v1'}}, + trustedPublicKeys:{'trusted-host':f.keys.publicKey}, + trustedArtifacts:[f.compiled.programIdentity.artifactHash], + allowedInputKeys:['request','requirementId','designId','planId'], + disclosurePolicyId:'complex-test-public', + authorizeDisclosure:async()=>true,now:()=>NOW, + }); + for(const action of [ + ()=>drifted.load({runId:frame.runId}), + ()=>drifted.resume({runId:frame.runId}), + ()=>drifted.getPublicAgentTask({runId:frame.runId}), + ]){ + await assert.rejects(action(),{code:'FK_I_HOST_DENIED'}); + assert.deepEqual(canonicalBytes(f.store.load(frame.runId)),before); + assert.deepEqual(f.store.listJournal(frame.runId),history); + } + frame=(await respond(f,frame,answers.plan)).frame; + frame=(await respond(f,frame,answers.review)).frame; + assert.equal(frame.status,'succeeded'); + evidence('C17',f,frame,{policyChangedResolver:'v1 -> v2', + driftRejectedWithoutCAS:true}); +}); + +test('C18 two interleaved Runs isolate branching, start identity and Journal',async t=>{ + const f=await fixture(t); + let a=await start(f,{runId:'C18-run-A'}); + const b=await start(f,{runId:'C18-run-B',state:{ + ready:false,request:'software task'}}); + assert.equal(a.status,'waiting'); + assert.equal(b.status,'failed'); + assert.equal(f.store.load(b.runId).receipts.length,0); + const replay=await f.host.start({...f.compiled,runId:'C18-shadow', + startKey:'C18-run-A',state:{ready:true,request:'software task'}}); + assert.equal(replay.runId,a.runId); + assert.equal(replay.pending.requestId,a.pending.requestId); + await assert.rejects(f.host.start({...f.compiled,runId:'C18-conflict', + startKey:'C18-run-A',state:{ready:false,request:'software task'}}), + {code:'FK_I_START_CONFLICT'}); + a=(await fourSteps(f,a)).frame; + assert.equal(a.status,'succeeded'); + assert.equal(f.store.load(b.runId).status,'failed'); + assert.equal(f.store.load(b.runId).receipts.length,0); + audit(f,a.runId);audit(f,b.runId); + evidence('C18',f,a,{parallelRuns:2, + duplicateStartIdempotent:true,conflictingStartRejected:true}); +}); + +test('C19 36-node chain with invalid output at step 18 has no ghost dispatch',async t=>{ + const count=36; + const f=await fixture(t,{source:longSource(count), + allowedInputKeys:['request',...Array.from({length:count},(_,i)=>'step'+i)]}); + let frame=await start(f,{runId:'C19-fail-midchain'}); + const receipts=[]; + for(let i=0;i<18;i++){ + const p=await task(f,frame); + assert.equal(p.nodeId,'agent'+i); + receipts.push(p.requestId); + const value=i===17?{invalid:'bad'}:{['step'+i]:'value-'+i}; + frame=(await respond(f,frame,value)).frame; + } + assert.equal(frame.status,'errored'); + assert.equal(frame.terminal.causeCode,'FK_R004'); + assert.equal(Object.hasOwn(frame.state,'step17'),false); + assert.equal(Object.hasOwn(frame.state,'step18'),false); + assert.equal(new Set(receipts).size,18); + const previous=canonicalBytes(f.store.load(frame.runId)); + const resumed=await f.host.resume({runId:frame.runId}); + assert.equal(resumed.status,'errored'); + assert.deepEqual(canonicalBytes(f.store.load(frame.runId)),previous); + evidence('C19',f,frame,{declaredAgents:36,dispatched:18, + remainingNotDispatched:18,causeCode:'FK_R004'}); +}); + +test('C20 conflicting independently signed late-Agent answers accept only one CAS',async t=>{ + const f=await fixture(t); + let frame=await start(f,{runId:'C20-equivocation'}); + frame=(await respond(f,frame,answers.requirements)).frame; + const p=await task(f,frame); + assert.equal(p.nodeId,'design'); + const left=receiptFor(p,{designId:'DES-A'}); + const right=receiptFor(p,{designId:'DES-B'}); + const [one,two]=await Promise.allSettled([ + f.host.accept({receipt:left,attestation:attest(f,left)}), + f.makeHost().accept({receipt:right,attestation:attest(f,right)}), + ]); + const settled=[one,two]; + assert.equal(settled.filter(x=>x.status==='fulfilled').length,1); + assert.equal(settled.filter(x=>x.status==='rejected').length,1); + assert.equal(settled.find(x=>x.status==='rejected').reason.code, + 'FK_I_RECEIPT_CONFLICT'); + const winning=settled.find(x=>x.status==='fulfilled').value; + frame=await settle(f.host,winning); + assert.equal(frame.pending.nodeId,'plan'); + assert.ok(['DES-A','DES-B'].includes(f.store.load(frame.runId).state.designId)); + const journal=f.store.listJournal(frame.runId); + assert.equal(journal.length,frame.revision+1); + frame=(await respond(f,frame,answers.plan)).frame; + frame=(await respond(f,frame,answers.review)).frame; + assert.equal(frame.status,'succeeded'); + evidence('C20',f,frame,{twoValidCompetingHostReceipts:true, + exactlyOneCAS:true,winningDesign:frame.state.designId}); +}); diff --git a/tests/interactive/s13-chatgpt-tool-bridge.test.mjs b/tests/interactive/s13-chatgpt-tool-bridge.test.mjs new file mode 100644 index 0000000..912fd24 --- /dev/null +++ b/tests/interactive/s13-chatgpt-tool-bridge.test.mjs @@ -0,0 +1,229 @@ +/** + * ChatGPT-authored response -> exact FlowKit Pending -> trusted Host acceptance. + * + * Explicit preparation mode (FLOWKIT_S13_PREPARE=1) publishes the actual + * Pending and prints its exact requestId/taskDigest for an external ChatGPT + * turn. ChatGPT returns its authored response through an authenticated GitHub + * tool action. That is NOT a FlowKit native JS-tool invocation or S13 Host bridge. + * + * Normal CI mode requires the complete response fixture. It deterministically + * recreates the same Pending, checks runId/requestId/taskDigest, then performs + * Host signing/CAS and SQLite close/reopen within Phase 2. The two phases use + * separate Actions invocations and do NOT share one physical SQLite database. + * + * Evidence is M-level ChatGPT participation only. It does NOT independently + * attest provider/session origin, native Grok/Codex tools or cross-UID isolation. + * Native Host S13 remains BLOCKED. + */ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import {readFile,mkdtemp,rm} from 'node:fs/promises'; +import {fileURLToPath} from 'node:url'; +import path from 'node:path'; +import {tmpdir} from 'node:os'; +import {generateKeyPairSync,randomBytes,sign} from 'node:crypto'; +import {compileInteractive} from '../../src/interactive/compile.mjs'; +import {createSqliteCheckpointStore} from '../../src/interactive/sqlite-store.mjs'; +import {createTrustedInteractiveHost} from '../../src/interactive/trusted-host.mjs'; +import {createAgentReceiptStatement,encodeAgentReceiptStatement} + from '../../src/interactive/host-attestation.mjs'; +import {canonicalBytes,sha256Bytes} from '../../src/interactive/canonical.mjs'; + +const workflowFile=fileURLToPath( + new URL('../../examples/interactive/s13-chatgpt.workflow.js',import.meta.url)); +const skillFile=fileURLToPath( + new URL('../../examples/interactive/skills/reviewer.md',import.meta.url)); +const replyFile=fileURLToPath( + new URL('../fixtures/s13-chatgpt-reply.json',import.meta.url)); +const runId='s13-chatgpt-bridge-run-20261111'; +const startKey='s13-chatgpt-bridge-20261111'; +const namespace='s13-chatgpt-bridge-tenant'; +const privateCanary='HOST_PRIVATE_'+randomBytes(24).toString('hex'); + +const initialState={ + ready:true, + request:'请审查 FlowKit 如何通过可执行 JavaScript Program 约束 Agent 工作流，并给出一项可验证的下一步。', + constraints:[ + 'State 只保留当前工作流所需的轻量状态和少量值。', + '业务文件/产物由用户配置的 Host 存储处理，Context 可以负责复杂关联。', + '流程推进及路由必须由 FlowKit Program 决定，模型不能绕开可信 Host。', + ], + hostOnlySecret:privateCanary, +}; +const policy={ + deploymentId:'s13-chatgpt-tool-bridge', + bindingVersions:{ + 'runtime.skillResolver':'reviewer-skill-test-v1', + 'runtime.validateResult':'reviewer-schema-test-v1', + }, +}; +const resultFields=['flowkitBoundary','nextAction','summary']; + +test('ChatGPT-authored response is bound to exact Pending and accepted by trusted Host',async()=>{ + const dir=await mkdtemp(path.join(tmpdir(),'flowkit-s13-chatgpt-')); + const file=path.join(dir,'run.sqlite'); + let store; + try{ + const compiled=await compileInteractive(workflowFile,{ + output:'memory', + featureStage:'P0-A', + skillsRoot:'examples/interactive/skills', + skillCheck:true, + }); + assert.equal(compiled.diagnostics.some(x=>x.severity==='error'),false, + JSON.stringify(compiled.diagnostics)); + assert.ok(compiled.programIdentity?.artifactHash); + const skill=await readFile(skillFile,'utf8'); + const skillHash=sha256Bytes(Buffer.from(skill,'utf8')); + const keys=generateKeyPairSync('ed25519'); + const runtime={ + skillResolver:async({agent,skill:declared})=>{ + assert.equal(agent,'reviewer'); + assert.equal(declared,null); + return skill; + }, + validateResult:(_schema,output)=>{ + if(!output||typeof output!=='object'||Array.isArray(output)) { + return ['result must be object']; + } + const keys=Object.keys(output).sort(); + if(keys.join(',')!==resultFields.join(','))return ['result field set does not match']; + return resultFields.every(k=>typeof output[k]==='string' + && output[k].length>=15 && output[k].length<=1000)?[]:['result field lengths invalid']; + }, + }; + const hostFor=s=>createTrustedInteractiveHost({ + store:s, + runtime,policy, + trustedPublicKeys:{'s13-test-transport':keys.publicKey}, + trustedArtifacts:[compiled.programIdentity.artifactHash], + allowedInputKeys:['request','constraints'], + disclosurePolicyId:'s13-public-only-v1', + authorizeDisclosure:async({task})=> + task?.kind==='agent' && task?.nodeId==='review' + && task?.runId===runId, + }); + store=createSqliteCheckpointStore(file,{namespace}); + const firstHost=hostFor(store); + const frame=await firstHost.start({ + programBytes:Buffer.from(compiled.code,'utf8'), + programIdentity:compiled.programIdentity, + runId,startKey,state:initialState, + }); + assert.equal(frame.status,'waiting'); + const pending=await firstHost.getPublicAgentTask({runId}); + assert.equal(pending.kind,'agent'); + assert.equal(pending.nodeId,'review'); + assert.equal(pending.runId,runId); + assert.deepEqual(pending.input,{ + request:initialState.request,constraints:initialState.constraints, + }); + assert.equal(JSON.stringify(pending).includes(privateCanary),false); + assert.equal(JSON.stringify(pending).includes('hostOnlySecret'),false); + const taskDigest=sha256Bytes(canonicalBytes(pending)); + const firstJournal=store.listJournal(runId); + assert.ok(firstJournal.length>=3); + assert.equal(store.load(runId).pending.requestId,pending.requestId); + + // Actual public Agent task, published only after durable SQLite Pending. + // No Host secrets, Store path, full State or private signing key are printed. + console.log('S13_CHATGPT_PUBLIC_TASK '+JSON.stringify({ + taskDigest,task:pending, + skillSha256:skillHash, + artifactHash:compiled.programIdentity.artifactHash, + })); + let modelReply; + try{ + modelReply=JSON.parse(await readFile(replyFile,'utf8')); + }catch(error){ + if(error?.code!=='ENOENT')throw error; + } + if(!modelReply){ + assert.equal(process.env.FLOWKIT_S13_PREPARE,'1', + 'Complete ChatGPT response fixture is required for normal CI; set FLOWKIT_S13_PREPARE=1 only for explicit Pending preparation'); + assert.equal((await firstHost.load({runId})).status,'waiting'); + console.log('CHATGPT_PENDING_PREPARED '+JSON.stringify({ + runId,requestId:pending.requestId,revision:frame.revision, + journalRevisions:firstJournal.length, + result:'Real public task printed for next ChatGPT tool turn', + })); + return; + } + assert.equal(modelReply.runId,runId); + assert.equal(modelReply.requestId,pending.requestId); + assert.equal(modelReply.taskDigest,taskDigest); + assert.ok(modelReply.result); + assert.equal(runtime.validateResult(null,modelReply.result).length,0); + + // Host lifecycle: close actual SQLite Store after durable Pending, then + // reopen and continue without recompiling at the resume boundary. + store.close(); + store=createSqliteCheckpointStore(file,{namespace}); + const resumedHost=hostFor(store); + const recovered=await resumedHost.load({runId}); + assert.equal(recovered.status,'waiting'); + const afterRestartTask=await resumedHost.getPublicAgentTask({runId}); + assert.equal(afterRestartTask.requestId,pending.requestId); + assert.equal(sha256Bytes(canonicalBytes(afterRestartTask)),taskDigest); + const receipt={ + runId, + requestId:pending.requestId, + nodeId:pending.nodeId, + attempt:pending.attempt, + kind:'agent-result', + result:modelReply.result, + }; + // Local Host signer is separate from the public model task, but this + // fixture has NO independent native ChatGPT session attestation. + const now=Date.now(); + const statement=createAgentReceiptStatement({ + run:store.load(runId),receipt,keyId:'s13-test-transport', + proofId:'chatgpt-github-bridge-result-001', + issuedAt:now-1000,expiresAt:now+60000, + }); + const attestation={ + statement, + signature:sign(null,encodeAgentReceiptStatement(statement),keys.privateKey) + .toString('base64url'), + }; + const before=store.load(runId); + assert.equal(before.status,'waiting'); + // Signed, but bogus signature must not mutate Run. + await assert.rejects(resumedHost.accept({ + receipt,attestation:{statement,signature:'forged'}, + }),{code:'FK_I_PROVENANCE_REJECTED'}); + assert.equal(store.load(runId).revision,before.revision); + const completed=await resumedHost.accept({receipt,attestation}); + assert.equal(completed.status,'succeeded'); + assert.equal(completed.terminal.nodeId,'done'); + assert.equal(completed.state.hostOnlySecret,privateCanary); + assert.equal(completed.state.summary,modelReply.result.summary); + assert.equal(completed.state.nextAction,modelReply.result.nextAction); + assert.equal(completed.state.flowkitBoundary,modelReply.result.flowkitBoundary); + const receiptIndex=store.getReceipt(runId,receipt.requestId); + assert.equal(receiptIndex.acceptedRevision,completed.acceptedRevision); + assert.equal(store.listJournal(runId).length,completed.revision+1); + const duplicate=await resumedHost.accept({receipt,attestation}); + assert.equal(duplicate.duplicate,true); + assert.equal(duplicate.acceptedRevision,completed.acceptedRevision); + assert.equal(duplicate.revision,completed.revision); + assert.equal((await resumedHost.resume({runId})).status,'succeeded'); + console.log('CHATGPT_RESPONSE_FIXTURE_ACCEPTED '+JSON.stringify({ + runId,requestId:receipt.requestId, + modelReplyDigest:sha256Bytes(canonicalBytes(modelReply.result)), + artifactHash:compiled.programIdentity.artifactHash, + taskDigest,skillHash, + status:completed.status, + terminal:completed.terminal.nodeId, + acceptedRevision:completed.acceptedRevision, + journalRevisions:completed.revision+1, + duplicateAck:true,restoredFromSQLite:true, + phase2DatabaseReconstructed:true,crossRunDatabaseContinuation:false, + policyPinned:true, + secretNotExposedToPublicTask:true, + warning:'ChatGPT-authored response delivered through GitHub tool; not a FlowKit native JS-tool call; no independent native session attestation; S13 remains BLOCKED', + })); + }finally{ + try{store?.close()}finally{await rm(dir,{recursive:true,force:true})} + } +});