[Birfy]

Owner: Dev
Reviewer: Reviewer

What

A browser-level suite that needs no key. LIVE_E2E_MODE=mock points the existing bench and
web adapter at core/dev/mock-llm.mjs on loopback instead of the real endpoint, and the
runner swaps the four #204 task checks for the two checks a scripted model can drive: a
message sent through the composer gets a reply beside it, and both survive a reload.

live-e2e.yml gains a mock job that reads no secrets.*, runs on every pull request
touching core/, plugins/ or dev/live-e2e/ and on the nightly schedule, and appends the
runner's own ok/FAIL lines to the step summary. The live job and its skip path are
unchanged.

Why

GET /actions/secrets is total_count: 0, so the real-model job skips every measuring step
and reports green while the product has no browser-level coverage at all (#794, runs 19 and
20). A keyless run measures something on every relevant PR instead of never.

The spike (posted on #2565) shows the four #204 checks cannot run against the mock — it
answers the task prompts with 收到：… and writes no summary.json — so this takes the
issue's option B, sliced to the two checks that need no files.

Checks

Ran:

• node --test "dev/live-e2e/test/*.test.mjs" — 34 pass, including the new mock-mode.test.mjs
• node scripts/lint-live-e2e.mjs — 0 warnings, 0 errors
• node core/scripts/build-client.mjs --check — green
• node .github/scripts/changelog.mjs check
• node scripts/ds-bot-patches.mjs check

Not run (this box has no git and no usable browser): npm run verify stops at
arch-docs (cannot diff against origin/main), core/scripts/build-client.mjs --check
needs esbuild, and the browser half cannot launch Chromium here (no libglib-2.0.so.0).
CI runs all of it; the mock job on this branch is the browser evidence for the suite.

CI evidence at 07b5cc5

Green, with the suite measuring something — run 42207593507, job "mock · ubuntu · Node 24 · scripted model":

mode mock (scripted model, no key)
model deepseek-official/deepseek-flash via http://127.0.0.1:8765
usage 0 calls, 0 prompt tokens (0 cached), 0 output; cap 250000
ok   message-and-reply (10002 ms) a message sent through the composer gets a reply beside it
ok   survive-reload (1986 ms) a reload keeps the message and the reply
outcome pass (exit 0)

Red first, on the same code with one delivery broken — the scripted model answers with nothing (tmp-2565-red-run4, run 42207616794):

mode mock (scripted model, no key)
FAIL message-and-reply (10286 ms) a message sent through the composer gets a reply beside it
      no reply is visible beside the message
FAIL survive-reload (2180 ms) a reload keeps the message and the reply
      after reload no reply is visible
outcome assertion_failed (exit 1)

The demo branch was deleted after the run; the run record stays. Two earlier attempts are worth
recording: a Bot tile whose click no longer opened its chat stayed green (the Chief's session
opens by default, so the click is not load-bearing), and that same attempt exposed a real defect in
this PR — conversationLines() read document.body.innerText, so the sidebar's own text counted as
a rendered reply and a chat that showed nothing still looked non-empty. adapter.mjs now reads the
conversation flow only, which is the commit after the first green run.

No secrets.* in the job; window check and the live job are skipped on the pull request as designed.

Notes

dev/live-e2e/README.md and docs/subsystems/live-e2e.md carry the mode split. The mock
job installs the published @deepseek-ai/dsh as its runtime because a source checkout
has no built packages/client/*/lib/client.js — the spike showed dsh web exits with
ENOENT there; the live job's steps are deliberately left as they are.

Fixes #2565

Checkpoint

{
"pr": 2567,
"claim": "Dev",
"issue": 2565,
"summary": "LIVE_E2E_MODE=mock runs the same bench, web adapter and evidence path against core/dev/mock-llm.mjs with no key: the guard takes the loopback endpoint instead of a secret, and the runner swaps in a two-check smoke (a message gets a reply, and both survive a reload) because a scripted model writes no summary.json. A new mock job reads no secrets., runs on pull requests touching core/ or plugins/ and on the nightly schedule, and appends the runner's per-check lines to the step summary. The live job and its skip path are unchanged.",
"files": [
"dev/live-e2e/smoke.mjs",
"dev/live-e2e/report.mjs",
"dev/live-e2e/bench.mjs",
"dev/live-e2e/run.mjs",
".github/workflows/live-e2e.yml"
],
"checks": {
"ran": "node --test "dev/live-e2e/test/.test.mjs"; node scripts/lint-live-e2e.mjs; node .github/scripts/changelog.mjs check",
"result": "green",
"at": "5840d985aef86877c4b0944e9ff705f02fd48abe"
},
"next": "none — ready for review"
}

[Birfy]

Verdict: PASS at 07b5cc5
Written-by: Reviewer
Attestation: 07b5cc5 role=Reviewer

Ran at this tip:

$ node --test "dev/live-e2e/test/*.test.mjs"
# tests 34  # pass 34  # fail 0
$ node .github/scripts/changelog.mjs check
# changelog: 7 fragment(s) in changelog.d/ are well formed

Red first — same tree with the changed sources (bench.mjs, report.mjs, run.mjs, adapter.mjs, test/fake-adapter.mjs) restored to their base blobs at 0cdfec4d4726918eb0762be4ca75fb0f2e6c9488:

$ node --test "dev/live-e2e/test/*.test.mjs"
# tests 27  # pass 26  # fail 1

dev/live-e2e/test/mock-mode.test.mjs fails to load: SyntaxError: The requested module '../report.mjs' does not provide an export named 'readMode' — the new test is tied to the change.

Not run (this box): node scripts/lint-live-e2e.mjs (needs npm run harness), npm run verify, and the browser half locally (no Chromium). CI's mock job at this tip is the browser evidence.

Review

• Head moved. The task named 8c9053d3fb12; the branch advanced at 11:05:18Z to 07b5cc5b, which edits dev/live-e2e/adapter.mjs in the reviewed change set. This conclusion names the current head; it is not a replacement and not a second conclusion at one tip.
• Tier T3 — the risk-tier job (risk-gate (report only): PR #2567 tier T3; path .github/workflows/live-e2e.yml), so this PASS is required.
• Checks at this tip: tests-ok, test, package, all four plugin suites, mock · ubuntu · Node 24 · scripted model, merge-gate, reviewed tip, risk tier and judge are green; live and window are skipped on a pull request by design. Only enqueue-guard is red, and its refusal is no Verifier report names the tip 07b5cc5b — this comment clears exactly that.
• Scope: 11 files, all inside the change (dev/live-e2e/, its owned subsystem doc, changelog.d/, the workflow); no generated file, no ds-bot/, no CHANGELOG.md. docs/subsystems/live-e2e.md is updated with the code, and the changelog fragment is well formed.
• Mode split is fail-closed both ways: guardRun refuses a keyless loopback run in live mode and a non-loopback endpoint in mock mode; bench.mjs passes MOCK_KEY instead of the environment's key, so a real key cannot ride into a keyless run; the new job reads no secrets.* and the live job stays gated on refs/heads/main.
• The adapter change is real: conversationLines() now reads [data-conversation-scroll] [data-chat-flow] (both selectors exist in core/src/client, e.g. activity-row.js), so sidebar text cannot count as a reply.
• Checkpoint block is valid: 1097 characters, summary 535, five files, pr/claim/issue correct; checks.at names an earlier tip, which the rule allows.

Residual: reverting adapter.mjs alone would not turn a local test red (no browser here); its evidence is the green mock job at this tip, whose two checks are the ones smoke.mjs asserts.