Skip to content

e2e: characterization suite for the 1.7.4-1.7.7 release notes (packaged app, no provider key) - #146

Open
kai-openswarm wants to merge 1 commit into
openswarm-ai:devfrom
kai-openswarm:b9/characterization-e2e
Open

e2e: characterization suite for the 1.7.4-1.7.7 release notes (packaged app, no provider key)#146
kai-openswarm wants to merge 1 commit into
openswarm-ai:devfrom
kai-openswarm:b9/characterization-e2e

Conversation

@kai-openswarm

Copy link
Copy Markdown

What

A regression net for the user-facing claims in the release notes (backend/apps/help/changelog.py, 1.7.4–1.7.7): e2e/tests/v1.7-characterization/ — 4 specs, 19 tests, plus a shared support.ts and a README section. Each test names the changelog line it pins, so a later refactor of the files that carry that behaviour (the chat surface, the canvas and window state, the shell's dashboard host, the layout slice) has to keep the claim true, not just compile.

Same rules as the rest of e2e/: packaged app, no provider key, macOS + Windows. Sessions come from the app's own launch route; the messages an agent would stream (AskUI approval cards, ShowUI links) are dispatched through the store's own reducer in the wire shape the backend emits; the backend is called from inside the renderer (already bearer-patched), so no spec handles a token or a port. Named v1.7-… so it sorts after the existing specs and never changes their starting profile.

  • canvas-fullscreen (6): composer yields to open windows; a wheel inside a window never moves the canvas; the default wash is flat colour; nothing drags while a window is fullscreen and Escape exits; exactly one fullscreen owner, ever; dock tiles are images or glyphs, never letters.
  • chat-toolui (6): AskUI approval card answers once with a receipt; an answer nothing is waiting for is reported honestly; older asks go quiet behind the live one; a widget link opens a browser card, leaving fullscreen first; the provider-retry pill survives status frames and never shows a scary card.
  • resilience (3): a failed or unscoped sessions read never wipes the board; the packaged backend is really killed — a quick recovery is silent, a sustained outage shows "Reconnecting to OpenSwarm…" and heals when it can (each outage test boots its own app so the respawn budget starts clean).
  • workflows-and-settings (4): off means off (Run Now on a switched-off workflow is refused and the refusal is in History; a deleted one cannot run); Settings › Memory shows the store's own list; the dictation cue defaults to 70%.

Verified

  • 24/24 (these 19 + smoke.spec.ts) in one run against a packaged 1.7.7 build on macOS, pristine profile; each file also run repeatedly standalone.
  • On GitHub-hosted windows-latest the packaged app currently cold-boots past the harness's own budgets (smoke.spec.ts fails there in beforeAll), so the Windows leg was not the proof here; the specs use no OS-specific paths beyond a PowerShell branch for finding the backend PID.
  • File/folder note: e2e/tests was already over the linter's 7-item cap on dev; this adds one folder.

The suite surfaced a few things on the way, sent separately: #142 (a fresh read after a save could be answered from the 1 s GET cache — the memory test carries a 1.5 s wait until that lands), the launch-snapshot fix, the mac dev-build fix, and the installer verifier port.

…ed app, no provider key)

A regression net for the user-facing claims in backend/apps/help/changelog.py:
each test names the changelog line it pins, so a later refactor of the files
that carry that behaviour has to keep the claim true, not just compile.

- canvas-fullscreen: composer yields to open windows; a wheel inside a window
  never moves the canvas; flat default wash; nothing drags while fullscreen and
  Escape exits; exactly one fullscreen owner, ever; dock tiles are never letters
- chat-toolui: AskUI approval cards answer once with a receipt, an unparked
  answer is reported honestly, older asks go quiet behind the live one, widget
  links open browser cards leaving fullscreen first, the provider-retry pill
  survives status frames and never shows a scary card
- resilience: a failed or unscoped sessions read never wipes the board; the
  packaged backend is really killed - a quick recovery is silent, a sustained
  outage shows the reconnecting pill and heals when it can
- workflows-and-settings: off means off (Run Now refused + History row; a
  deleted workflow cannot run), Settings > Memory is the store's own list, the
  dictation cue defaults to 70%

Sessions come from the app's own launch route; streamed messages are dispatched
through the store's reducer in the wire shape the backend emits; the backend is
called from inside the renderer so no spec handles a token or a port. Sorted
after the existing specs so it never changes their starting profile.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant