Skip to content

Capture: ingest mode + userscript for playing web clients in your own browser - #289

Closed
shinkuan wants to merge 7 commits into
v3from
feat/288-ingest-userscript-capture
Closed

shinkuan wants to merge 7 commits into
v3from
feat/288-ingest-userscript-capture

Conversation

@shinkuan

Copy link
Copy Markdown
Owner

Closes #288.

What

A third capture mode, ingest, for playing the web clients (Mahjong Soul, Tenhou) in the user's own browser, with their real profile and login, Firefox included. No system proxy, no CA, and no browser launched by Akagi.

game page ── userscript (wraps window.WebSocket, read-only)
                 │  JSON envelopes over ws://127.0.0.1:23411/ingest
                 ▼
          IngestBackend ──→ FlowBridges ──→ Bridge::parse ──→ MjaiBus

Backend: src/capture/ingest/

  • An axum loopback server with three routes:
    • GET /ingest: the WebSocket a source pushes frames into.
    • GET /akagi.user.js: the userscript, rendered with this address. Userscript managers offer to install from it.
    • GET /ping: answers pong.
  • Frames go through FlowBridges into the platform bridge, exactly like the Chromium backend. The Inspector ws_frame recording is now shared between the two backends (capture/frame.rs).
  • Wire protocol, v1: hello / open / frame / close. Text frames are sent verbatim and binary frames as base64. Both directions are sent, because Mahjong Soul pairs responses with requests.
  • New config section [capture.ingest] addr, default 127.0.0.1:23411, plus a supervisor arm and an ingest_userscript IPC command for the "copy script" button.

Safety

  • Loopback only. The backend refuses to start on any other address.
  • Origin check at the handshake. A game site from GAME_SITES is accepted. Any other page gets 403, so an arbitrary website cannot feed Akagi fake frames. A connection with no Origin (a local program) is accepted. The same list renders the userscript's @match lines, so the two cannot drift apart.
  • Wrong platform or protocol version. The page is closed with 1008 and a toast. The script then stays quiet until the page reloads.

Userscript (akagi.user.js)

  • Strictly read-only. Every frame reaches the game unchanged. Forwarding is fire-and-forget, and nothing it does can throw into the page.
  • Hooking. It uses a Proxy over the native constructor and keeps instanceof, subclassing and ws.constructor intact. It uses the native addEventListener and url getter, so a page subclass that overrides them is still captured.
  • Ordering. Messages from one socket keep their order, Blob reads included. An up frame is forwarded only after the native send succeeded, from a copy of the buffer.
  • While Akagi is unreachable:
    • It holds only the last minute of frames, capped by count and by size.
    • It drops held frames from sockets that have since closed, so a finished game is never replayed as live.
    • It goes dormant once the reconnect backoff reaches its ceiling, and resumes when a connection succeeds.

Frontend

  • A third mode card in Setup and a Settings option, with an install / copy panel. The mode label is now three-way in Setup, Overview and the capture tile.
  • Bug fix: Settings rebuilt capture as {mode, chromium}, which dropped capture.http; the backend then reset it to its defaults on the next save. Capture edits now go through lib/captureConfig.ts, which spreads the existing section. Regression test: captureConfig.test.ts.

Out of scope

Autoplay in this mode (there is no page handle to drive), a packaged browser extension, and changes to the Chromium backend.

Tests

  • tests/capture_ingest.rs. It starts the real server and checks, over real sockets:
    • /ping and the rendered userscript are served;
    • a foreign origin gets 403;
    • a wrong-platform page is closed with 1008;
    • Tenhou frames pushed by a game page produce start_game / start_kyoku on the bus and ws_frame rows on the Inspector timeline;
    • shutdown closes the source and ends the backend cleanly.
  • Unit tests for origin matching (lookalike hosts, http, null), envelope parsing, payload decoding, loopback validation, userscript rendering, and config serde and back-compat.
  • frontend/src/lib/userscript.test.ts runs the committed script against a fake page (15 cases). It covers forwarding and ordering, page shape, the subclass override, failure isolation, buffer snapshots, a refused send, the hold caps and window, forgetting closed sockets, policy refusal, dormancy, reconnect, and single install.
  • cargo fmt --check, cargo clippy --all-targets -D warnings, cargo test, tsc -b and eslint all pass.

Manual check still needed

Install the userscript in Chrome and Firefox and play a Tenhou game and a Mahjong Soul game. Two things to watch for:

  • whether Chrome shows a local-network-access prompt for the page's connection to 127.0.0.1;
  • whether either site's CSP blocks that connection.

…ur own browser

Add a third capture backend, CaptureMode::Ingest. Akagi listens on a
loopback WebSocket (default 127.0.0.1:23411), and a userscript running on
the game page in the user's own browser pushes the page's WebSocket frames
to it. The frames go into the existing protocol bridges exactly as the
Chromium backend routes them, so no system proxy, CA or Akagi-launched
browser is needed.

- src/capture/ingest/: an axum server with three routes. /ingest is the
  source WebSocket, /akagi.user.js serves the userscript rendered for this
  address, and /ping answers pong.
- Who may connect: the handshake Origin must be an allowlisted game site,
  and any other page gets 403. The listen address must be loopback. A page
  for a different platform than the configured one is closed with a toast.
- The userscript wraps window.WebSocket at document-start and only reads.
  Frames reach the game unchanged. Blob reads are ordered per socket.
  Frames are held, with a cap, until Akagi is reachable, and the script
  reconnects with backoff.
- Share the Inspector ws_frame recording between the Chromium and ingest
  backends (capture/frame.rs).
- Frontend: a third mode in Setup and Settings, with an install / copy
  panel for the userscript and a three-way capture label. Settings no
  longer drops capture.http when the mode or Chromium fields change
  (regression test in captureConfig.test.ts).
- Docs: README (en/zh-TW/zh-CN) quick start and troubleshooting, plus the
  capture and config module READMEs.

Closes #288
… refused

Review follow-ups on the ingest userscript:

- While Akagi is unreachable, hold only the last minute of frames, capped
  by count and by size. Forget the held frames of a socket that closed
  before Akagi connected, so a game that ended while Akagi was down is
  never replayed into it as live.
- Stop encoding and holding once the reconnect backoff reaches its
  ceiling, and resume when a connection succeeds.
- A policy close (1008: wrong platform, outdated script) stops the
  script until the page reloads. It used to reconnect, and re-toast, every
  second.
- Forward an up frame only after the native send succeeded, and copy
  mutable buffers before encoding them later.
- Use the native addEventListener and url getter, so a page subclass that
  overrides them, and runs inside super(), is still captured. Keep
  ws.constructor === WebSocket.
- Server: a dropped stop sender ends the connection instead of spinning.
- Docs: describe the hold rules, and which sources pass the Origin check.
The ingest endpoint only starts once the configuration is saved, and the
Setup wizard saves nothing until Finish. Its Install button could
therefore never be pressed. In Setup, the panel now says where Install
will be after setup and keeps Copy script. Settings is unchanged: there
the button enables as soon as the change is saved.
The previous fix only explained why Install could not work in Setup. That
was more confusing than a working button. The endpoint starts once the
configuration is saved, but the moment someone picks userscript capture
is when they want to install the script.

- capture/ingest/preview.rs: a stand-in server on the same address that
  serves /akagi.user.js and /ping and nothing else. There is no /ingest,
  so no frame reaches a bridge before capture starts. At most one
  preview runs; IngestBackend::run stops it before binding, so the two
  never compete for the port.
- IPC: start_userscript_preview / stop_userscript_preview. The stop is
  scoped to an address, so a late stop cannot take down the preview the
  UI moved to.
- IngestUserscriptPanel keeps a preview running while ingest is picked
  but not live (debounced against address edits) and stops it on unmount.
  Install now works in Setup step 4 and on an unsaved Settings change. A
  port that is already taken is reported under the buttons.
- Remove the setup-only install text; README steps and the capture
  README updated.
- Tests: a preview unit test (serves only the script, 404 on /ingest,
  scoped stop, address switch frees the old port, taken port reported),
  and the integration test now starts a preview first and checks that the
  backend takes the port over.
Opening /akagi.user.js directly only installs anything when the browser
has a userscript manager to intercept it. Without one, the browser just
shows a page of JavaScript with no explanation. The same happens when
the manager lives in a browser that is not the default one.

- capture/ingest/install_page.rs: a static, script-free page served at /
  by the backend and by the install preview. It links /akagi.user.js and
  explains what to do when that shows code: where to get Tampermonkey or
  Violentmonkey, and Chrome/Edge's "Allow user scripts". It comes in
  en/ja/zh-TW/zh-CN via ?lang=. The query only selects built-in strings
  and is never echoed into the page.
- The Install button now opens /?lang=<UI language>; the panel shows that
  URL for use in a non-default browser.
- Tests: install page unit tests (every language, fallback, no echo), and
  the preview and integration tests fetch the page.
- README steps and the capture README route table updated.
Drop the filler "small userscript" phrasing. The install page and the
README now state what happens directly: a userscript reads the
WebSocket of the supported games in this browser, read-only.
Chrome asks before a public site may connect to this computer's own
addresses. If the user blocks that prompt, the userscript's connection
to Akagi fails exactly like Akagi not running: nothing reaches Akagi,
and nothing says why. This happened in manual testing.

- Userscript: console status lines prefixed [Akagi]. They report the
  script starting, each game socket watched, connecting, losing the
  connection, pausing, and a refusal with its reason. When Akagi cannot
  be reached, it warns once (not per retry) and asks the Permissions API
  about the local-network permission under each name Chrome has used.
  If that permission is denied, it says so and explains how to re-allow
  it.
- Akagi's panel gains a step about allowing local network access (all
  four locales). The install page gains the same section and points to
  the [Akagi] console lines when no notification appears.
- README (en/zh-TW/zh-CN): step 5 mentions the prompt; troubleshooting
  now starts from the console lines.
- Tests: console status, a single unreachable warning, denied vs granted
  permission, and the refusal message.
@shinkuan shinkuan closed this Oct 16, 2026
@shinkuan
shinkuan deleted the feat/288-ingest-userscript-capture branch October 16, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Capture: ingest mode + userscript for playing web clients in your own browser

1 participant