Zentty Mobile Companion (iOS + Android) - #55
Open
dedene wants to merge 22 commits into
Open
Conversation
…mance companion/wire: zod v4 schemas for all 8 message families (32 message types) with JSON test vectors as the cross-language contract. Zentty/Companion/Wire: Swift Codable mirror; ZenttyLogicTests round-trips every vector. Protocol v1 with version negotiation and forward-compat unknown-field tolerance.
…d dashboard feed Security: Ed25519 device identity in Keychain (graceful in-memory fallback), X25519+HKDF handshake with transcript signing, directional ChaCha20-Poly1305 sessions with replay protection, pairing store with one-time QR offers. Bridge: NWListener WebSocket + Bonjour (_zentty._tcp), inert until a device is paired and the toggle is on; dashboard snapshot/delta feed off the agent status pipeline; input routing through existing pane resolution. Settings: Mobile Devices section with QR pairing sheet, paired-device list with revoke, listener status.
…port companion/relay: Node WS relay — Ed25519 challenge auth, from-stamped frame routing, peer presence, token-bucket rate limits (tighter bucket for plaintext pairing frames), /healthz, multi-stage Dockerfile for self-hosting. Mac side: outbound relay transport behind the same transport seam as the LAN listener (one session per peer, demuxed by sender), jittered reconnect backoff, companion.relayUrl config; pairing offers embed the relay URL. Wire: relay.* frame schemas + vectors on both sides; conformance suite gained a relay lane, format validation (base64url, error codes), and a cross-language Ed25519 interop check. Fixes a live interop bug where Swift signed decoded nonce bytes while the relay verifies the transmitted base64url string.
…shboard, and crypto interop companion/mobile: Expo app 'Zentty' (be.zenjoy.zentty.mobile, desktop icon flattened for mobile). Protocol core mirrors the Mac crypto byte-for-byte (X25519+HKDF, ChaCha20-Poly1305 sessions, replay window, pairing proof) behind a SodiumLike seam; connection manager does direct-LAN-then-relay with backoff. Screens: QR/manual pairing, Macs list with transport badges, live dashboard with attention-pinned rows and stale-state banner. Cross-language guarantee: fixed-key crypto vectors checked into companion/wire/vectors/crypto, asserted by both jest and a new ZenttyLogicTests suite (byte-identical directional keys and sealed frames; Ed25519 cross-verification).
Surfaces GHOSTTY_ACTION_RENDER as a coalesced .contentChanged event behind a refcounted gate so the render hot path stays a single atomic read when no phone is watching. CompanionPaneTextFeed debounces 150ms, dedupes unchanged text, stamps monotonic seq + grid size, and serves one-shot scrollback. Input key map completed (CSI arrows, control bytes) through the existing sendText path.
…laceholder CompanionLeaseManager grants the phone's measured grid (sanity-clamped), resizes the surface through the suspended-viewport-sync seam, occludes rendering, and installs a placeholder overlay with an always-visible Take Back Control button. Heartbeat 5s / expiry 15s on an injectable clock; supersede swaps grids without flicker; disconnects leave the expiry clock as the single authority so transport blips never drop a lease. Restore paths covered for expiry, release, take-back, pane close, and bridge stop.
ClaudeTranscriptParser normalizes session JSONL into wire transcript entries (per-block fan-out with stable ids, polymorphic tool_result flattening, meta-line skipping, torn-line buffering). Feed tails the live transcript via ClaudeHookSessionStore lookup with cwd-slug fallback for restored panes, serves bounded snapshots then deltas, and maps rotation to session_ended. Adapter registry makes Codex a file drop later; dashboard hasTranscript now reflects adapter availability.
Node/ws harness impersonating the Mac bridge with the app's own core crypto (role mac): pairing offer minting, handshake, scripted dashboard snapshot and delta. Proven headlessly against the phone core and used for the first on-simulator smoke pass.
…ranscript tab Runtime crypto swapped to @StableLib pure-TS primitives (X25519, ChaCha20-Poly1305-IETF, Ed25519) behind the SodiumLike seam — the native libsodium binding lacked both on device; libsodium-wrappers stays as the test adapter so two implementations cross-check the checked-in interop vectors. Keychain entitlement persisted via app.json. Pane detail: Terminal tab (live text, input bar, key toolbar with CSI arrows, seq-coalesced), quick-actions bar mapped to the Mac router's actionId contract and pinned in both tabs, takeover with measured-grid lease request + heartbeat + revoke-to-mirror, and a Conversation tab (transcript bubbles, collapsible tool cards, inline approvals) shown only when the pane has an adapter. Sticky per-pane tab choice. Verified end-to-end on the iOS simulator: pair, dashboard, terminal, approve, transcript, takeover reflow, live delta, offline.
Gateway endpoints on the relay (/register, /wake), APNs (HTTP/2 ES256) and FCM (HTTP v1 RS256) clients built on Node built-ins and gated on config — no keys means push is disabled and sessions still work. Mac fan-out signs wake pings with the device identity beside the local attention notification; content is E2E-sealed to the paired phone with a key derived from both long-term identities (Ed25519->X25519, HKDF salt zentty-push/v1, ChaCha20-Poly1305) so the gateway stays zero-knowledge. Mobile registers its APNs/FCM token over the session, decrypts in a local Notification Service Extension config plugin, and deep-links to the pane. Seal scheme locked across languages by a checked-in interop vector both Swift and TS verify. Real APNs/FCM delivery pending key provisioning; everything else is unit-tested against throwaway keys with injected transports.
…tness Review follow-ups (crypto/auth confirmed sound; these are lifecycle and resource-exhaustion): - Disabling the feature now severs live LAN sessions and revokes active takeover leases, not just the listener. - Transcript snapshot reads a bounded 256KB tail off the main actor so a large session file can't stall the desktop UI. - Relay: ws maxPayload, unauth idle timeout, global + per-IP connection caps, relay.watch rate-limit + watcher reaping, signature-first push endpoints with bucket/registry caps, send() backpressure drop. - Mobile: handshake/ready timeout ends dead 'connected' sessions, exponential backoff on session-drop, bounded transcript retention, corrupt-pairings blob no longer silently wipes all pairings.
On panes with a Conversation tab, take-control is a power-user escape hatch, not the main path (approve/type already work without it). Collapse its idle state to a quiet inline link there; panes whose only surface is the terminal (no adapter) keep the full card. Active lease states stay full cards everywhere so the Release control is always visible.
# Conflicts: # Zentty.xcodeproj/project.pbxproj
Replace the generic Ionicon glyphs in the pane list and detail views with the desktop app's agent logos (Claude, Codex, Cursor, Gemini, Grok, Mistral, Kimi, Droid, Copilot, Hermes, Amp, OpenCode, Pi, Zentty). The SVGs are generated from ZenttyResources/Assets.xcassets into an embedded module, tinted via currentColor to match the desktop's monochrome treatment, and rendered with react-native-svg. Unknown tools fall back to the terminal glyph.
…ring Checkpoint of in-progress work so it is not lost: - pane.bytes.* wire family with vectors, per-epoch ring buffer with absolute offset resume, chunking and multi-watcher fan-out (CompanionPaneBytesFeed). The producer side is not yet wired - ingestPaneBytes has no caller outside tests until the libghostty PTY tee lands. - XtermTerminalView WebView emulator on the phone, fed by paneBytesLane with epoch tracking and reset-on-truncate. - Companion render keepalive so a mirrored surface keeps repainting under both backgrounding and a control lease. - push-key-mirror native module so the iOS NSE can unseal wake banners. - Special-key input path that bypasses bracketed paste, so cursor-key CSI and a submitting Return survive. - Fix StyleSheet.absoluteFillObject -> absoluteFill (RN 0.86); typecheck green.
Rebuilds against dedene/ghostty zentty/companion-tee, which adds ghostty_surface_set_pty_tee on top of the existing smooth-scroll patch. Verified: _ghostty_surface_set_pty_tee is exported from ghostty-internal.a and the callback typedef shipped in the vendored header. Other worktrees share FrameworksLocal/ and will pick this up on their next build. The new symbol is additive, so existing builds keep working.
…ound Four independent tracks, each verified against the full gate. Desktop control lease (Peter's chosen design: centered live grid, dimmed surround). The surface was already pinned to the phone's grid, but the terminal view kept filling the pane, so the leased content rendered in a corner. The view is now framed at exactly the leased pixel size and centered, with the scrim punching a cutout so only the surround dims, and the card carrying a '45 x 30' grid line. Also fixes a stale CAMetalLayer drawableSize under lease -- once assigned explicitly the layer stops tracking bounds, so the leased grid would have rendered scaled rather than 1:1. The card is positioned clear of the cutout, and an unresolvable leased size now dims the whole pane rather than undimming it. Input audit logging (spec 3, never implemented). Every phone-injected text, key and quick action is recorded with pane id, the authenticated device id, kind and outcome -- structurally never the content, since passwords get typed from phones too. Wire replay bound. attached.replay had no length bound while chunk.data did, so a cold attach to a full 1 MiB ring exceeded the relay's 256 KiB frame cap and the relay CLOSES such connections. Bounded to one chunk's worth: the binding constraint turned out not to be the frame cap but the per-device byte bucket, whose whole one-second capacity is that same 256 KiB -- a frame sized to the cap starves the live chunks that follow and the pane never converges. Companion CI -- the repo had no .github at all. Path-filtered to companion/**, runs typecheck + tests. Fixing it surfaced that a clean 'pnpm install' was already broken: pnpm 11 ignores package.json's pnpm.onlyBuiltDependencies and pnpm-workspace.yaml still held a literal placeholder, so esbuild's build script was refused -- which also broke relay/Dockerfile.
…y tee The byte lane finally has a producer. Chain: CompanionPaneBytesFeed (on the 0->1 attached-watcher edge) -> AppDelegate -> MainWindowController -> TerminalPaneHostView -> LibghosttyAdapter -> LibghosttySurface -> ghostty_surface_set_pty_tee. Threading is the whole problem: the tee fires on libghostty's io-reader thread under the renderer mutex, where time spent back-pressures the child process. LibghosttyPTYTeeAccumulator does nothing there but take a lock, memcpy, and set a flag; drains are coalesced to one main-thread hop per window, so a pane running `yes` does not flood the main actor. Seq is now authoritative from the tee rather than derived from the ring. That converts silent corruption into a detectable gap: any drop shows up as a forward jump in the absolute offset, which resyncs the phone through the truncated warm-attach path it already implements. Fixes found in review of the first cut: - Accumulator overflow sheds the whole staged window instead of trimming the excess. Trimming meant a full 512 KiB memmove per callback for as long as the buffer stayed saturated -- on the io-reader thread, under the renderer mutex, back-pressuring the child and lengthening the very main-thread stall that caused the overflow. A self-reinforcing loop. - CompanionPaneBytesRing is a real circular buffer now. Its removeFirst was harmless while the ring had no producer; on the live path it was a ~1 MiB memmove per 32 KiB chunk on the main actor. - A failed install is no longer recorded as success. It left the phone with a valid attach reply and then silence forever, since its resync is only ever driven by an arriving chunk. - A pane closed with Cmd-W never reaches handlePaneClosed (only shell exit does), stranding a 1 MiB ring per closed watched pane and still answering later attaches with stale bytes. The ring is dropped when the provider reports the pane no longer resolves.
Completes the realtime path. A phone attaching to an already-running TUI used to get only the ring's byte tail, which starts mid-escape-sequence and misses every mode set before the retained window -- so the screen rendered as garbage. It now gets a replayable screen capture first, then the live tail. Mac: ghostty_surface_snapshot is called on cold attach and on any truncated warm attach (the cases where the phone must repaint), throttled per pane since the capture walks every active cell under libghostty's renderer mutex. The reply carries the capture with replay: "" and startSeq set to the capture offset, so the snapshot is excluded from seq arithmetic and the ring tail never double-applies bytes already baked in. Phone: the lane resets, sizes the emulator to the mac's grid, writes the snapshot, then writes the replay. Geometry is mac-authoritative on this lane -- the phone letterboxes to the captured grid while mirroring read-only and only fits to its own size under a control lease, because the snapshot and the live tail are both in mac-grid coordinates and any phone-side rewrap diverges. Fixes found in review, two of which defeated the feature outright: - The snapshot shipped under a minted placeholder epoch instead of the surface's real stream epoch. A cold attach installs the tee at attach time, so no byte has been teed yet -- exactly the flagship case. The screen repainted correctly and was then wiped by the first live chunk's epoch mismatch, whose retry was eaten by the capture throttle. The epoch now travels with the capture. - PaneController.attachBytes re-wrapped the caller's effects by listing hooks by hand and dropped onGrid. It is optional on the interface so the compiler said nothing, and the pane screen is the only production caller -- the whole geometry policy was dead code in the shipped path. The wrapper spreads now. - A cold attach falling back to a byte tail reported truncated: false whenever the tail happened to fit, so the phone wrote a mid-escape tail on top of an already-painted screen without resetting. A tail is never a continuation. - The snapshot cap was sized against the relay's frame cap, repeating the mistake the replay cap had already been corrected for: the binding constraint is the per-device byte bucket, whose entire one-second capacity is that same 256 KiB. A max-size snapshot ate ~89% of it, and the post-attach redraw burst -- densest exactly then -- was rate-limited away. Cut to 48 KiB. - Added conformance vectors for the snapshot fields. Invalid-case vectors are deliberately NOT included: Swift's Codable cannot reject a negative Int or a non-base64 String, only zod can, and a vector must decode identically in both languages. Those cases live in wire/test/pane-bytes-bounds.test.ts.
The CI install was failing with ERR_PNPM_IGNORED_BUILDS. pnpm 11 moved this setting out of package.json into pnpm-workspace.yaml, but it also renamed it: the key is `allowBuilds` (a map of package -> bool), not `onlyBuiltDependencies` (a list). pnpm accepts the old name into the file and then silently ignores it, so a fresh install still refuses to run the script. The placeholder that was in this file originally (`esbuild: set this to true or false`) was pnpm's own scaffold, written when it hit the error -- it only ever needed its value filled in. Verified against a clean tracked-files-only checkout rather than a warm node_modules, which is what hid this the first time: install, wire build, typecheck and the full test suite all pass from scratch.
On a branch with an open PR both `push` and `pull_request` fired, and the concurrency group cannot dedupe them: the two events carry different refs and different SHAs (the PR event runs against the merge commit). Limiting push to main leaves PR branches covered by the pull_request event alone.
…endpoint
Two fixes to direct-LAN reachability.
1. iOS/Android local networking. app.json declared no network keys at all, so a
real build could not reach the Mac over the LAN:
- NSLocalNetworkUsageDescription: without it iOS 14+ never shows the local
network prompt, so .local resolution and LAN sockets fail outright.
- NSAppTransportSecurity.NSAllowsLocalNetworking: the direct transport is
ws:// (cleartext at the transport layer, E2E-encrypted above it), which ATS
blocks by default. Scoped to local networking; public hosts stay TLS-only.
- NSBonjourServices: required to browse _zentty._tcp, which the Mac already
advertises with its deviceId in the TXT record.
- Android: usesCleartextTraffic for the same ws:// reason, plus the network
state and multicast permissions mDNS needs.
This worked in development only because the Expo dev client ships its own
local-network entitlements for Metro -- it would have broken in TestFlight.
2. Self-healing lanHint. session.ready now restates the Mac's current endpoint,
and the phone persists it when it differs from the cached one.
The cached hint is less fragile than it first looks: hostName returns the
Bonjour .local name, not an address, so it re-resolves per connect and
already survives a DHCP change. What it does not survive is the Mac being
renamed or the listener landing on a different port -- and today the only
recovery from that is re-pairing.
Restating it on every handshake means ANY working path refreshes the cache,
including the relay, which is exactly what is still reachable when a stale
hint has broken the direct path. An ABSENT hint means "no listener right
now", never "forget the cached one" -- a Mac reachable only over the relay
today may be reachable directly later.
Bonjour browsing on the phone is deliberately NOT included. .local resolution is
already mDNS, so browsing only adds surviving a rename without any prior
connection, and it costs a native module. Worth revisiting only if the above
turns out not to be enough.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #52.
A React Native (iOS + Android) companion for Zentty on macOS. Pair a phone, watch every running agent, get pushed when one needs you, and answer from the couch — approve a Claude Code prompt, type into a pane, or take real control of a terminal reflowed to phone width. No accounts; pairing is a QR scan and two keypairs.
The whole thing is verified end to end on an iOS simulator against a fake-Mac harness that speaks the real wire protocol: pair, dashboard, terminal, approve, transcript, takeover, live update, offline. Screenshots below.
How it fits together
Four pieces, one repo. The macOS world at the root is untouched; everything new lives under
companion/(one pnpm workspace) plus aZentty/Companion/Swift module in the app.companion/wire) — zod schemas for every message, with JSON test vectors that both the TypeScript and the Swift sides round-trip. Protocol drift becomes a failing test instead of a runtime mystery. The crypto derivation is pinned the same way: fixed-key vectors that CryptoKit and @StableLib both have to reproduce byte for byte.Zentty/Companion/) — reuses the existing agent-status pipeline, pane text (readText), and input paths rather than reinventing them. A phone is just another client, over a network instead of the unix socket. Runs inert until a device is paired.companion/relay) — a zero-knowledge Node service that forwards opaque encrypted frames between paired devices, plus the push gateway. One Docker image; self-hosters point the app at their own URL. It never sees plaintext.companion/mobile) — Expo SDK 57, named Zentty, using the desktop icon.Transport is direct-LAN-first (Bonjour) with the relay as fallback; the encrypted session rides over either identically, so the relay and the LAN are equally untrusted.
What you can do from the phone
Security
Ed25519 device identity in the Keychain, X25519 + HKDF handshake with transcript signing, directional ChaCha20-Poly1305 sessions with replay protection. Pairing proves possession of a one-time secret; devices are revocable from either side. The relay authenticates devices by public key and stamps the sender so nobody can spoof a peer. A code review pass confirmed the crypto and the takeover seam are solid; its follow-ups (feature-disable now severing live sessions, relay DoS caps, a few mobile robustness bugs) are in the last commit.
Not in this PR / needs you
Testing
3743 Swift logic tests and 308 TypeScript tests, all green. The wire vectors and crypto vectors are checked in and enforced on both sides. Full flow screenshotted on the simulator.