Repository navigation
Fast mode: Jev picks browser actions, the LLM only writes text - #35
Merged
Merged
Conversation
ahamsel
force-pushed
the
claude/jev-fast-mode
branch
from
September 28, 2026 17:20
f71e9f8 to
2410a14
Compare
Fast mode lets TypeSafe's Jev decision model (through OpenRouter, same key) pick each browser action, while the LLM only writes typed text and the final answer. Off by default; toggle in Settings (OpenRouter only). - lib/jev/client.ts: System One decision API (choice/noul questions) - lib/jev/page.ts: page state from the accessibility snapshot alone (every browser_evaluate costs ~0.5-1s); unnamed links take their child text; large pages are trimmed to task-relevant main content - lib/jev/fast-mode.ts: one Jev call per step with a flat action choice plus goal_done/stuck/submit; failed or no-op actions are excluded per URL; handoff to the LLM planner when blocked, stuck, repeating, over budget, or before irreversible-looking clicks - lib/jev/gates.ts: Jev preflight skips the LLM router (first turn only) and split analysis when confident; answer check accepts a supported answer without the LLM reviewer (never rejects alone) - Jev decisions logged and saved as step artifacts; performance summary counts Jev calls - scripts/bench-fast-mode.ts (npm run bench:fast-mode) compares fast mode with the LLM-only agent on fixed public-web tasks Benchmark (8 tasks, 1 run each): LLM-only 8/8, median 19.0s; fast mode 8/8, median 4.3s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ahamsel
force-pushed
the
claude/jev-fast-mode
branch
from
September 28, 2026 17:20
2410a14 to
214fe93
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on #33 (OpenRouter) and #34 (Playwright 1.63), both merged.
Adds fast mode: Jev, TypeSafe's "System One" decision model, picks each browser action through OpenRouter with the same API key. Jev doesn't generate text. It answers typed questions (choose one of N, yes/no) with calibrated probabilities in ~0.3s, so it replaces the LLM wherever the question is really a choice. The LLM still writes anything textual (search queries, the final answer), and the regular LLM planner takes over whenever Jev can't finish. Off by default; Fast mode (Jev) toggle in Settings (OpenRouter only).
Step loop (
lib/jev/fast-mode.ts): one Jev call per step asks a flatactionchoice (click_<ref>,type_<ref>, scroll, back, done, blocked) plusgoal_done,stuckandsubmit_after_typingin the same call. Hands off to the LLM planner (with a summary of the steps taken) when Jev reports blocked or stuck, repeated actions change nothing, the next click looks irreversible (buy, pay, delete, send...), there's no starting page, or the step/time budget runs out. Actions that errored or changed nothing are removed from the options, per URL.Page state (
lib/jev/page.ts): built from the accessibility snapshot alone, because each Playwright MCPbrowser_evaluatecosts ~0.5-1s. Elements carry landmark info (main content vs site navigation); unnamed links take their child text (docs sites like MDN wrap link text in<code>); pages with more than ~250 actionable elements are trimmed to task-relevant main content.Jev gates in front of LLM yes/no calls (
lib/jev/gates.ts):Observability: Jev decisions are logged (
jev_decision) and saved asstepN_jev.jsonartifacts with probabilities for tuning thresholds; steps are recorded withsource: "jev"; the performance summary counts Jev calls and time.Benchmark (
npm run bench:fast-mode): runs fixed public-web tasks headless in both modes and checks answers against expected patterns.One run per task per mode with
google/gemini-3.8-flash; OpenRouter latency varies run to run, so treat these as indicative. In that run the MDN task handed off to the LLM because themap()link had no accessible name and was dropped by the parser, and a failing click was retried; both are fixed here, and MDN now completes in fast mode.Privacy: in fast mode the page text excerpt, element labels and URLs of every step are sent to TypeSafe through OpenRouter. Documented in README, ARCHITECTURE and next to the Settings toggle.
Testing
npm run buildnpm run healthelectron/changed (node --check electron/main.cjs)npm test(28 pass;tests/jev.test.tscovers the parser, candidate selection, Jev client, step loop against a fake browser, and the gates),npm run lint,npx tsc --noEmitChecklist
🤖 Generated with Claude Code