Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
# MODEL_SYNTH_MODEL=anthropic/claude-sonnet-5 # Strong synth model
# MODEL_REVIEW_MODEL=google/gemini-3.8-flash # Finish-review model
# MODEL_SYNTH_ENABLED=1 # Enable synth pass on finish
# WEBPILOT_FAST_MODE=1 # Jev picks browser actions (OpenRouter only)
# JEV_MODEL=typesafe/jev-1.13 # Jev decision model
# MODEL_TIMEOUT_MS=120000 # Per-call timeout (ms)

# OpenRouter credentials (https://openrouter.ai/settings/keys)
Expand Down
17 changes: 17 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,23 @@ Dependabot opens a weekly grouped PR for `playwright` and `playwright-core` (sta
3. Run one real task (`npm run agent:cli -- "Go to https://example.com and tell me the page heading."`) to confirm the snapshot format still parses.
4. For releases, run the `Package Desktop` workflow to package and smoke-test the desktop app on every platform.

## Fast Mode (Jev)

Fast mode (`fastMode` in settings, `WEBPILOT_FAST_MODE=1`) puts TypeSafe's Jev decision model in charge of each browser step. Jev returns typed decisions with calibrated probabilities in ~0.3s instead of generating text, so it replaces the LLM wherever the question is really a choice. It is reached through OpenRouter with the same API key (`/api/v1/systemone`, model `typesafe/jev-1.13`, override with `JEV_MODEL`); it is not available with Ollama.

Modules under `lib/jev/`:

- `client.ts`: the decision API (`choice` and `noul` questions, retries, answer parsing).
- `page.ts`: builds the page state from the accessibility snapshot alone (a numbered list of actionable elements with landmark info, plus a text excerpt). Every Playwright MCP `browser_evaluate` costs ~0.5-1s, so fast mode avoids them. Pages with more than ~250 elements are trimmed to the ones in main content that share words with the task.
- `fast-mode.ts`: the step loop. One Jev call per step asks a flat `action` choice (`click_<ref>`, `type_<ref>`, scroll, back, done, blocked) plus `goal_done`, `stuck` and `submit_after_typing`. The LLM only writes text for inputs and phrases the final answer.
- `gates.ts`: Jev yes/no gates in front of LLM calls. A preflight asks "needs the browser?" and "independent sites in parallel?" in one call, skipping the LLM router and split analysis when Jev is confident. An answer check accepts a supported final answer without the LLM reviewer; Jev never rejects on its own.

Control returns to the LLM planner (with a summary of the steps taken) when Jev reports the task blocked or stuck, several actions in a row make no visible change, the next click looks irreversible (buy, pay, delete, send...), there is no starting page, or the step/time budget runs out. Actions that made no change are removed from the options before Jev decides again.

Each Jev step is recorded with `source: "jev"`, and `stepN_jev.json` artifacts keep Jev's probabilities for tuning thresholds. `npm run bench:fast-mode` compares fast mode with the LLM-only agent on the same tasks.

Privacy: in fast mode the page text excerpt, element labels and URLs of every step are sent to TypeSafe (through OpenRouter).

## Tool Schema Versioning

The agent exposes a stable WebPilot browser tool schema from `lib/tool-schema.ts` instead of scattering raw provider payloads through the runtime. The current schema version is `webpilot.browser-tools.v1`.
Expand Down
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ WebPilot is meant to be a hackable, local-first alternative to closed agentic br

- Desktop app built with Electron and Next.js.
- Browser automation through Playwright MCP, running on the latest stable Playwright.
- Fast mode: TypeSafe's Jev decision model (through OpenRouter) picks each browser action in ~0.3s while the LLM only writes text, with automatic fallback to the LLM planner.
- Model providers: OpenRouter (one key for Gemini, Claude, GPT, Qwen, DeepSeek and hundreds more) and local Ollama.
- Runtime settings UI for provider, model, browser source, profile strategy, headless mode, and isolation.
- Local run recording with logs, step traces, artifacts, timing, and final results.
Expand Down Expand Up @@ -79,6 +80,18 @@ MODEL_BASE_URL=http://127.0.0.1:11434/v1

The settings UI can discover local Ollama models and hides known embedding-only models when Ollama reports enough metadata to identify them.

## Fast Mode

Turn on **Fast mode (Jev)** in Settings (OpenRouter only). Instead of asking the LLM to plan every click, WebPilot asks [Jev](https://openrouter.ai/typesafe/jev-1.13), a decision model that returns a choice with probabilities in about 0.3 seconds. The LLM still writes anything that needs text (search queries, the final answer), and the regular planner takes over whenever Jev is stuck, blocked, or about to click something irreversible.

Measure it on your own tasks:

```bash
OPENROUTER_API_KEY=sk-or-... npm run bench:fast-mode
```

In fast mode the page text and element labels of each step are sent to TypeSafe through OpenRouter.

## Browser Modes

The app supports these runtime browser modes:
Expand All @@ -103,6 +116,7 @@ npm run browsers:install
npm run browser:smoke
npm run health
npm run agent:cli -- "Go to https://example.com and summarize the page"
npm run bench:fast-mode
npm run mcp:stdio
npm run desktop:build
```
Expand Down
2 changes: 2 additions & 0 deletions RUNNING.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ Optional:
- `MODEL_NAV_MODEL=...`
- `MODEL_SYNTH_MODEL=...`
- `MODEL_SYNTH_ENABLED=1`
- `WEBPILOT_FAST_MODE=1` (Jev picks browser actions; OpenRouter only)
- `JEV_MODEL=typesafe/jev-1.13`
- `RUN_STORE_DIR=./agent_runs`
- `THREAD_STORE_DIR=./agent_threads`
- `MAX_STEPS=160`
Expand Down
12 changes: 12 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,18 @@ This verifies the versioned browser tool schema adapter against current WebPilot

It also covers the OpenRouter/Ollama model client with a mocked `fetch`: request URLs and headers, tool-call round trips including `reasoning_details`, retries on rate limits, error messages, migration of legacy Gemini/OpenAI/Claude settings, and parsing of the OpenRouter models catalog. No API key or network access is needed.

`tests/jev.test.ts` covers fast mode without network access: accessibility-snapshot parsing, candidate selection, the Jev client, the step loop against a fake browser (done, no-progress handoff, irreversible-click guard, typing with submit), and the Jev gates.

## Fast Mode Benchmark

```bash
npm run browsers:install
OPENROUTER_API_KEY=sk-or-... npm run bench:fast-mode
npm run bench:fast-mode -- --only mosaic,voyager --modes fast --repeat 3
```

Runs a fixed set of public-web tasks (Wikipedia, MDN, example.com) headless in both the LLM-only mode and fast mode, checks each final answer against expected patterns, and writes JSON and Markdown reports with time, steps, LLM calls, Jev calls and fast-mode handoffs to `e2e_reports/`. Model latency on OpenRouter varies run to run, so compare medians over `--repeat 3` or more before drawing conclusions.

## Deterministic Browser Smoke

```bash
Expand Down
2 changes: 2 additions & 0 deletions app/activity/[runId]/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,8 @@ const PERFORMANCE_LABELS: Record<string, string> = {
synthDurationMs: "Synth time",
coordinatorCallCount: "Coordinator calls",
coordinatorDurationMs: "Coordinator time",
jevCallCount: "Jev decisions",
jevDurationMs: "Jev time",
};

function formatSize(sizeBytes?: number) {
Expand Down
14 changes: 14 additions & 0 deletions components/settings/ProviderForm.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -502,6 +502,20 @@ export function ProviderForm() {
label="Synthesis enabled"
/>

{provider === "openrouter" && (
<div className="min-w-0 space-y-1">
<Toggle
checked={runtime.fastMode === true}
onChange={(v) => update({ fastMode: v })}
label="Fast mode (Jev)"
/>
<p className="text-[12px] leading-snug text-wp-text-secondary">
Jev, a fast decision model on OpenRouter, picks each browser action in about 0.3s and the navigation model only writes text.
It falls back to the normal planner when stuck. Page text and element labels are sent to TypeSafe for each step.
</p>
</div>
)}

<div className="min-w-0 space-y-3 rounded-[var(--wp-radius)] border border-wp-border bg-wp-surface/30 p-3">
<div className="flex min-w-0 flex-wrap items-center justify-between gap-2">
<label className="text-xs font-medium text-wp-text-secondary uppercase tracking-wider">
Expand Down
4 changes: 3 additions & 1 deletion electron/main.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -687,6 +687,7 @@ function runtimeSettingsDefaults() {
synthModel,
reviewModel,
synthEnabled: boolFromInput(process.env.MODEL_SYNTH_ENABLED, true),
fastMode: boolFromInput(process.env.WEBPILOT_FAST_MODE, false),
browser: sanitizeBrowserSettings(),
};
}
Expand All @@ -697,7 +698,7 @@ function sanitizeRuntimeSettings(value) {
// Settings saved for Gemini/OpenAI/Claude migrate to OpenRouter defaults:
// their API keys and model ids are not valid there.
const input = isLegacyProvider(raw.provider)
? { synthEnabled: raw.synthEnabled, browser: raw.browser }
? { synthEnabled: raw.synthEnabled, fastMode: raw.fastMode, browser: raw.browser }
: raw;
const provider = normalizeProvider(input.provider || defaults.provider);
const providerDefaults = defaultModelsForProvider(provider);
Expand All @@ -709,6 +710,7 @@ function sanitizeRuntimeSettings(value) {
synthModel: typeof input.synthModel === "string" && input.synthModel.trim() ? input.synthModel.trim() : providerDefaults.synthModel,
reviewModel: typeof input.reviewModel === "string" && input.reviewModel.trim() ? input.reviewModel.trim() : providerDefaults.reviewModel,
synthEnabled: boolFromInput(input.synthEnabled, defaults.synthEnabled),
fastMode: boolFromInput(input.fastMode, defaults.fastMode),
browser: sanitizeBrowserSettings(input.browser),
};
}
Expand Down
Loading
Loading