[Feat] Served OneJev multimodal decision backend (--model onejev)

CHANGELOG.md

@@ -4,6 +4,11 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); ver
 
 ## Unreleased
 
+### Fixed
+
+- Served OneJev uses one HTTP decision deadline across its permitted retry. A retry receives only the remaining
+  `ONEJEV_TIMEOUT_S` budget, and an exhausted budget fails without sending another request.
+
 ### Changed
 
 - The direct TypeSafe path pins `jev-1.13.0` instead of the moving `jev-latest` alias (which moved to Jev 1.14.0
@@ -14,6 +19,12 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); ver
 
 ### Added
 
+- `--model onejev`: [OneJev](https://github.com/OmniJev/OneJev) (Apache-2.0) as a **served** multimodal System One
+  client against `qev`/`flide` (`ONEJEV_URL`, optional `ONEJEV_MODEL` / `ONEJEV_API_KEY` / `ONEJEV_TIMEOUT_S`). Posts
+  choice/noul with TypeSafe-compatible `media` and `<image:N>` state markers; refuses an observation without a
+  screenshot (OmniJev policy). Contract tests use a fake HTTP server; live GPU validation is not claimed. Browser
+  and `build_model` only (#88).
+
 - `s1a probe --order reverse|shuffle:N` (with `--seed` and `--json PATH`): the option-order check. Every case is
   asked in file order and again with the same keys and texts in another dict order; the probe reports the flip
   rate, accuracy in each order and a row per case with every order asked. Verdict and exit code are unchanged

NOTICE

@@ -21,3 +21,5 @@ This product includes software developed by third parties:
 - evals/labelled/injection-public.jsonl: tool outputs from AgentDojo, ethz-spylab/agentdojo
   (MIT, Copyright (c) 2024 Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca
   Beurer-Kellner, Marc Fischer, and Florian Tramèr). Full license text: THIRD_PARTY_LICENSES.md
+- s1a/decision_models/onejev.py: HTTP client for OneJev / qev (OmniJev/OneJev), Apache-2.0;
+  the weights and server are not vendored. Details: THIRD_PARTY_LICENSES.md

THIRD_PARTY_LICENSES.md

@@ -116,3 +116,13 @@ TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONIN
 THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
 CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 DEALINGS IN THE SOFTWARE.
+
+## OneJev / qev
+
+The `--model onejev` adapter (`s1a/decision_models/onejev.py`) talks to a locally served
+[OneJev](https://github.com/OmniJev/OneJev) checkpoint through the project's `qev` or
+[Flide](https://github.com/OmniJev/Flide) HTTP server (`POST /v1/systemone` with a `media` field).
+OneJev and its serving code are Apache License 2.0 (Copyright OmniJev Team). This repository does not
+vendor OneJev weights, training code, or the `qev`/`flide` packages; install and run them separately
+per the model card. The unrelated PyPI package named `qev` is not the OneJev server — install from the
+OneJev GitHub repository as the model card instructs.

docs/agents.md

@@ -19,7 +19,7 @@ hook of a running agent. The injection guard rail fails closed: a decision error
 
 Every tool agent takes `--model jev|clm|laya|laya-served|cua|llm|random|rule`, `--rethink on|off`, `--episodes N`, `--seed S`,
 `--max-steps` and `--timeout`, and writes a Harbor-shaped job folder under `evals/results/<agent>/`. A browser agent
-takes `--model jev|clm|laya|laya-served|cua|omnijev|llm`, `--goal` and the same `--rethink` flag.
+takes `--model jev|clm|laya|laya-served|cua|omnijev|onejev|llm`, `--goal` and the same `--rethink` flag.
 The desktop agent adds `--rethink-attempts` and `--rethink-timeout` and stalls after 3 actions without observed
 progress. The browser front takes the same three names; `docs/browser-front.md` decision 19 describes its branch. A rail takes
 `--model jev|clm|laya|laya-served`, the models that answer `noul`.

docs/architecture.md

@@ -12,7 +12,7 @@ score, seconds, steps and dollars.
 sequenceDiagram
     participant A as openJiuwen agent
     participant E as page or environment
-    participant S1 as Jev, Laya, Cua-S1 Nano or OmniJev (System 1)
+    participant S1 as Jev, Laya, Cua-S1 Nano, OmniJev or OneJev (System 1)
     participant C as chat model (System 2)
     A->>E: observe
     E-->>A: state + enumerated options
@@ -35,7 +35,7 @@ the description, the rules text the model reads, a budget, and the front-specifi
 - Rail front (`RailSpec`, `s1a/rails.py`): `DecisionModelRail` asks one question at one DeepAgent callback hook,
   acts on it above a threshold, and is evaluated on a labelled set. Agent: `injection_guard`.
 
-Shared: `s1a/decision_models/` (the decision-model interface, the Jev, Laya, Cua-S1 and OmniJev adapters, the baselines, the answer
+Shared: `s1a/decision_models/` (the decision-model interface, the Jev, Laya, Cua-S1, OmniJev and OneJev adapters, the baselines, the answer
 validation; `docs/decision-models.md`), `s1a/decision_models/wire.py` (the HTTP transport and its two backends),
 `s1a/config.py` (env loading, the chat model, the browser launch flags),
 `s1a/tool/rethink.py` (a rail that blocks repeats and asks for a plan on a stall), `s1a/jobs.py`
@@ -50,6 +50,7 @@ validation; `docs/decision-models.md`), `s1a/decision_models/wire.py` (the HTTP
 | `laya` | [Laya](https://huggingface.co/convaiinnovations/laya) (`convaiinnovations/laya`, 0.4B) | in process, `uv sync --extra laya`; no key | a 512 to 1024 token window |
 | `cua` | [Cua-S1 Nano](https://huggingface.co/cua-ai/cua-s1-nano-0.1) (`cua-ai/cua-s1-nano-0.1`, 855K) | in process, `uv sync --extra cua`; no key | a 256-byte context (header, goal, state, then rules), 96 bytes per option |
 | `omnijev` | [OmniJev](https://github.com/tinnel123666888/OmniJev) (Qwen3.5 VL, 0.8B by default, 2B or 4B) | in process, browser agents only, `uv sync --extra omnijev` and a clone named by `OMNIJEV_REPO`; no key; a GPU for useful speed | a viewport screenshot per tick, plus the text state (recent actions first, page text capped at 2,000 characters) |
+| `onejev` | [OneJev](https://github.com/OmniJev/OneJev) (0.8B–27B, Apache-2.0) via `qev`/`flide` | served HTTP, browser agents only; `ONEJEV_URL`; no cloud key | a viewport screenshot per tick as `media` with `<image:N>` in state |
 | `random`, `rule` | the tool front's two baselines | in process | the candidates |
 | `llm` | the chat model | for the comparison columns | the transcript |
 
@@ -63,7 +64,7 @@ the first import of both, routes the harness logs to files under `runs/logs` bef
 MCP stdio protocol only.
 
 Every `run` prints one JSON object on stdout: a tool agent's series summary with its `job_dir`, a browser agent's
-answer, a rail's evaluation. A browser agent takes `--model jev|laya|cua|omnijev|llm`; its policy switches are run-time flags:
+answer, a rail's evaluation. A browser agent takes `--model jev|laya|cua|omnijev|onejev|llm`; its policy switches are run-time flags:
 `--batch on|off`, `--prefetch on|off`, `--goal-values on|off`. `s1a-mcp` serves the same agents to an MCP host over stdio,
 one Runner for the server's lifetime and one run at a time. `uv run python -m evals.table evals/results`
 aggregates every job folder per eval and model into one table. `scripts/showcase.sh` plays one visual episode per
@@ -72,7 +73,7 @@ eval and model outside the matrix and `python -m evals.replay` renders a pair si
 
 Every tool agent, `desktop` included, takes `--model jev|laya|cua|llm|random|rule`, `--rethink on|off`,
 `--episodes N`, `--seed S`, `--max-steps`, `--timeout` and `--headed`, and writes a Harbor-shaped job folder under
-`evals/results/<agent>/`. Every browser agent takes `--model jev|laya|cua|omnijev|llm` and `--goal`. A rail takes
+`evals/results/<agent>/`. Every browser agent takes `--model jev|laya|cua|omnijev|onejev|llm` and `--goal`. A rail takes
 `--model jev|laya`, the two models that answer `noul`. `decide` and `probe` take `--model jev|laya|cua`. On a browser
 agent `laya` needs `LAYA_MAX_LEN` raised to the page's size; `cua` reads a 256-byte context (header, goal, state,
 then rules) and 96 bytes per option, a baseline on any page. Exit codes: 0 for a finished run, including one whose

docs/capabilities.md

@@ -16,6 +16,7 @@ For issue trackers and serving support, see [System1-Omni model tracker #83](htt
 | LAYA (`--model laya`; default `convaiinnovations/laya`, English; `LAYA_SUBFOLDER=multilingual` selects multilingual) | Text | `choice`, `noul` | Tool, browser, rails, `decide`/`probe`; browser state/question shaping is implemented | In-process Python `laya` | Historical English and multilingual CPU/MPS runs are documented below. Other/typed/custom checkpoints and CUDA are unverified here. |
 | LAYA (`--model laya-served`; served checkpoint/revision is reported by the endpoint) | Text | `choice`, `noul` | Tool, browser, rails, `decide`/`probe`, MCP | Agents HTTP client can connect directly to a Laya worker or through System1-Omni `omni-jev` at `/v1/systemone` | Historical direct-worker and Omni-front MPS ticket-router measurements are recorded below (90 decisions/configuration, 63 correct each); historical CPU served request replay is separate and is not task-success evidence. Other hardware/checkpoints unverified. |
 | OmniJev (`--model omnijev`; default `tinnel123/OmniJev-0.8B@v1.1` on `Qwen/Qwen3.5-0.8B`; configurable 2B/4B) | Screenshot plus text state/context | `choice`, `noul` | Browser front via `s1a run` and MCP `run_agent`; direct `build_model()` factory path. Not registered in tool/rails, CLI `decide`/`probe`, or MCP `decide` | In-process OmniJev clone via `MSO1.system_one`; no System1-Omni served vision path | Historical A100 80 GB 4B runs, Windows 11 CPU 0.8B runs, and A40 46 GB 4B runs are detailed below. They do not establish broad task success; current-main reproduction unverified. |
+| OneJev (`--model onejev`; default request id `OmniJev/OneJev1.6-4B`; set `ONEJEV_MODEL`) | Screenshot(s) plus text state/context via TypeSafe `media` / `<image:N>` | `choice`, `noul` | Browser front via `s1a run` and MCP `run_agent`; direct `build_model()` factory path. Not registered in tool/rails, CLI `decide`/`probe`, or MCP `decide` | HTTP client to `qev`/`flide` at `ONEJEV_URL` (`POST /v1/systemone` + `media`) | Contract tests use a fake HTTP server. Live GPU / GGUF hardware validation **not recorded in this checkout** (#88). |
 | Cua-S1 Nano (`cua-ai/cua-s1-nano-0.1/text`, `--model cua`, default variant) | Text | `choice` only | Tool/browser paths where registered; no screenshot input | In-process `cua_s1.nano.NanoScorer` | No real Nano inference artifact found in this checkout. **CPU/GPU/MPS unverified here.** |
 | Cua-S1 4B 0.2 text adapter (`Qwen/Qwen3.5-4B` + `cua-ai/cua-s1-4b-0.2/text`, `CUA_S1_VARIANT=4b`, `CUA_S1_MODALITY=text`) | Text | `choice` only | Tool decisions and text-only client paths; desktop option grounding is implemented | In-process Transformers/PEFT scorer | Contract tests use a fake scorer; real checkpoint/device validation **not present in this checkout**. |
 | Cua-S1 4B 0.2 multimodal adapter (`Qwen/Qwen3.5-4B` + `cua-ai/cua-s1-4b-0.2/multimodal`, variant `4b`) | One current screenshot plus text task/context | `choice` only; maximum 26 options in text and multimodal modes; multimodal requires exactly one screenshot | Screenshot-dependent desktop decision adapter; this does not itself complete the Omni capture/execute bridge | In-process Transformers/PEFT scorer with screenshot passed as a temporary file | Contract tests use a fake scorer; separate historical external evidence reports a bounded MPS fixture run (see below). Current-main reproduction and CUDA validation remain unverified. |

docs/configuration.md

@@ -53,6 +53,10 @@ Variables can be exported in your shell or placed in a `.env` file at the root o
 | `OMNIJEV_REVISION` | `omnijev` model | `v1.1` | Revision of a Hugging Face `OMNIJEV_CHECKPOINT`. |
 | `OMNIJEV_BASE` | `omnijev` model | `Qwen/Qwen3.5-0.8B` | Hugging Face ID or local directory of the Qwen3.5 backbone of the same size as the checkpoint. |
 | `OMNIJEV_PROMPT` | `omnijev` model | `full` | `full` sends the text state, goal and rules with each question; `short` sends the goal and the ask only, the screenshot carrying the page. |
+| `ONEJEV_URL` | `onejev` model | *(unset, required)* | Base URL of a served OneJev (`qev`/`flide`), e.g. `http://127.0.0.1:8000`. Browser and `build_model` only; requires a screenshot in the observation. |
+| `ONEJEV_MODEL` | `onejev` model | `OmniJev/OneJev1.6-4B` | Model id sent in the `/v1/systemone` body (checkpoint the server loaded, e.g. `OmniJev/OneJev1.6-0.8B`). |
+| `ONEJEV_API_KEY` | `onejev` model | *(unset)* | Bearer token when the server expects one; loopback `qev`/`flide` usually needs none. |
+| `ONEJEV_TIMEOUT_S` | `onejev` model | `30` | Deadline per decision in seconds, the one refused-connection / 502 / 504 retry included. A finite number above 0. Raise for large checkpoints or a cold load. |
 | `CUA_DRIVER_BIN` | `desktop` agent | `cua-driver` | Path to the `cua-driver` executable on Windows or macOS when not located on `PATH`. |
 | `CUA_DRIVER_PERMISSION_MODE` | `desktop` agent | `standard` | Permission mode passed to `cua-driver mcp` (`standard`, or `bounded` for restricted capability manifests). |
 | `HF_HOME` | Hugging Face runtime | `~/.cache/huggingface` | Cache directory where Laya and Cua-S1 checkpoints are downloaded on first run. |

docs/decision-models.md

@@ -12,7 +12,7 @@ TypeSafe "System One model". This repository uses the terms interchangeably. `--
 `clm` (CLM's `clm-serve` over HTTP, see [clm.md](clm.md)),
 `laya` (in process, behind `uv sync --extra laya`), `laya-served` (Laya served over HTTP by system1-omni, see
 [served-laya.md](served-laya.md)), `cua` (Cua-S1 Nano in process, behind `uv sync --extra cua`),
-`random` and `rule` (the tool front's baselines).
+`onejev` (OneJev over HTTP via `qev`/`flide` with a screenshot), `random` and `rule` (the tool front's baselines).
 `build_model(model_name, seed=, rule=)` builds one from the environment; `llm` names the chat model, which `build_model` does not build.
 
 The registered paths, input modalities and question types are summarized in the [decision-model capability matrix](capabilities.md), separately from run evidence and hardware validation.
@@ -65,6 +65,7 @@ shorthands; `warm()` and `close()` open and release the backend.
 | `laya-served` | `ServedLayaModel(client, model=, max_len=)` | `laya-served` | one `POST /v1/systemone` per request to `LAYA_SERVED_URL`, body built with `laya_question()` as for `laya`; one retry on a dropped connection, 502 or 504, and after `Retry-After` on 503, all within `LAYA_SERVED_TIMEOUT_S`; the same window check as `laya`; `model` is the served checkpoint and revision, and `raw` keeps `served_by`, `url`, `request_id` and `server_timing` (see [served-laya.md](served-laya.md)) |
 | `cua` | `CuaS1Model(scorer, collator, model=, context_bytes=, option_bytes=)` | `cua` | Cua-S1 Nano, one `score_elements` pass per request on a thread; choice questions only, text only, deterministic; the context is header, state and rules; the checkpoint reads its first 256 bytes, and the first overflowing request logs one warning; `from_env` reads `CUA_S1_*` (see [configuration.md](configuration.md)) |
 | `omnijev` | `OmniJevModel(agent, model=)` | `omnijev` | OmniJev (a Qwen3.5 vision-language decision model), one `MSO1.system_one` call per request on a thread; choice and noul questions over the observation's screenshot, deterministic; the text state, goal and rules go in front of each question (a browser state puts its recent actions first and caps the page text at 2,000 characters, so a long page cannot push the history out of the 6,000-character context), a browser element row becomes its label and value; a choice's probabilities, which OmniJev leaves `abstain` out of, are renormalised over the offered keys; an observation without an image is `MODEL_CALL_FAILED`; `from_env` reads `OMNIJEV_*` (see [configuration.md](configuration.md)); in process only: system1-omni has no OmniJev vision worker yet (its supported models are Laya, Open-Jev text and Cua-S1 screenshots), so there is no served `omnijev` path |
+| `onejev` | `OneJevModel(client, model=)` | `onejev` | OneJev (OmniJev org, Apache-2.0) served by `qev`/`flide` over TypeSafe-compatible `POST /v1/systemone` plus a `media` field; choice and noul over the observation's screenshot(s), deterministic; state carries `<image:N>` markers and each screenshot is a PNG/JPEG/WebP data URL in `media` (digest and size in `raw.sent_images`, never the bytes); an observation without an image is `MODEL_CALL_FAILED` (same policy as `omnijev`); one refused-connection / 502 / 504 retry within `ONEJEV_TIMEOUT_S`; `raw` keeps `served_by`, `url`, `request_id` and `server_timing`; `from_env` reads `ONEJEV_*` (see [configuration.md](configuration.md)); browser and `build_model` only — not tool/rails/`decide`. Local tests use a stub server; live GPU validation is **not recorded** in this checkout |
 | `random` | `RandomModel(seed)` | `random` | uniform over the offered keys (contract confidence = `1/|options|`), one seeded stream per episode; choice questions only |
 | `rule` | `RuleModel(name, rule)` | the rule's name | one-hot, confidence 1; a key outside the menu raises `RuntimeError`, a bug in the rule |
 
@@ -80,6 +81,26 @@ and API keys are referenced in [configuration.md](configuration.md). Latency and
 price: `benchmarks.md`. Laya (Convai Innovations, open weights, 0.4B parameters) and Cua-S1 Nano (Cua, 855K
 parameters) answer the same `choice` question in process.
 
+
+### Served OneJev
+
+`onejev` is registered in the browser front and `build_model()`. Point `ONEJEV_URL` at a local
+[`qev`](https://github.com/OmniJev/OneJev) or [`flide`](https://github.com/OmniJev/Flide) server that speaks
+`/v1/systemone` with `media`. Example smoke (not run in CI; needs a GPU or GGUF build):
+
+```bash
+# terminal 1 — install from the OneJev repo (pip install qev is a different package)
+qev serve --model OmniJev/OneJev1.6-4B --port 8000
+# or: flide serve --model OmniJev/OneJev1.6-4B --port 8000
+
+# terminal 2
+ONEJEV_URL=http://127.0.0.1:8000 ONEJEV_TIMEOUT_S=60 \
+  uv run s1a run flights --model onejev --rethink off
+```
+
+This is a setup recipe only. Capability registration and stub-server contract tests do **not** establish task
+success rates or hardware validation; see [capabilities.md](capabilities.md).
+
 ### Served JEV-VL
 
 `jev-vl-served` is registered in the browser front and `build_model()`. It sends the observation's text state,

s1a/browser/browse.py

@@ -48,6 +48,7 @@
     "laya-served",
     "cua",
     "omnijev",
+    "onejev",
     "llm",
 )  # a decision model over HTTP or in process, or the chat model
 RUNS_DIR = HOME / "runs" / "browser"
@@ -253,7 +254,7 @@ async def browse(
     workspace = str(logs_dir / "workspace")  # the harness scaffolds SOUL.md, memory/ and friends here, not in the cwd
     instance = BrowserInstanceConfig(launch_args=browser_launch_args(headless))
     match model_name:
-        case "jev" | "jev-vl-served" | "clm" | "laya" | "laya-served" | "cua" | "omnijev":
+        case "jev" | "jev-vl-served" | "clm" | "laya" | "laya-served" | "cua" | "omnijev" | "onejev":
             if decision_model is None:
                 raise RuntimeError(f"--model {model_name} needs a decision model")
             slot_model = BrowserDecisionModel(
@@ -320,7 +321,7 @@ def parser(spec: BrowserAgentSpec) -> argparse.ArgumentParser:
         "--model",
         choices=BROWSER_MODEL_NAMES,
         required=True,
-        help="who decides each browser step: jev, jev-vl-served, clm or laya-served (over HTTP; jev-vl-served requires a screenshot), laya, cua or omnijev (in process; omnijev reads a screenshot), or llm (the chat model in MODEL_NAME)",
+        help="who decides each browser step: jev, jev-vl-served, clm, laya-served or onejev (over HTTP; jev-vl-served and onejev require a screenshot), laya, cua or omnijev (in process; omnijev reads a screenshot), or llm (the chat model in MODEL_NAME)",
     )
     build.add_argument(
         "--goal", default=spec.goal, required=spec.goal is None, help="the task; the spec's goal when it has one"