Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 20 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -301,10 +301,24 @@ to get past. It is a page a person has to answer. lyra names the check and stops
says not to wait and not to look for another way past, to report the task as blocked
(`finish` with `outcome=blocked`) and carry on with what remains.

Nothing solves, waits out or retries a check. Detection reads the frame hosts, the page
title and the first 3000 characters of visible text; an article that quotes a check's
wording further down is not named. A detection in `navigate` is audited as
`challenge_seen`; `read_page` only reports it in its reply.
lyra itself solves, waits out or retries no check. Detection reads the frame hosts, the
page title and the first 3000 characters of visible text; an article that quotes a
check's wording further down is not named. A detection in `navigate` is audited as
`challenge_seen` (with the attempt number); `read_page` only reports it in its reply.

- **The same wall again.** `navigate` counts each check by vendor and site. The reply
carries `attempts` and `max_attempts`; when the same check stops the agent for the
`LYRA_BROWSER_CHALLENGE_MAX_ATTEMPTS`th time (default 3) the status is
`challenge_abandoned` and the hint says to stop and report it as blocked. Coming back
to the same check is not progress, and a site that sees it is right to harden against
the account.
- **An operator's solver.** `LYRA_BROWSER_CAPTCHA_SOLVER=package.module:function` names
a callable lyra hands an `interactive` check to when nobody is watching (never a
`blocked` one, never with a person at the window). It is called as
`solver(info, page)` with `info = {vendor, kind, evidence, url, attempt}`, may be
async, and whatever it does is its own responsibility; lyra ships none. Its answer is
reported (`solver`), never trusted: the page is checked again, and only a page that is
no longer a check answers `ok` with `challenge_solved`. Audited as `challenge_solver`.

### Hover and scroll

Expand Down Expand Up @@ -1020,6 +1034,8 @@ throwaway keyring.
| `LYRA_BROWSER_TRUSTED_SEND_ORIGINS` | — | Sites where *sending* is also pre-approved — `SUBMIT` and `UPLOAD` — for acceptance runs against your own product. Same entry forms, **separate list**: a site in `TRUSTED_ORIGINS` is not send-trusted by being there. Never `PUBLISH` or `DOWNLOAD`; each approval stays single-use, and it steps aside during a takeover. Audited as `consent_channel=trusted_send`. |
| `LYRA_BROWSER_FOLLOW_BYPASS` | on | Claude Code client only: while Claude Code runs in bypassPermissions mode, consent requests are answered without a dialog (`consent_channel=bypass`). The mode arrives from the plugin's PreToolUse hook, never from the model; see docs/CLAUDE_CODE_PLUGIN.md (Bypass mode). |
| `LYRA_BROWSER_DENIED_ORIGINS` | — | Sites refused whatever else says yes: the trusted lists, a live grant and the `autonomous` channel all step aside. Same entry forms. |
| `LYRA_BROWSER_CHALLENGE_MAX_ATTEMPTS` | `3` | How many times the same bot check (vendor at a site) may stop `navigate` before it answers `challenge_abandoned`. |
| `LYRA_BROWSER_CAPTCHA_SOLVER` | — | `module:function` of a solver an interactive bot check is handed to when nobody is watching. None is shipped; see [Bot checks](#bot-checks-challenge). |
| `LYRA_BROWSER_CONSENT_TIMEOUT` | `300` | Seconds a prompt waits for an answer before it counts as a denial (`timed out waiting for the user`). The tool call is held that long, so for an unattended Hermes gateway set it lower — e.g. `60`. |
| `LYRA_BROWSER_RELEASE_GRACE` | `2` | Seconds before a finished action's unused single-use scope is reclaimed. |
| `LYRA_BROWSER_OWNER_IDLE_TIMEOUT` | `900` | Seconds a session may hold the browser without using it. Handing it on closes the window. |
Expand Down
44 changes: 43 additions & 1 deletion docs/UAT.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,8 @@ uat-runs/<run-id>/

- `run` — `status` (`completed`, `incomplete`, `error`), `exit_reason` (`finish`,
`budget_exhausted`, `wall_timeout`, `brain_error`, `browser_error`,
`harness_exit`, `hook_failed`, `no_finish`, `interrupted`), timing.
`harness_exit`, `hook_failed`, `no_finish`, `interrupted`, `challenge_abandoned`),
timing.
- `outcome` — what `finish` said (`reached_goal`, `partial`, `blocked`) or
`unknown` when it never did; steps used against the budget; call count.
- `findings` — severity (`blocker`, `major`, `minor`), URL, step, expected,
Expand Down Expand Up @@ -257,6 +258,47 @@ This is POSIX code (process groups, signals). It was run on Linux only: where th
`/proc` (macOS) it falls back to `killpg` alone for the liveness check, which has not been
run, and Windows is not supported.

## Logins, one-time codes and bot checks

A persona that has to sign in gets the login **by name**. The spec says where each secret
lives, never the secret:

```yaml
credentials:
shop:
origin: https://app.example.com # the one site it may be typed into
username: tester@example.com # or username_env
password_env: SHOP_TEST_PASSWORD # or password_file
totp_file: ~/.config/shop-totp-seed # or totp_env: a base32 seed, as sites show it
```

With `credentials` set the persona is offered `fill_credential(name, field, selector,
submit=false)`, where `field` is `username`, `password` or `totp` (the code of this
moment, RFC 6238, computed when it is typed). The server types the value through
`type_text`, so the same consent, enforcement and audit redaction apply, and only on the
credential's own `origin`: anywhere else the answer is `wrong_origin` and nothing is
typed. A secret that cannot be read is `credential_unavailable`, naming the variable or
file. The value is in no argument, result, trace row or report: the persona, the run
file and the report hold the name and the field.

Every password and TOTP seed of the spec is also held by a scrubber: a page that echoes
it, a result that quotes it, or a POST body `network_capture` keeps is written (and shown
to the model) with `<secret>` in its place, in its plain, URL-encoded and JSON forms.
Usernames and the six-digit codes are not scrubbed. Screenshots are pixels: a password
field shows dots, a code field shows the code of that moment.

Bot checks are named as in [the README](../README.md#bot-checks-challenge). In a run the
third meeting with the same check (`challenge_abandoned`) ends the persona's browsing:
every later browser tool is refused, the policy event `challenge_abandoned` is recorded,
and without a `finish` the report ends `incomplete` with exit reason
`challenge_abandoned`. `target.captcha_solver` (`module:function`) installs a solver for
the run's server; none is shipped.

`scripts/verify_unattended_e2e.py` is the gate: on the demo site (`/login` -> `/2fa` ->
`/account`, fresh canary password and seed per run) a scripted persona signs in with
`fill_credential` in a real headless Chrome, meets `/gate` three times, and every file
of both runs is searched for the password and the seed.

## Hooks

`hooks.before` and `hooks.after` are shell commands. A failing `before` (non-zero)
Expand Down
187 changes: 187 additions & 0 deletions scripts/verify_unattended_e2e.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
#!/usr/bin/env python3
"""Real-browser gate: an unattended sign-in with a password and a one-time code, a bot
check that is given up on, and no secret anywhere in what the run leaves.

The built-in demo site serves ``/login`` -> ``/2fa`` -> ``/account`` (the password and
the TOTP seed are fresh canaries for each run) and ``/gate``, a page that reads as a bot
check. A scripted brain plays the persona through the real runner and a real headless
Chrome:

1. sign-in run: ``fill_credential`` types the email, the password and the current code
(the persona never sees them), the forms are sent, and the run ends on ``/account``.
``network_capture`` keeps the two POSTs whole, the way an analytics check would.
2. challenge run: ``/gate`` three times -> ``challenge``, ``challenge``,
``challenge_abandoned``; the next browser call is refused, and the report ends
``challenge_abandoned``.

Then every file of both runs (trace, events, audit, report, run file, network and console
logs, observations, harness and capture metadata) is searched for the password and the
seed: there must be none.

python scripts/verify_unattended_e2e.py [--keep]
"""

from __future__ import annotations

import argparse
import asyncio
import base64
import os
import secrets
import shutil
import sys
import tempfile
from pathlib import Path

REPO = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "src"))


def expect(condition: bool, message: str, detail: object = "") -> None:
if not condition:
raise AssertionError(f"{message}: {detail}")
print(f"PASS {message}")


def spec_for(base: str, out: Path, seed_file: Path, script: list[dict]):
from lyra_browser.uat.spec import RunSpec

origin = base.rstrip("/")
return RunSpec.model_validate(
{
"persona": {
"id": "signin",
"name": "Returning customer",
"entry_url": f"{base}login",
"goal": "sign in and open the account page",
"step_budget": 12,
},
"target": {
"trusted_origins": [origin],
"trusted_send_origins": [origin],
"network_capture": ["/login", "/2fa"],
},
"brain": {"backend": "scripted", "script": script},
"credentials": {
"demo": {
"origin": origin,
"username": "demo@example.com",
"password_env": "LYRA_GATE_DEMO_PASSWORD",
"totp_file": str(seed_file),
}
},
"out_dir": str(out),
"headless": True,
}
)


def fill(field: str, selector: str, **extra) -> dict:
return {
"tool": "fill_credential",
"args": {"name": "demo", "field": field, "selector": selector, **extra},
"expect": "ok",
}


async def signin_run(base: str, out: Path, seed_file: Path):
from lyra_browser.uat.runner import run_spec

script = [
{"tool": "navigate", "args": {"url": f"{base}login"}, "expect": "ok"},
fill("username", "#user"),
fill("password", "#pass"),
{"tool": "click", "args": {"selector": "#signin", "submits": True}, "expect": "ok"},
{"tool": "wait_for", "args": {"text": "authenticator", "timeout_ms": 8000}},
fill("totp", "#code", submit=True),
{"tool": "wait_for", "args": {"text": "Signed in as", "timeout_ms": 8000}},
{"tool": "get_url", "args": {}},
{"tool": "finish", "args": {"outcome": "reached_goal", "summary": "Signed in."}},
]
report = await run_spec(spec_for(base, out, seed_file, script))
print(report.one_line())
expect(report.run.exit_reason == "finish", "sign-in run finished", report.brain.error)
fills = [s for s in report.trace if s.tool == "fill_credential"]
expect([s.status for s in fills] == ["ok", "ok", "ok"], "three fields typed", fills)
expect(
all(set(s.args) <= {"name", "field", "selector", "submit"} for s in fills),
"the trace holds the credential's name and field only",
[s.args for s in fills],
)
last = report.trace[-2]
expect(last.url_after.endswith("/account"), "the run ended signed in on /account", last)
return report


async def challenge_run(base: str, out: Path, seed_file: Path):
from lyra_browser.uat.runner import run_spec

script = [{"tool": "navigate", "args": {"url": f"{base}login"}, "expect": "ok"}]
script += [{"tool": "navigate", "args": {"url": f"{base}gate"}} for _ in range(3)]
script += [{"tool": "read_page", "args": {}}, {"tool": "note", "args": {"text": "walled"}}]
report = await run_spec(spec_for(base, out, seed_file, script))
print(report.one_line())
statuses = [s.status for s in report.trace]
expect(
statuses
== ["ok", "challenge", "challenge", "challenge_abandoned", "challenge_abandoned", "ok"],
"the same check: named twice, abandoned the third time, then refused",
statuses,
)
expect(
report.run.exit_reason == "challenge_abandoned" and report.run.status == "incomplete",
"the run ends as challenge_abandoned",
report.run,
)
return report


def leaks(run_dir: Path, needles: list[str]) -> list[str]:
found = []
for path in run_dir.rglob("*"):
if not path.is_file():
continue
data = path.read_bytes()
found += [f"{path}: {n[:4]}…" for n in needles if n.encode() in data]
return found


async def main(keep: bool) -> None:
from lyra_browser.uat import demo_site

password = "gate-" + secrets.token_urlsafe(12)
seed = base64.b32encode(secrets.token_bytes(20)).decode().rstrip("=")
demo_site.Handler.password = password
demo_site.Handler.totp_seed = seed
os.environ["LYRA_GATE_DEMO_PASSWORD"] = password
server, base, _alt = demo_site.start()
out = Path(tempfile.mkdtemp(prefix="lyra-unattended-e2e-"))
seed_file = out / "seed.txt"
seed_file.write_text(seed)
try:
print(f"demo site {base}, runs in {out}")
signed = await signin_run(base, out / "runs", seed_file)
walled = await challenge_run(base, out / "runs", seed_file)
for report in (signed, walled):
run_dir = Path(report.artifacts.dir)
found = leaks(run_dir, [password, seed])
expect(not found, f"no password or seed in any file of {run_dir.name}", found)
captured = (Path(signed.artifacts.dir) / "network.jsonl").read_text(encoding="utf-8")
expect(
"/login" in captured and "<secret>" in captured,
"the captured sign-in POST is kept, with the password replaced",
)
finally:
server.shutdown()
if keep:
print(f"kept {out}")
else:
shutil.rmtree(out, ignore_errors=True)


if __name__ == "__main__":
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--keep", action="store_true", help="Keep the run directories.")
args = ap.parse_args()
asyncio.run(main(args.keep))
print("ALL PASS")
70 changes: 68 additions & 2 deletions src/lyra_browser/challenge.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,27 @@

``classify`` is the judgement and takes plain values, so it is tested without a
browser. ``observe`` collects those values from a page. Nothing here solves, waits out
or retries a challenge.
or retries a challenge by itself.

Two things sit around that judgement:

- ``ChallengeLedger`` counts how often the same check (vendor, at an origin) has stopped
the agent. Past ``Config.challenge_max_attempts`` the answer is ``challenge_abandoned``:
coming back to the same wall is not progress, and a site that sees it is right to
harden its stance against the account.
- A *solver hook* (``Config.captcha_solver``, ``"module:function"``) the operator may
install. lyra ships none. When one is configured and nobody is attending, an
interactive check is handed to it once per attempt; whatever it does is its own
responsibility (a human-in-the-loop service, a test double), and the page is observed
again afterwards to see whether the check is gone.
"""

from __future__ import annotations

from dataclasses import asdict, dataclass
import importlib
import inspect
from collections.abc import Awaitable, Callable
from dataclasses import asdict, dataclass, field
from typing import Any

INTERACTIVE = "interactive"
Expand Down Expand Up @@ -124,6 +139,57 @@ async def observe(page: Any, http_status: int | None = None) -> Challenge | None
)


@dataclass
class ChallengeLedger:
"""How many times each check (vendor at an origin) has been met in this session."""

seen: dict[tuple[str, str], int] = field(default_factory=dict)

def record(self, origin: str, challenge: Challenge) -> int:
key = (origin, challenge.vendor)
self.seen[key] = self.seen.get(key, 0) + 1
return self.seen[key]

def clear(self, origin: str, challenge: Challenge) -> None:
self.seen.pop((origin, challenge.vendor), None)


Solver = Callable[[dict[str, Any], Any], Awaitable[Any] | Any]
"""``solver(info, page)``: ``info`` is the challenge with ``url`` and ``attempt``;
``page`` is the live page. Its return value is reported, never trusted: the page is
observed again to see whether the check is gone."""


def load_solver(spec: str) -> Solver:
"""``"package.module:function"`` to the callable it names."""
module_name, sep, attr = spec.partition(":")
if not sep or not module_name or not attr:
raise ValueError(f"captcha_solver must be 'module:function', got {spec!r}")
target = getattr(importlib.import_module(module_name), attr)
if not callable(target):
raise ValueError(f"captcha_solver {spec!r} is not callable")
return target


async def run_solver(solver: Solver, info: dict[str, Any], page: Any) -> str:
"""Call the solver; what it said, as a short string for the envelope and the audit."""
try:
answer = solver(info, page)
if inspect.isawaitable(answer):
answer = await answer
except Exception as exc: # noqa: BLE001 - a broken solver is reported, not raised
return f"error: {type(exc).__name__}: {exc}"[:200]
return str(answer)[:200]


def abandon_hint(challenge: Challenge, attempts: int) -> str:
return (
f"The same {challenge.vendor} check has stopped this {attempts} times. Do not "
"try again and do not look for another way past it: report it as blocked "
"(finish with outcome=blocked) and continue with what remains of the task."
)


def hint_for(challenge: Challenge, attended: bool) -> str:
"""What the agent should do next, depending on whether a person is watching."""
if challenge.kind == BLOCKED:
Expand Down
10 changes: 10 additions & 0 deletions src/lyra_browser/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,12 @@ class Config:
# denial. Five minutes suits a person at a screen; a gateway nobody is
# watching wants it short (docs/HERMES_INTEGRATION.md).
consent_timeout_s: float = 300.0
# How many times the same bot check (vendor at an origin) may stop the agent before
# navigate answers ``challenge_abandoned`` instead of ``challenge``.
challenge_max_attempts: int = 3
# ``"module:function"`` of an operator-installed solver an interactive bot check is
# handed to when nobody is attending. None: checks are named and handed over only.
captcha_solver: str | None = None
# Sites the operator has pre-approved for NAVIGATE (and the INTERACT it
# implies), so one they always use is not asked about again. Entry forms:
# see parse_trusted_origins. An answer, not a consent channel, and never a
Expand Down Expand Up @@ -674,6 +680,10 @@ def from_env(cls, client: str | None = None) -> Config:
# substitutes an empty string, and that must not read as "off".
follow_bypass=_as_optional_bool(_env("LYRA_BROWSER_FOLLOW_BYPASS")) is not False,
consent_timeout_s=_as_float(_env("LYRA_BROWSER_CONSENT_TIMEOUT"), default=300.0),
challenge_max_attempts=max(
1, _as_int(_env("LYRA_BROWSER_CHALLENGE_MAX_ATTEMPTS"), default=3)
),
captcha_solver=_env("LYRA_BROWSER_CAPTCHA_SOLVER") or None,
trusted_origins=parse_trusted_origins(_env("LYRA_BROWSER_TRUSTED_ORIGINS")).entries,
trusted_send_origins=parse_trusted_origins(
_env("LYRA_BROWSER_TRUSTED_SEND_ORIGINS")
Expand Down
Loading
Loading