Skip to content

[Bug]: 0.4.0 — observation capture wedges the session queue ("still settling" forever, reload deadlock) and action context expires before the next call #527

Open

Description

@SAXEM1997

Prerequisites

BrowserSkill Version or Commit

0.4.0 (bsk 0.4.0, daemon protocol 1.7, extension 0.4.0, DSH plugin @wxg-prc-cpg/browser-skill-dsh-plugin@0.4.0)

Operating System

Linux (x86_64) — daemon/CLI run inside a container

OS Details and Architecture

Debian-based container x86_64; browsers run on a separate Windows host reached through the authenticated remote WSS gateway.

Installation Method

Prebuilt binary release from GitHub Releases

Browser and Browser Extension Version

Edge 156.0.0.0 + extension 0.4.0; Chrome 156.0.0.0 + extension 0.4.0 (two separate instances, both remote)

Exact Command or API Call

DSH tool calls (plugin 0.4.0):

bsk_session({action:"start", browser:"<instance>", noFocus:false, url:"https://example.com"})
bsk_session({action:"sync", session:"<sid>"})        # immediately after start
bsk_interact({action:"click", session:"<sid>", target:"@e2", contextId:"<ctx>"})

and the equivalent CLI calls (bsk session start --focus, bsk cooperate sync).

Minimal Reproduction Steps

Defect A — observation capture wedges the per-session command queue

  1. With the plugin default observationEnabled: true, start a session against a remote browser (foreground or background) with a URL, then call sync immediately.
  2. sync fails with Previous operation is still settling; this action was not dispatched.
  3. From then on every command on that session fails:
    • bsk tab list --session <sid> → session already has an unfinished command
    • bsk reload → This action requires the context returned by the latest cooperate sync (but sync can never succeed)
      → hard deadlock; only session stop works.
  4. Control A: start the same browser session through the CLI directly (no plugin observer) → navigate + cooperate sync succeed immediately.
  5. Control B: set the plugin config observationEnabled: false and restart → the exact same plugin flow (start → immediate sync) succeeds on the first attempt.

This reproduces on both Edge and Chrome, foreground and background, so it is not a browser/window-focus issue.

Defect B — action context expires before the next call, and "unchanged" sync does not renew it

  1. sync returns state.target.expires; measured TTL ≈ 15–20 s.
  2. A second sync returns kind: "unchanged" with the same expires value (no renewal).
  3. A click issued after expiry fails with Page interaction coverage changed; wait and synchronize.
  4. The click has nevertheless been executed (the page had already navigated when checked afterwards) — the error does not mean "not dispatched".

Expected Behavior

  • Observation thumbnails must not block the session command queue; a hung capture should time out and release the queue.
  • A successful sync should renew the action context (or expose a way to extend it), and an expired context should be reported as such, not as "page coverage changed".
  • Errors should state whether the action was dispatched.

Actual Behavior

  • A stuck observation screenshot holds the single in-flight command slot forever; sync, tab, navigate, reload all queue behind it, and reload (the suggested recovery) itself requires a sync context → deadlock.
  • Action context TTL is shorter than a typical agent step latency, and sync with kind: "unchanged" keeps the stale expires.
  • coverage changed is returned even when the click completed.

Relevant Logs / Diagnostics Output

Error: bsk cooperate sync failed: Previous operation is still settling; this action was not dispatched
(hint: confirm the tab is still in a loaded state and retry; reloading the tab usually resets a stuck DevTools session)

error: previous session command is still running
hint: wait for the current command to finish, cancel it with Ctrl-C, or stop/restart the session
details: session already has an unfinished command

Error: bsk reload failed: This action requires the context returned by the latest cooperate sync
(hint: use `bsk cooperate wait --session SID`, then `bsk cooperate sync --session SID`; replan ...)

Error: bsk click failed: Page interaction coverage changed; wait and synchronize

# sync payload excerpt (second consecutive sync, note unchanged expires)
"kind":"unchanged","previous_context_id":"<redacted>",
"state":{"mode":"ready","revision":20,"target":{"expires":1793348168226,"x":406,"y":504.3}}

Is this a regression?

Yes (please specify the last known working version below)

Last Known Working Version (if regression)

Not observed on 0.3.3 with the same host topology, but 0.3.3 predates automatic shared control and automatic observation, so the comparison is not apples-to-apples.

Additional Context

Defect A disappears completely when the plugin's observationEnabled is set to false, which points at the periodic screenshot --session capture (ObservationService) as the command that hangs and blocks the queue. Suggested fixes: bound the capture with a timeout and abort/withdraw it on failure, never let observation occupy the action lane, and make reload usable while the queue is blocked (or offer a queue reset).
For Defect B, either renew the target on every successful sync or make the TTL explicit/configurable; and separate the "expired context" error from "page content actually changed".

Activity

  1. tom613951 commented on Oct 31, 2026

    @tom613951

    Additional reproduction: kind never converges from baseline on pages with a sub-10-second live region

    Reporting a narrower trigger than a TTL expiry, measured on bsk 0.4.0 / extension 0.4.0 / protocol 1.7, Windows + Chrome 156.0.0.0, local browser (no remote WSS gateway). Both defects reproduce.

    Environment

    daemon_version     0.4.0
    protocol_version   1.7
    extension_version  0.4.0
    chrome             156.0.0.0
    host               Windows, local browser (no WSS gateway)
    

    observationEnabled was true at first, then set to false in the DSH plugin config. Toggling it did not change CLI-side behaviour, which is consistent with the capture path living in the extension rather than the plugin.

    Controlled A/B on one session

    Two pages, one daemon, one session (zhwp), bsk cooperate sync repeated 6× at 1.2 s intervals. Identical harness both times.

    A — https://example.com (static):

     i   rev       kind      ms  delta
     0     9   baseline      90
     1    11  unchanged      19  +2
     2    13  unchanged      19  +2
     3    15  unchanged      20  +2
     4    17  unchanged      19  +2
     5    19  unchanged      19  +2
    

    baseline on the first capture only, then unchanged. Actions succeed:

    0: OK value=4
    1: OK value=4
    ...
    5: OK value=4
    --- ok=6 fail=0
    

    B — https://bbs.huaweicloud.com/blogs/article (same session, after navigate):

     i   rev       kind      ms  delta
     0    91   baseline    1133
     1    95   baseline    1128  +4
     2    99   baseline    1131  +4
     3   104   baseline    1104  +5
     4   108   baseline    1124  +4
     5   112   baseline    1080  +4
    

    baseline on every capture, never unchanged. Every action refused:

    0: REFUSED The action's observed content changed; s
    ...
    --- ok=0 fail=6
    

    Two stable differences: kind never leaves baseline, and sync costs ~1100 ms instead of ~20 ms — a ~55× increase, consistent with a full VOM capture each call rather than an incremental comparison.

    What the page does

    The editor renders a live autosave counter in its footer — "21 秒后保存", "4 秒后保存", counting down and resetting on a sub-10-second cadence. That counter sits inside the editor's form container. PR #497 deliberately keeps form values, checked state, modals and the action's own region exact, and the <main>/landmark relaxation excludes positioned panels, so a counter inside the form container is never learnable as volatile. The result is that this page cannot reach unchanged at all.

    --depends-on does not help, because the refusal happens on the whole-page capture before target scoping:

    $ bsk evaluate --session <sid> --context <ctx> \
        --depends-on "input.write-blog-blog-title" --json "(()=>1)()"
    {
      "code": "permission_denied",
      "message": "The action's observed content changed; synchronize and replan"
    }
    

    Session state also degrades

    An earlier session (opyq) showed unchanged on example.com at first, then began returning baseline on the same unchanged page after roughly 20 minutes of repeated sync/refuse cycles — after which example.com was refused 6/6 as well. A brand-new session restored unchanged immediately. So there appear to be two independent ways to lose convergence:

    1. a page with an unlearnable live region inside a protected container, and
    2. accumulated session state, which stops converging even on a static page.

    Defect A, as observed

    After several refusals the session wedges, matching your report:

    $ bsk cooperate sync --session <sid>
    error: shared page control or observation changed
    details: Could not establish connection. Receiving end does not exist.
    

    The details line is a content-script messaging failure, not a cooperation refusal. bsk reload cannot recover it because reload needs a context from the sync that just failed. bsk session stop --all is the only exit. Reproduced on about:blank and on example.com after a daemon restart; cleared by a fresh session followed by bsk cooperate wait.

    Not reproduced

    state in this build carries only editing, mode, revision — there is no target and no target.expires, so I could not reproduce the specific expires-not-renewed path. Everything above is the baseline-never-settles path.

    Suggested narrowing

    The case worth isolating is live text inside a form/list/dialog/panel versus live text in a plain container. The first appears to guarantee a permanent baseline. It may deserve either an escape hatch, or a documented note that such pages require a --depends-on naming a container outside the live region — the current docs advise omitting --depends-on when unsure, which is exactly the state that cannot recover here.

    Happy to re-test against main once #529 ships in a release and report whether the baseline case persists.

  2. MoyuP commented on Oct 31, 2026

    @MoyuP

    DSH plugin data point: the "still settling" refusal is not recoverable from DSH either

    macOS arm64, local Chrome, bsk/extension/DSH plugin 0.4.0, DSH desktop 0.2.1-rc.5.

    Sequence: bsk_session start OK → bsk_page navigate OK (reached: load) → first bsk_session sync fails 3/3 with "Previous operation is still settling; this action was not dispatched". settle: true fails identically. Sleeping ~5 s and syncing once succeeds; screenshot and stop are clean afterwards.

    0.4.0 mechanism (from source):

    • cooperation sync bypasses the plugin's per-session queue, while the observation thumbnail screenshot runs inside it. Every tool call first calls acquireForeground, which kills the running observation:... screenshot. The sync then reaches the extension while it is still settling the cancelled capture, so the dispatcher returns cdp_failed (apps/extension/src/tools/dispatcher.ts, "Previous operation is still settling; this action was not dispatched").
    • Retrying makes it worse: on release, ObservationService.acquireForeground schedules a new thumbnail immediately (scheduleCapture(..., 0)), which the next retry preempts again. That matches 3/3 immediate retries failing and one ~5 s wait succeeding.
    • --settle cannot help: crate::cli::cooperate::settle::recoverable only retries permission_denied + cooperation_sync_required + effect_state: none; this refusal is cdp_failed / dispatched: false, so settle returns on the first attempt without ever calling cooperate wait.
    • The suggested recovery is unreachable: reload is a shared action and requires the latest sync context, so sync-fail → no context → reload refused. cooperate wait (DSH: bsk_session action:"wait-user") is the closest native equivalent, but it is named and described for hand-back, not for this recovery.

    Even on main (1b395fd) the settle predicate, the cdp_failed hint and the context gate are unchanged, so #529 fixes the ordering trigger but not this recovery path.

    Suggestions:

    1. In settle::recoverable, also retry the transient cdp_failed + data.dispatched === false "still settling" refusal with a bounded wait, or surface it as session_busy so the DSH plugin's runWithSessionBusyRetry applies.
    2. Do not suggest reload for this condition (it is context-gated). Suggest waiting for the session to settle and syncing again, and/or expose an explicit wait/recovery action in DSH instead of only wait-user.
    3. Document that 0.4.0 thumbnail captures can collide with the first sync, and that "brief wait, then sync" is the safe retry.

    Happy to re-test against the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions