Skip to content

feat: update Companion-owned Gateway safely in place#1049

Open
TheAngryPit wants to merge 11 commits into
openclaw:mainfrom
TheAngryPit:codex/gateway-in-place-update-20260722
Open

feat: update Companion-owned Gateway safely in place#1049
TheAngryPit wants to merge 11 commits into
openclaw:mainfrom
TheAngryPit:codex/gateway-in-place-update-20260722

Conversation

@TheAngryPit

@TheAngryPit TheAngryPit commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Closes #901

Credit to @iamalin for originating and detailing the lock-step Gateway proposal in #901. This implementation preserves that product direction and extends it with a recoverable update transaction.

Stacked Dependency

This PR is intentionally stacked on #999 at exact head 10fbccb0f44d15ec1aaeb385b157ccc2738a2310.

#999 owns the current package lifecycle, reload-schema, LKG, and setup/reconnect contracts that this in-place update path consumes. This PR contains the additional update/rollback transaction only. After #999 merges, this PR should be rebased onto main; its displayed diff will shrink to the #901-specific delta.

What Problem This Solves

The Windows Companion can update while its Companion-owned WSL Gateway remains frozen at the version installed during onboarding. That allows the Tray and local Gateway to drift, leaving users on a runtime combination the Companion was not built or tested against.

Why This Change Was Made

The Companion now compares the installed OpenClaw package with its pinned required version and offers a user-confirmed in-place alignment path. Before mutation it exports and verifies an offline rollback point, records durable transaction receipts, revalidates distro ownership, package version, node command policy, and registration state at destructive boundaries, then updates and resynchronizes the existing Companion-owned Gateway without unregistering or replacing it.

Emergency restore remains a separate confirmed operation. It is resumable across destructive phases, fails closed on ambiguous or corrupt receipts, preserves the distro name and identity, restores the registered default user and complete node allowlist, and retains the immutable rollback VHD until post-restore health is proven.

Rollback retention is configurable as one previous version (default), two versions, or indefinite retention, with an optional additive age window.

Non-goals:

  • no npm latest chasing;
  • no unattended update;
  • no onboarding rerun or normal-update distro recreation;
  • no remote Gateway management;
  • no change to the Windows executable updater;
  • no new credentials, permissions, or external authority plane.

User Impact

Users receive a clear prompt when their local Companion-owned Gateway differs from the version required by the installed Companion. They can align it in place while preserving agents, configuration, memory, channels, node identity, and WSL ownership, with an explicit verified rollback path if the update does not recover cleanly.

Evidence

  • Failed-first tests reproduced policy drift, package-version drift, interrupted update receipts, ambiguous restore state, incorrect rollback selection, dropped probe retries, and mutation-boundary races.
  • Two scoped Codex Auto-review cycles found recovery and TOCTOU gaps; accepted in-scope findings were corrected.
  • The final post-review allowlist TOCTOU fix received an independent code review plus focused and full regression proof.
  • No operator runtime, live WSL distro, credentials, or private data were used for validation.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs or instructions
  • Tests or validation
  • Security hardening
  • Chore or infrastructure

Scope

  • Tray or WinUI UX
  • Windows node capability
  • Local MCP or winnode
  • Gateway, connection, or pairing
  • Setup or onboarding
  • Permissions, privacy, or security
  • Tests, CI, or docs

Validation

  • dotnet test tests/OpenClaw.Connection.Tests/OpenClaw.Connection.Tests.csproj: 528/528 passed
  • dotnet test tests/OpenClaw.SetupEngine.Tests/OpenClaw.SetupEngine.Tests.csproj: 766/766 passed
  • dotnet test tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj: 3,177 passed; 31 explicit environment-dependent skips
  • dotnet test tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj: 1,931/1,931 passed
  • .\build.ps1 -NoTrustRepository: Shared, Cli, WinNodeCli, SetupEngine, and WinUI succeeded
  • git diff --check: passed
  • scoped openclaw-autoreview: two bounded cycles completed
  • independent final review: receipt handling closed; final allowlist mutation-boundary race identified and corrected

Real Behavior Proof

  • Environment tested: isolated non-synced Windows worktree; no operator-runtime installation
  • PR head or commit tested: 64eb7b2b8551c2c9872cab0bcecb812a960ea9f9
  • Exact steps or command run: focused failed-first recovery tests, complete affected test projects, official build script, scoped structured review
  • Evidence after fix: exact-version alignment, durable update/restore receipts, rollback retention, ownership checks, node-policy restoration, and retry/resume state machines are covered by current-head tests
  • Observed result: all affected suites and the official build passed
  • Screenshot or artifact links verified? (N/A)
  • Not verified or blocked: fresh GitHub Windows/WSL E2E remains the publication CI gate; no claim of operator-runtime E2E proof

Security Impact

  • New permissions or capabilities? (No)
  • Secrets or tokens handling changed? (No)
  • New or changed network calls? (No; the existing pinned installer path is reused)
  • Command or tool execution surface changed? (Yes)
  • Data access scope changed? (No)
  • If any answer is Yes, explain the risk and mitigation:
    • The Companion can invoke the existing pinned Gateway installer only after explicit user confirmation.
    • A verified offline rollback point and durable receipt are created before mutation.
    • Distro registration, canonical path, machine identity, package version, default user, and complete node policy are revalidated fail-closed at mutation boundaries.
    • Newer installed packages are never silently downgraded.

Compatibility and Migration

  • Backward compatible? (Yes)
  • Config or environment changes? (Yes)
  • Migration needed? (No)
  • If yes, list the exact upgrade steps:
    • Existing settings default to one retained verified rollback point.
    • Existing Companion-owned Gateway records continue to use the same distro, identity, and connection ownership.
    • Current and legacy node command-policy shapes are read and preserved during alignment and restore.

Review Conversations

  • I replied to or resolved every bot review conversation addressed by this PR.
  • I left unresolved only conversations that still need maintainer judgment.

Agent Transcript

Redacted operator/agent implementation timeline
  • The operator confirmed that the feature must update the existing Companion-owned Gateway in place, preserve agents/configuration/memory, and always retain a rollback route.
  • The implementation began from issue Keep the local WSL OpenClawGateway version in lock-step with the Windows Tray app #901's pinned, user-confirmed lock-step design and deliberately rejected distro recreation, npm-latest behavior, silent updates, and custom installer paths.
  • Failed-first validation exposed recovery gaps around cross-record receipts, package-version drift, restore phases, rollback selection, node-policy preservation, concurrent probes, and destructive-boundary races.
  • The update transaction was hardened with verified offline VHD export, durable receipts, exact normalized package attestation, Companion-owned path and machine-identity checks, default-user restoration, complete node-policy round trips, resumable restore phases, and configurable retention.
  • Scoped Auto-review identified additional fail-closed and TOCTOU defects. Each accepted in-scope finding was verified against the real owner path, corrected, and covered by focused tests.
  • After the two-cycle review limit, an independent final review found one remaining allowlist mutation-boundary race. That race was corrected and re-proven with focused tests, the full Connection suite, and the official build.
  • The Keep the local WSL OpenClawGateway version in lock-step with the Windows Tray app #901 commit was then rebased onto fix(setup): validate current OpenClaw package lifecycle #999. Conflict resolution preserved fix(setup): validate current OpenClaw package lifecycle #999's current identityPath, reload-schema, LKG, and setup/reconnect contracts; duplicated old-base tests were removed rather than forked.
  • Fresh CI exposed one deterministic result-ordering mismatch for a reparse-backed rollback root. The operation already failed closed, but unreadable-receipt classification masked the more precise path-collision result. Boundary classification now runs first; focused proof, the complete Connection suite, the official build, and an independent review all passed.
  • Final post-stack local proof: Connection 528/528, SetupEngine 766/766, Shared 3,177 pass with 31 environment-dependent skips, Tray 1,931/1,931, official build green, and diff check clean.
  • Remaining boundary: GitHub's fresh Windows/WSL E2E checks are the next proof layer. No operator runtime, secrets, or private data were used.

@shanselman

Copy link
Copy Markdown
Collaborator

Bot perspective: I think the full-VHDX rollback strategy may be operationally untenable for routine Gateway updates.

Every update runs wsl --export ... --vhd, creating another complete dynamic VHDX. With the default retention of one point, steady-state storage can approach the live distro plus one full rollback copy. During the next update, the live VHDX, retained rollback, and newly exported rollback can temporarily approach three copies of the distro's allocated data. Failed or recovery-pending transactions intentionally retain their copy as well.

I also could not find a free-space preflight before the distro is stopped and export begins. On a machine with limited storage, that means an otherwise routine Gateway package update can consume many gigabytes, fill the system drive, and fail after disrupting the running Gateway.

Could we reconsider whether a complete VHDX is proportionate for every update? A safer layered policy might be:

  1. For normal package updates, use the official updater's staged package rollback plus openclaw backup create --verify for Gateway state.
  2. Reserve full VHDX recovery points for migrations known to change incompatible state or schemas.
  3. If full VHDX remains mandatory, estimate the live VHDX size, require sufficient free space plus a safety margin before stopping WSL, show the expected storage cost in the confirmation UI, and prune eligible old points before export without deleting the latest known-good point.
  4. Surface total rollback storage and provide an explicit cleanup action.

What failure model requires a byte-for-byte distro copy for every package update, and is that benefit worth the potentially 2x steady-state and 3x transient storage cost on ordinary user machines?

@TheAngryPit
TheAngryPit force-pushed the codex/gateway-in-place-update-20260722 branch from 1250e3c to 4e4ad7d Compare July 23, 2026 21:33
@TheAngryPit
TheAngryPit force-pushed the codex/gateway-in-place-update-20260722 branch from 4e4ad7d to 64eb7b2 Compare July 23, 2026 21:49
@clawsweeper

clawsweeper Bot commented Jul 23, 2026

Copy link
Copy Markdown

ClawSweeper status: review started.

I am starting a fresh review of this pull request: feat: update Companion-owned Gateway safely in place This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

TheAngryPit commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, Scott @shanselman . You're right about the storage model.

We optimized for the strongest possible rollback on machines with plenty of disk and made the most expensive recovery path routine. That does not scale to a normal user machine, especially as the Gateway distro grows over time.

We'll rework this into layered rollback:

  • Routine package updates will use the official updater's staged package rollback plus openclaw backup create --verify for Gateway state.
  • Full VHDX recovery points will be reserved for migrations that can change incompatible distro or schema state, or when the user explicitly asks for that level of protection.
  • Any VHDX path will estimate the required space, enforce a safety margin before stopping WSL, show the expected cost in the confirmation UI, and prune eligible old points without deleting the latest known-good point.
  • Settings will show total rollback storage and provide an explicit cleanup action.

For retention, our current direction is to keep the latest verified pre-update recovery point until the next update is healthy, then apply a seven-day grace period by default. Users will be able to choose a longer window or indefinite retention.

Before changing the implementation, we'll verify that the official staged rollback works with the exact pinned package used by the Companion and document precisely what the native backup includes and excludes. If either leaves a recovery gap, we'll map that gap to the specific cases that still need a VHDX instead of requiring one for every update.

This is a better design. Thanks for catching it before we shipped the expensive path as the default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Keep the local WSL OpenClawGateway version in lock-step with the Windows Tray app

3 participants