Skip to content

[Feature]: atomic ocx update command with daemon drain, schema validation, and multi-client sync #4173

Description

@rrmlima

Problem to solve

When operators or automated agent harnesses upgrade OpenCodex via npm install -g @bitkyc08/opencodex@latest:

  1. Active In-Flight Runtime Rejection (packageTreeIntegrity 503): The running Bun proxy process detects filesystem changes under its active package directory (src/server/index.ts:1088 packageTreeIntegrity.status()), immediately throwing 503 Server_error: OpenCodex package files changed while this proxy was running; restart OpenCodex before retrying. on all /v1/*, /healthz, and /readyz endpoints.
  2. Desynchronized Client State: The upgrade updates the package files but does not coordinate the dependent client topology:
    • On-disk opencodex-catalog.json and client models caches (models_cache.json) lag behind the new binary until an explicit sync command runs.
    • Long-lived Codex CLI app-server processes retain outdated protocol handles and stale catalog state.
    • Exported client targets (e.g. ~/.commandcode/providers.json via PR feat(clients): add native Command Code integration and catalog sync #3833) do not automatically converge on the new version's schema or models.
  3. Manual and Error-Prone Orchestration: Operators and automation scripts are forced to compose external wrapper shell scripts (stopping the service, updating via npm, validating JSON schema, invoking ocx sync --restart-codex, and polling /readyz). If any step is executed out of sequence or by a non-interactive runner, the deployment remains in an unready 503 or deadlocked state.

Proposed solution

Implement a first-class, atomic CLI command:

ocx update [version] [--channel <stable|preview>] [--restart-codex] [--dry-run] [--json]

Architecture & Lifecycle Stages:

  1. Pre-flight Schema & Readiness Guard:
    • Execute internal config validate in memory against active configuration before initiating download or termination. If configuration is invalid, abort immediately without altering running processes or files.
  2. Managed Daemon Drain & Teardown:
    • Detect whether opencodex-proxy is running as a systemd user unit, launchd daemon, or standalone background process.
    • Signal a graceful drain barrier (DRAIN state) to complete active turns while rejecting new ingress with Retry-After.
    • Terminate the listener cleanly before filesystem mutation begins, eliminating the packageTreeIntegrity race condition entirely.
  3. Transactional Package Upgrade:
    • Execute package acquisition via standard global npm/bun registry resolvers to the target version (defaulting to latest published tag on the current channel).
  4. Post-Install Client & Process Convergence:
    • Execute internal catalog regeneration (sync-cache) atomically with lock acquisition.
    • Terminate stale app-server child processes cleanly using afterCatalogWriteHandleAppServers.
    • Trigger catalog-refresh across all registered client exports (mcode, pi, raycast, and commandcode).
  5. Verified Service Bootstrap & Health Handshake:
    • Restart the daemon/unit.
    • Poll local loopback /readyz with bounded timeout until HTTP 200 {"status":"ready"} is observed before yielding control back to the terminal.
    • Output structured JSON status when --json is specified.

Alternatives considered

  • External wrapper shell scripts (apply-fix.sh / cron / wrapper CLIs): Currently used in production deployments as a local workaround. Highly brittle across platforms (Linux systemd vs macOS launchd vs Windows services) and leaves non-expert users and agent harnesses vulnerable to runtime desyncs.
  • In-process auto-reload on package tree change: Attempting to hot-reload ES modules inside the running Bun process. Rejected due to state corruption risks in SQLite/memory caches, active WebSocket handles, and crypto sessions.

Additional context

  • Upstream files involved:
    • src/cli/dispatch.ts and src/cli/registry.ts (CLI command registration and option parsing)
    • src/server/index.ts (packageTreeIntegrity check and graceful drain coordination)
    • src/service.ts (daemon management across platforms)
    • src/integrations/catalog-refresh.ts and src/codex/catalog/sync.ts (multi-client synchronization)

Activity

  1. Ingwannu commented on Sep 10, 2026

    @Ingwannu
    Owner

    There is already a first-class updater on current dev, so I would scope this as improving its guarantees rather than adding a second updater.

    At c15a98c:

    • src/cli/help.ts and src/cli/registry.ts expose ocx update [--tag latest|preview].
    • For npm installations, bin/ocx.mjs intercepts the command before Bun starts. src/update/transactional-install.mjs stages and verifies a sibling package tree, swaps it with a backup, and includes rollback/recovery-marker handling.
    • The Bun/source fallback in src/update/index.ts is a different path. It checks stop/liveness and pending teardown before replacement, and preserves service/tray handling. It should not be described as having identical transactional guarantees to the npm launcher.

    A direct npm install -g bypasses that coordinated updater. The package-tree 503 is an intentional mixed-version runtime guard; removing it or hot-reloading modules is not the fix.

    The useful remaining proposal is a bounded, explicitly specified update transaction: preflight/dry-run reporting, per-installer failure and rollback semantics, and a verified readiness outcome. Please distinguish a failure observed with ocx update from a direct global npm replacement, and provide OS, install method, version transition, and redacted updater output if the existing command failed.

    Client convergence needs a separate consent/ownership boundary: catalog writes and termination of active Codex app-server sessions cannot be treated as harmless default cleanup. An explicit restart option must target only owned processes and disclose interruption; it must not sweep unrelated sessions or rewrite unmanaged client configurations. I am leaving this open as an enhancement, not marking the entire requested lifecycle as already implemented.

  2. lidge-jun commented on Sep 10, 2026

    @lidge-jun
    Owner

    리뷰 · 우선순위 36 / 80

    이 이슈는 npm install -g로 OpenCodex를 올릴 때 생기는 운영 고통을 한 방에 묶는 큰 기능 요청입니다. 증상은 세 갈래입니다. (1) 패키지 파일이 바뀌면 실행 중 프록시가 packageTreeIntegrity로 /v1/*·health에 503을 냅니다. (2) 카탈로그·models_cache·Codex app-server·외부 export(commandcode 등)가 새 바이너리와 어긋납니다. (3) 운영자/에이전트가 stop → npm → validate → sync → readyz를 손으로 이어 붙이다가 순서가 틀리면 503이나 반쯤 죽은 상태에 남습니다. 제안은 ocx update에 채널·restart-codex·dry-run·json을 붙여 preflight 검증, 데몬 drain, 패키지 교체, 카탈로그/클라이언트 수렴, /readyz 핸드셰이크까지 atomic하게 묶자는 것입니다.

    지금 dev를 보면 조각은 이미 많습니다. 최상위 ocx update [--tag latest|preview]가 src/cli/registry.ts에 있고, src/cli/system-command.ts에는 system update check|run|status와 /api/update/* 경로도 있습니다. src/integrations/catalog-refresh.ts, sync, ocx system sync, ocx system codex-restart, 서비스 매니저(src/service.ts), 그리고 최근 올라온 #4152 서비스 가드·#4164 launchctl bootout이 Lane A 정지/서비스 경로를 건드리고 있습니다. 스냅샷도 “unrelated PR에서 ocx service/start/stop/restart나 launchctl/systemctl을 돌리지 말라”고 적혀 있습니다. 즉 “업데이트 오케스트레이션이 전혀 없다”가 아니라, “이미 있는 update/stop/sync/service 조각을 더 큰 트랜잭션으로 합치자”에 가깝습니다. 그 합치기는 가치가 있지만, 범위가 CLI·server integrity·service·catalog-refresh·멀티 클라이언트 export를 한 PR에 넣으면 위험합니다.

    packageTreeIntegrity 503은 버그가 아니라 보호 장치입니다. 파일을 갈아끼운 채 옛 프로세스가 요청을 받는 걸 막습니다. 올바른 순서는 “먼저 리스너를 안전하게 내린 다음 파일을 바꾸고, 새 프로세스를 올린 뒤 ready를 확인”입니다. 그 순서를 CLI가 대신 해 주는 방향은 맞습니다. 다만 Windows Task Scheduler / launchd / systemd / standalone마다 drain·respawn 규칙이 이미 갈라져 있고(#4169·#4170이 그 메시지 경로만으로도 복잡함을 보여 줍니다), “atomic update” 한 이슈에 그 전부를 넣으면 리뷰·회귀 비용이 큽니다. types/config 분할과는 무관하지만, 서비스 레인과는 정면으로 겹칩니다.

    라인 문제:

    src/server/index.ts · packageTreeIntegrity - 이 가드를 약화하거나 hot-reload로 우회하는 설계는 이슈 본문도 거절했습니다. 유지한 채 “업데이트 전에 프로세스를 내린다”가 맞습니다.

    src/cli/registry.ts · ocx update - 이미 존재합니다. 새 하위명령처럼 쓰기보다, 현행 update의 수명주기를 drain→install→sync→ready로 확장하는 설계 문서를 먼저 쓰는 편이 중복을 줄입니다.

    src/service.ts / stop 경로 - 최근 #4152·#4164와 #4169/#4170이 이 구역을 건드리는 중입니다. 큰 update 오케스트레이션 PR이 같은 파일을 또 크게 바꾸면 충돌·회귀 위험이 큽니다.

    catalog-refresh / app-server - post-install 수렴은 가치가 큽니다. 다만 update 본체와 분리해 “update 성공 훅” 또는 sync after-update 단계로 쪼개는 편이 리뷰하기 쉽습니다.

    메인테이너의 판단이 필요한 지점

    • 기존 ocx update / system update를 확장할지, 새 이름을 만들지
    • Windows/macOS/Linux 서비스 백엔드를 한 PR에 넣을지, 플랫폼별 슬라이스로 나눌지
    • catalog-refresh·app-server 재시작을 update 기본에 넣을지, 플래그로 옵트인할지
    • Lane A 서비스 안정화(fix(stop): name the real refusal cause instead of asserting ownership #4170 등)가 끝난 뒤에야 이 오케스트레이션을 받을지

    너의 추천
    이슈는 열어 두되 epic/needs-design으로 쪼개세요. 바로 받을 만한 첫 슬라이스는 “현행 ocx update가 패키지를 바꾸기 전에 자기 홈의 프록시를 정상 stop/drain 하고, 끝난 뒤 /readyz를 기다린다” 정도입니다. sync·catalog-refresh·restart-codex는 다음 슬라이스입니다. 한 PR에 atomic 전체를 넣지 마세요. 지금 dev의 서비스 레인과 겹치므로, #4170 같은 stop 정확도 수정이 먼저 가는 편이 안전합니다. types/config 분할 close-don't-rebase 대상은 아닙니다.

    이 댓글은 grok-bot이 작성했습니다

  3. rrmlima commented on Sep 16, 2026

    @rrmlima
    ContributorAuthor

    Thanks @Ingwannu and @lidge-jun — agreed on the scoping: improve the existing updater's guarantees, don't add a second updater, and keep the packageTreeIntegrity 503 as the intentional mixed-version guard it is.

    I run a 24/7 deployment (Ubuntu 24.04, systemd user service, npm install) and hit exactly the failure modes this issue describes. Two concrete inputs for the updater spec, from production:

    1. Multi-prefix reality: the transactional npm path needs to know which install owns the running proxy. On this host, npm install -g from an interactive shell resolves to the nvm prefix (~/.nvm/versions/node/v24.x) while the systemd service executes /usr/local/lib/node_modules/.... A plain npm install -g @bitkyc08/opencodex@latest therefore updated one copy while the service kept running another (observed: nvm at 2.52.0 while /usr/local sat at 2.54.0 — the running binary never moved). Suggestion: preflight should enumerate all global installs of the package (npm ls -g across common prefixes / which -a resolution), refuse-or-warn when the prefix that owns the running proxy isn't the one being replaced, and record the resolved prefix in the dry-run report.

    2. Field-validated update sequence that never hit a 503 or stale state (what I'd want ocx update to guarantee natively, per Ingwannu's "bounded, explicitly specified update transaction"):

    1. Schema-validate config.json before stopping anything — an invalid config must leave the running service untouched.
    2. Drain: stop the proxy only after validation passes (stopping first is what keeps packageTreeIntegrity from ever firing mid-upgrade).
    3. Replace the package (transactional launcher path).
    4. Restart + /readyz handshake with a bounded timeout, and restore the service even if the install fails (the agent's turn resumes against a coherent tree instead of a stale 503).
    5. Catalog convergence as a post-ready, opt-in step: schema-backed sync, sync-cache, and client refreshes — and a uniqueness guard on the written catalog, since 2.56.0 could emit duplicate slugs under modelAliases configs (see [Bug]: ocx sync emits duplicate catalog slugs when modelAliases differs from canonical form #4730).

    What I have not observed: a failure inside ocx update itself on current dev (my incidents were all direct npm install -g, which I agree is out of the updater's contract). So I can't report an updater bug — these are the guarantees I'd volunteer to test. If a preview build of the transactional installer needs a real multi-prefix, systemd-user-service production host to validate against, this box is a willing testbed and I'll report exact outputs.

    Happy to split any of this into its own issue if you'd rather track preflight multi-prefix detection separately.

  4. added
    priority: P3Low: new provider/client integration, large or experimental feature (>2000 LOC or >50 files), RFC/ro
    on Sep 24, 2026
  5. lidge-jun commented on Sep 27, 2026

    @lidge-jun
    Owner

    Release train 4 triage (dev 24b2f39b, T4-I-4173): Keeping this open. ocx update and a stop-before-replace launcher exist, but they do not establish the proposed preflight/schema check, verified readiness/rollback transaction and client convergence across install modes. Next: define these guarantees per installer and service mode, then add failure-path regressions rather than a second updater.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    cliCLI, config inject, packaging flagsenhancementNew feature or requestpriority: P3Low: new provider/client integration, large or experimental feature (>2000 LOC or >50 files), RFC/ro

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions