Skip to content

feat(codex): pull an authenticated remote catalog into local Codex state #3729

Closed
Closed
feat(codex): pull an authenticated remote catalog into local Codex state#3729
Labels
catalogModel catalog, slugs, visibility, routed entriesenhancementNew feature or request

Description

@rrmlima
rrmlima
opened on Sep 6, 2026

Area

Catalog / models

What are you trying to accomplish?

I need to keep a Codex CLI or Desktop client synchronized with the complete model catalog generated by a remotely hosted OpenCodex instance.

OpenCodex already exposes the persisted catalog through the authenticated GET|HEAD /v1/catalog data-plane endpoint. The missing workflow is safely materializing that snapshot into the remote client's local CODEX_HOME, rebuilding models_cache.json, and reporting when a running Codex app-server or Desktop process still holds the previous catalog in memory.

This would make centrally hosted OpenCodex deployments usable without maintaining deployment-specific web roots, cron jobs, PowerShell downloaders, or ad hoc cache-rebuild scripts.

What prevents this today?

model_catalog_json accepts a local filesystem path, not a URL. ocx sync builds a catalog from locally configured providers, while ocx export --client generates client-specific configuration; neither consumes a complete catalog from another OpenCodex server.

Operators currently need to implement the entire client-side lifecycle themselves:

  1. authenticate to /v1/catalog;
  2. download into a same-directory temporary file;
  3. validate JSON structure and model slug uniqueness;
  4. preserve the last-known-good catalog on network, HTTP, or validation failure;
  5. atomically replace the local catalog;
  6. rebuild $CODEX_HOME/models_cache.json;
  7. detect stale Codex app-server/Desktop processes;
  8. schedule the workflow separately on Windows, macOS, and Linux.

This duplicates security-sensitive filesystem and credential handling that OpenCodex already implements elsewhere and makes reliable Desktop synchronization unnecessarily difficult for non-technical client users.

What should OpenCodex do?

Provide an explicit remote-catalog pull workflow that consumes the existing /v1/catalog contract and installs the result as a local Codex catalog.

Observable requirements:

  • accept an HTTPS OpenCodex catalog URL;
  • read authentication from an environment reference or existing secure credential mechanism, never a plaintext CLI argument;
  • support conditional requests through ETag / If-None-Match, treating 304 as a no-op;
  • reject URL-embedded credentials and avoid forwarding authorization across origins;
  • enforce response-size and timeout bounds;
  • validate the complete catalog before the first filesystem mutation, including a non-empty models array and unique, valid slugs;
  • write through the existing catalog serialization and atomic replacement primitives;
  • preserve the last-known-good catalog and cache on every download or validation failure;
  • update models_cache.json in the same coordinated operation;
  • preserve mtimes and avoid process handling when the pulled catalog is unchanged;
  • report stale Codex processes after a real update;
  • keep --restart-codex and Windows --restart-desktop-app explicit and opt-in, because either can interrupt active work;
  • never mutate server configuration, provider credentials, or the source catalog.

The first implementation can stay intentionally narrow: one-shot pull plus cache update. A managed cross-platform scheduled integration can be discussed separately after the command contract is stable.

Example usage or interface

export OPENCODEX_CATALOG_AUTH_TOKEN='...'

ocx catalog pull \
  https://proxy.example.com/v1/catalog \
  --auth-env OPENCODEX_CATALOG_AUTH_TOKEN \
  --restart-codex

Machine-readable automation:

ocx catalog pull \
  https://proxy.example.com/v1/catalog \
  --auth-env OPENCODEX_CATALOG_AUTH_TOKEN \
  --json

Expected no-op result after an unchanged conditional request:

{
  "status": "unchanged",
  "catalogWritten": false,
  "cacheSynced": false,
  "codexRestarted": false
}

The exact command and environment-variable names are proposed for discussion; the important contract is authenticated remote-to-local materialization with coordinated catalog/cache writes and opt-in process restart.

Alternatives or workarounds

  • Serve a deployment-specific /dl directory: works, but duplicates the existing authenticated /v1/catalog endpoint and makes nginx/web-root permissions part of catalog correctness.
  • Cron plus shell/PowerShell scripts: currently workable, but each operator must reimplement validation, locking, atomic replacement, cache generation, error handling, and secret hygiene.
  • Point model_catalog_json at an HTTPS URL: incompatible with Codex's local-path contract and startup behavior.
  • Extend ocx export --client: misleading because that command exports client-specific configuration rather than the complete Codex catalog.
  • Server-side watcher or publisher daemon: does not solve client-side models_cache.json or in-memory Desktop staleness and adds another service lifecycle.
  • Automatically restart Codex Desktop: rejected as a default because it can terminate active conversations; restart must remain explicit.

Additional context

Related existing work solves adjacent, but different, layers:

A production deployment currently uses a validated atomic catalog publisher plus a Windows downloader/cache rebuilder. That proves the workflow is useful, but the proposal deliberately avoids upstreaming deployment-specific paths, nginx configuration, cron syntax, or public unauthenticated /dl behavior.

Security boundaries for implementation review:

  • no token in argv, output, errors, redirects, or persisted catalog metadata;
  • HTTPS by default;
  • no cross-origin credential forwarding;
  • bounded body acquisition;
  • validation before mutation;
  • catalog/cache writes only under the existing per-CODEX_HOME serialization permit;
  • no implicit process termination.

Checks

  • I searched existing issues and documentation.
  • This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
  • I removed secrets and personal data.

Activity

  1. github-actions
    added
    enhancementNew feature or request
    catalogModel catalog, slugs, visibility, routed entries
    on Sep 6, 2026

  2. lidge-jun commented on Sep 6, 2026

    @lidge-jun
    lidge-jun
    on Sep 6, 2026
    Owner

    리뷰 · 우선순위 54 / 80

    이 이슈는 원격 OpenCodex가 이미 만들어 둔 모델 목록(/v1/catalog)을, 로컬 Codex의 CODEX_HOME 카탈로그·models_cache.json에 안전하게 받아 심는 한 방 워크플로를 요청합니다. 지금 dev(HEAD af344a28e, 패키지 2.44.0)에는 서버 쪽 조각은 있습니다. #809로 GET|HEAD /v1/catalog가 데이터 플레인에 있고, 직렬화는 src/server/catalog-download.ts가 /api/catalog와 같은 바이트를내도록 공유합니다. 크기 상한은 원격 경로만 MAX_REMOTE_CATALOG_BYTES(256 MiB)입니다. 로컬 쪽에는 model_catalog_json 주입(src/codex/inject.ts), withCatalogWriteSerialization 원자 교체, invalidateCodexModelsCacheWithPermit 캐시 갱신, ocx sync / ocx sync-cache의 --restart-codex·Windows --restart-desktop-app(src/cli/dispatch.ts, src/codex/desktop-app-restart.ts, #2292 계열)이 이미 있습니다. #3699의 src/integrations/catalog-refresh.ts는 Pi/Aside처럼 로컬에서 고른 카탈로그를 통합에 다시 밀어 넣는 쪽이고, 원격 HTTPS 카탈로그를 받아 Codex 홈에 심는 명령은 아닙니다.

    이슈가 말하는 빈틈도 맞습니다. model_catalog_json은 로컬 경로만 받고, ocx sync는 이 머신의 설정된 공급자로 카탈로그를 만들고, ocx export --client는 클라이언트용 설정 쪽입니다. 허브 연결(src/client/connect.ts 등)은 다른 계약의 원격 카탈로그 동기화가 있지만, “임의 HTTPS /v1/catalog URL + --auth-env로 한 번 pull” 운영자 UX는 없습니다. 제안 계약(토큰을 argv에 안 넣기, 검증 후 쓰기, 실패 시 last-known-good 보존, 변경 없을 때 mtime/프로세스 손대지 않기, 재시작은 명시 opt-in)은 기존 sync/restart 관례와 잘 맞습니다.

    다만 조건부 요청(ETag / If-None-Match / 304) 요구는 현재 dev의 /v1/catalog 설계와 정면으로 어긋납니다. src/server/index.ts는 자격 증명마다 본문이 달라질 수 있어 공유 캐시·검증자 재검증이 위험하다고 보고, 의도적으로 cache-control: no-store, ETag 없음, 304 없음을 고수합니다. 관리면 GET /api/catalog만 ETag를 줍니다. src/client/connect.ts 주석도 “/v1/catalog는 validator를 안 낸다(Phase 1)”고 적고, 로컬 해시로 소유권을 확인합니다. 그래서 이슈의 304 계약 그대로 구현하려면 서버 정책을 바꾸거나, 클라이언트만 content-hash로 “unchanged”를 판정하는 쪽으로 범위를 줄여야 합니다. 첫 구현을 one-shot pull + cache로 좁히자는 제안은 타당하고, 스케줄러/크로스플랫폼 데몬은 뒤로 미루는 편이 맞습니다.

    우선순위 54인 이유다. 중앙 호스팅 배포에는 실사용 가치가 있고 보안 경계도 잘 적혀 있지만, 지금 dev 최적화 축(macOS CI·릴리스 후보)과 겹치지 않는 새 CLI 표면 + 설계 합의가 필요합니다. 구현 전에 서버 304 정책을 유지할지부터 정해야 합니다. types.ts/config.ts 대분할에 통째로 무효화될 성격은 아니고, 중복 이슈로 바로 닫을 대상도 아닙니다(#3630은 로컬 공급자 자동 refresh라 인접하지만 다른 문제).

    경로/심볼 - src/server/index.ts GET|HEAD /v1/catalog - 인증·origin·크기 507·no-store·ETag/304 없음 (이슈의 조건부 요청과 충돌)
    경로/심볼 - src/server/catalog-download.ts serializePersistedCatalog / catalogEtag / MAX_REMOTE_CATALOG_BYTES - 바이트·etag 계산은 공유 모듈에 있으나 원격 라우트가 validator를 응답에 안 실음
    경로/심볼 - src/cli/dispatch.ts ocx sync / sync-cache - 로컬 공급자 동기화 + 재시작 플래그 관례. ocx catalog pull 서브커맨드는 없음
    경로/심볼 - src/codex/catalog-write-serialization.ts / src/codex/catalog/sync.ts - 원자 교체·models_cache.json 무효화 재사용 후보
    경로/심볼 - src/codex/desktop-app-restart.ts / #2292 - Desktop 전체 재시작은 대화 중단이므로 기본 off·명시 플래그 (이슈와 동일)
    경로/심볼 - src/integrations/catalog-refresh.ts (#3699) - Pi/Aside 등 로컬 선택 반영. 원격 pull 대체재 아님
    경로 - #809 / #709 / #3023 / #2292 - 인접 계층. 이 이슈는 클라이언트 materialization UX

    메인테이너의 판단이 필요한 지점

    • /v1/catalog에 ETag·304를 열지(보안 주석과 거래), 아니면 클라이언트가 로컬 해시로 unchanged를 판정하고 서버는 매번 전체 본문을 줄지
    • 명령 표면: 새 ocx catalog pull vs ocx sync --from-url 확장 vs hub-connect 전용 경로만 강화
    • 인증: --auth-env만 허용할지, 기존 OPENCODEX_API_AUTH_TOKEN / 키체인과 어떻게 맞출지
    • pull이 model_catalog_json이 가리키는 파일만 바꾸는지, Codex config 주입·통합 refresh(catalog-refresh)까지 묶을지
    • 첫 PR 범위: one-shot + cache + stale 프로세스 보고까지만 두고 restart는 기존 플래그 재사용할지

    너의 추천
    바로 구현 PR을 받기보다 설계 한 줄을 먼저 고정하세요. 권장: 서버 /v1/catalog의 no-ETag/no-304는 유지하고, 클라이언트는 다운로드 후 로컬 SHA로 unchanged를 판정하며, 쓰기는 기존 withCatalogWriteSerialization + cache invalidate + 기존 restart 플래그 재사용. 합의되면 help wanted / needs-design로 두고 좁은 CLI PR을 받습니다. 지금은 이슈를 열린 채로 두세요. 중복 close 대상 아님.

    이 댓글은 grok-bot이 작성했습니다

  3. rrmlima commented on Sep 6, 2026

    @rrmlima
    rrmlima
    on Sep 6, 2026
    ContributorAuthor

    Thanks for the detailed review. I agree that the first implementation should preserve the current /v1/catalog security contract rather than couple this client workflow to a server-side validator change.

    Proposed Phase 1 contract

    I suggest fixing the design as follows:

    1. Keep /v1/catalog exactly as it is today: authenticated, Cache-Control: no-store, no ETag, and no 304 contract.

    2. Add a one-shot client command:

      ocx catalog pull <https-url> [--auth-env <NAME>] [--json] [--restart-codex]

    3. Download and validate the complete response outside catalog serialization.

    4. Determine unchanged by comparing a locally computed SHA-256/content snapshot with the active local catalog.

    5. When unchanged, preserve catalog/cache mtimes and do not inspect or terminate processes.

    6. When changed, commit the active catalog and models_cache.json through the existing per-CODEX_HOME serialization permit and atomic writer paths.

    7. Preserve the last-known-good catalog/cache on transport, HTTP, size, validation, lock, or write failure.

    8. Read credentials only through an environment reference or an existing secure credential resolver; never accept a literal token argument or forward authorization across origins.

    9. Keep process handling explicit. --restart-codex may reuse the current app-server mechanism after a real write. Full Windows Desktop restart remains out of Phase 1 and can only be considered later as an explicit opt-in.

    10. Exclude scheduling, daemon behavior, GUI, config persistence, /dl, server changes, and managed integration from the first PR.

    This keeps the server policy unchanged, avoids expanding the authentication boundary, and limits the initial patch to client-side materialization plus cache convergence.

    Decisions requested before implementation

    My recommendation is:

    • command surface: new ocx catalog pull namespace, because ocx sync builds from local providers and ocx export --client emits client-specific configuration;
    • authentication: optional --auth-env <NAME>, falling back to OPENCODEX_API_AUTH_TOKEN only if that matches the project's desired credential convention;
    • destination: the active local path already resolved by model_catalog_json / CODEX_HOME; no arbitrary output path in Phase 1;
    • config ownership: do not inject or rewrite Codex config in Phase 1; only update an already resolved active catalog path and cache;
    • process behavior: report stale app-servers by default and restart only with the existing explicit flag.

    If this contract matches the maintainer direction, I can prepare the focused regression-first PR against the latest dev without changing /v1/catalog.

  4. rrmlima
    added a commit that references this issue on Sep 12, 2026

    feat(codex): pull an authenticated remote catalog into local Codex state

    2aa821d

  5. lidge-jun commented on Sep 13, 2026

    @lidge-jun
    lidge-jun
    on Sep 13, 2026
    Owner

    Resolved on dev. The authenticated remote catalog pull landed through #4481 (merge commit d865aac, verified as an ancestor of origin/dev), carrying your #4413 with a Co-authored-by trailer in the landed commit.

    The landed version is stricter than the branch in two places that matter for this ask. A failed cache sync no longer leaves a new catalog paired with a stale models_cache.json: the previous bytes are restored, or the file is removed when there was no catalog before, and the failure is raised instead of being reported as catalogWritten: false. And --restart-codex only reports success when every target process actually stopped; a partial restart exits 1 with code "restart_incomplete" while still reporting the write that did happen.

    Closing manually because these merge into dev rather than the default branch, so GitHub does not auto-close the link.

  6. lidge-jun
    closed this as completedon Sep 13, 2026
  7. FacuM
    added a commit that references this issue on Sep 13, 2026

    feat(codex): pull an authenticated remote catalog into local Codex state

    394b96d
  8. agentHits
    added a commit that references this issue on Sep 17, 2026

    feat(codex): pull an authenticated remote catalog into local Codex state

    2c394d9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    catalogModel catalog, slugs, visibility, routed entriesenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions