[stack 2/8] feat: expand providers and make native model discovery account-aware

AGENTS.md

@@ -80,6 +80,37 @@ This applies to `AGENTS.md`-following agents as much as to humans. If a task
 asks you to write up a security finding, put the write-up in scratch space and
 say where it is; do not add it to `devlog/`, `structure/`, or `docs-site/`.
 
+## Native Codex + OpenCodex coexistence invariant
+
+OpenCodex extends native Codex; it does not replace the native OpenAI identity or create a second
+authority for native model metadata.
+
+- On the normal loopback install, keep Codex's built-in `openai` provider identity and the user's
+  ordinary ChatGPT/Codex login. Route it through OCX with the managed `openai_base_url` override;
+  do not re-tag native threads to an OCX provider.
+- While OCX owns active routing, `$CODEX_HOME/opencodex-catalog.json` is the only active merged
+  catalog. Never create or point Codex at parallel merge files such as
+  `~/.codex/model-catalogs/native-plus-ocx.json`.
+- Managed sync/build must target that canonical catalog **before** config injection. Never use the
+  current root `model_catalog_json` as an OCX write target: it may still be the user's pre-OCX
+  catalog. Cache invalidation must consume the exact catalog path written by the same sync.
+- Restore/eject may restore a user-owned catalog pointer from the journal, but catalog cleanup must
+  still target only the canonical OCX-managed catalog. Never strip routed rows from that restored
+  user catalog.
+- Bare native OpenAI rows must come from authoritative native discovery for the installed Codex
+  client/account and retain upstream capability metadata. In particular, do not drop or synthesize
+  `context_window`, reasoning ladders, `supports_search_tool`, `tool_mode`, or
+  `use_responses_lite` for newly rolled-out models.
+- If the OCX merged catalog cannot be materialized, or it does not contain the currently selected
+  bare native GPT/Codex model, prefer Codex's native catalog over generic fallback metadata or a
+  stale/custom root `model_catalog_json`. The injection journal preserves the user's prior
+  config/catalog pointer and restores it on stop/eject.
+- A user-owned external `model_provider` or root `openai_base_url` remains an ownership boundary;
+  OCX must not silently take it over.
+
+Any change to injection, catalog sync, native discovery, install/start/ensure, or restore must keep
+these invariants covered by focused regression tests.
+
 ## Commands
 
 ```bash
@@ -197,4 +228,4 @@ Use the canonical labels `needs-triage`, `needs-info`, `ready-for-agent`, `ready
 
 This repository uses a single-context domain layout with root `CONTEXT.md` (when present) and `docs/adr/`. See `docs/agents/domain.md`.
 
-Compound Engineering overlay: `.compound-engineering/` (tracked `config.yaml`, gitignored `config.local.yaml`). Artifact root `.compound-engineering/artifacts/`. Portable skills `~/.agents/skills/ce-*`; native Cursor plugin is fallback only when this overlay is absent.
+Compound Engineering overlay: `.compound-engineering/` (tracked `config.yaml`, gitignored `config.local.yaml`). Artifact root `.compound-engineering/artifacts/`. Portable skills `~/.agents/skills/ce-*`; native Cursor plugin is fallback only when this overlay is absent.

docs-site/src/content/docs/guides/codex-integration.md

@@ -31,6 +31,28 @@ The proxy listens on port `10100` by default and serves `POST /v1/responses`,
 `POST /v1/responses/compact`, `POST /v1/images/generations`, `POST /v1/images/edits`,
 `GET /v1/models`, `GET /healthz`, and the `/api/*` management surface.
 
+### Native Codex and OpenCodex together
+
+The loopback setup is intentionally **native-first**. You stay signed in to Codex with your normal
+ChatGPT/Codex account, threads keep the native `openai` provider id, and native OpenAI models remain
+bare ids such as `gpt-6.1-sol`. OpenCodex adds routing and extra providers around that native path;
+it does not require a second Codex identity.
+
+While OpenCodex owns that routing, `$CODEX_HOME/opencodex-catalog.json` is the active merged catalog.
+`ocx start`, `ocx ensure`, and `ocx sync` enforce that pointer. If `config.toml` previously
+pointed at another custom merge file, that value is journaled for restore but is not kept as the
+active catalog while OCX is running. This prevents a stale custom catalog from downgrading a newly
+rolled-out native model to Codex's generic fallback metadata.
+
+Native rows are discovered from ChatGPT's Codex model endpoint with the installed Codex client
+version and the effective Codex account. Their upstream capability fields are authoritative,
+including context window, reasoning levels, search/deferred-tool support, Responses-lite behavior,
+modalities, and visibility. If the OCX merged catalog cannot be built, OpenCodex prefers Codex's
+native catalog rather than leaving an unrelated custom root `model_catalog_json` active.
+
+Do not manually create a second `native-plus-ocx.json` style catalog to combine the two. The
+canonical OCX catalog is already the native-plus-routed merge.
+
 ### Built-in image generation (`image_gen`)
 
 Codex's built-in `image_gen` tool does not go through `/v1/responses` — the codex-rs extension
@@ -135,9 +157,9 @@ mode leaves this profile untouched.
 
 :::caution
 Root keys such as `openai_base_url`, `model_provider`, and `model_catalog_json` **must** sit before the
-first `[table]` header. The injector guarantees that placement, removes its own stale/duplicate
-copies, and never overwrites a user-owned root `openai_base_url`; if one exists, sync updates the
-catalog but reports that routing was not injected.
+first `[table]` header. The injector guarantees that placement and removes its own stale/duplicate
+copies. A user-owned root `openai_base_url` is an ownership boundary: OpenCodex leaves both routing
+and the root catalog pointer untouched rather than managing only half of the configuration.
 :::
 
 ## Shared model catalog
@@ -184,13 +206,16 @@ start and on `ocx sync`, opencodex:
 
 1. **Backs up** the pristine catalog once to `~/.opencodex/catalog-backup.json` (so featuring is
    reversible).
-2. **Fetches** eligible providers' live model catalogs (cached ~5 min; falls back to the last good
-   list, then configured `models[]`). Forward auth has no model endpoint, and Cursor uses its
-   `GetUsableModels` RPC rather than `/models`.
-3. **Merges** routed models in as namespaced entries (`provider/model`), cloned from a native Codex
-   catalog template so Codex's strict parser accepts them.
-4. **Filters** `config.disabledModels` and each provider's non-empty `selectedModels` allowlist.
-5. **Re-ranks** so featured models sort first (see below), then writes the merged catalog back.
+2. **Discovers native OpenAI rows live** from the Codex model endpoint using the installed client
+   version and the effective ChatGPT/Codex account. New native slugs are accepted without waiting
+   for an OpenCodex release, and their upstream metadata is preserved.
+3. **Fetches** eligible routed providers' live model catalogs (cached ~5 min; falls back to the last
+   good list, then configured `models[]`). Forward auth has no generic provider model endpoint, and
+   Cursor uses its `GetUsableModels` RPC rather than `/models`.
+4. **Merges** routed models in as namespaced entries (`provider/model`), cloned from a native Codex
+   catalog template so Codex's strict parser accepts them without mutating authoritative native rows.
+5. **Filters** `config.disabledModels` and each provider's non-empty `selectedModels` allowlist.
+6. **Re-ranks** so featured models sort first (see below), then writes the merged catalog back.
 
 Routed catalog entries also get their GPT-5 identity rewritten to the real upstream model name.
 Reasoning controls come from provider/model metadata across Codex's `low | medium | high | xhigh |
@@ -246,38 +271,33 @@ rows below them.
 ### Catalog troubleshooting
 
 If a model is missing from Codex, or the catalog order/visibility looks wrong, check in order:
-
-1. **`selectedModels`** on the provider — a non-empty allowlist exposes only those ids to Codex;
+\n2. **Active catalog ownership** — while OCX owns routing, the root `model_catalog_json` should point
+   to `$CODEX_HOME/opencodex-catalog.json`. A parallel `native-plus-ocx.json` or other merged file
+   is drift. Run `ocx sync` (or `ocx ensure`) to repair the managed pointer; `ocx stop` restores
+   the pre-OCX user value from the journal.\n2. **`selectedModels`** on the provider — a non-empty allowlist exposes only those ids to Codex;
    empty or omitted exposes all discovered models. An id not in the allowlist never reaches the
-   catalog.
-2. **`disabledModels`** (top level) — hides models from both the catalog and `/v1/models`, and flips
-   bare native GPT slugs to `visibility: "hide"`.
-3. **`liveModels: false` with empty `models`** — when live discovery is off and `models` is empty or
-   omitted, opencodex exposes no routed models for that provider.
-4. **Cursor `GetUsableModels`** — the Cursor adapter discovers models through its protobuf
+   catalog.\n3. **`disabledModels`** (top level) — hides models from both the catalog and `/v1/models`, and flips
+   bare native GPT slugs to `visibility: "hide"`.\n4. **`liveModels: false` with empty `models`** — when live discovery is off and `models` is empty or
+   omitted, opencodex exposes no routed models for that provider.\n5. **Cursor `GetUsableModels`** — the Cursor adapter discovers models through its protobuf
    `GetUsableModels` RPC, not `/models`, so a Cursor-side change can alter which ids are visible
-   independently of other providers.
-5. **Cache and `ocx sync`** — live catalogs are cached for about five minutes (`modelCacheTtlMs`,
-   default `300000`). Run `ocx sync` to force a fresh fetch and rewrite the catalog immediately.
-6. **Running Codex `app-server`** — rewriting the on-disk catalog is not enough while a long-lived
+   independently of other providers.\n6. **Cache and `ocx sync`** — live catalogs are cached for about five minutes (`modelCacheTtlMs`,
+   default `300000`). Run `ocx sync` to force a fresh fetch and rewrite the catalog immediately.\n7. **Running Codex `app-server`** — rewriting the on-disk catalog is not enough while a long-lived
    Codex `app-server` (Desktop / CLI background host) keeps the previous list in memory. `ocx sync`
    and `ocx sync-cache` warn when those processes are detected. Restart them with
    `ocx sync --restart-codex` (or stop the matching `app-server` processes yourself), then let Codex
-   recreate them so the new list appears.
-7. **`hideUnavailableModels`** — when enabled, a provider that is dead (all accounts need reauth, or
+   recreate them so the new list appears.\n8. **`hideUnavailableModels`** — when enabled, a provider that is dead (all accounts need reauth, or
    discovery fails N times) drops from `/v1/models` and the new-session picker while the admin Models
    tab still shows last-good rows with a reason. Codex and Cursor cache their pickers; start a **new
    session** (or restart the client / run `ocx sync --restart-codex`) before expecting the filtered
    list. Existing sessions keep routing to last-good models.
 
 :::caution[Other local writers]
-Catalog writes (`opencodex-catalog.json`, `config.toml`) are atomic **inside** opencodex, which only
-prevents half-written files when two opencodex-owned writers race. That does **not** stop another
-local process, file watcher, or sync agent from rewriting catalog visibility or order after opencodex
-has written. Codex keeps its separate `models_cache.json` and can refresh it independently, changing
-the visible list without rewriting `opencodex-catalog.json`. If models flip unexpectedly while the
-proxy is running, stop or reconfigure the competing writers, then run `ocx sync` — this is an
-external-writer hazard, not a confirmed opencodex defect.
+Catalog writes (`opencodex-catalog.json`, `config.toml`) are atomic **inside** opencodex. Another
+local process can still rewrite them afterwards. While OCX owns routing, changing the root
+`model_catalog_json` to a competing merged catalog is configuration drift; the next `ocx start`,
+`ocx ensure`, or `ocx sync` repairs the managed pointer. Codex also keeps
+`models_cache.json` and can retain a stale in-memory catalog in a long-lived app-server, so restart
+that client after a catalog repair when needed.
 :::
 
 ## Proxy connection errors
@@ -341,4 +361,4 @@ ocx restore back # point plain Codex at the running proxy again
 
 When opencodex runs as a managed [background service](/reference/cli/#ocx-service), it sets
 `OCX_SERVICE=1` so a service-driven restart does **not** thrash the Codex config — only an explicit
-`ocx stop` / `ocx service stop` restores native Codex.
+`ocx stop` / `ocx service stop` restores native Codex.

src/AGENTS.md

@@ -19,10 +19,26 @@ This file applies to `src/` and inherits the repository-wide rules in `/AGENTS.m
 - Adapter changes must preserve the internal event contract, streaming behavior, tool calls, cancellation, error mapping, and image handling relevant to that adapter.
 - Authentication, OAuth, token, credential, management API, and CORS changes are security-boundary changes.
 
+## Codex native coexistence
+
+For Codex integration work, the loopback path is additive: keep the built-in `openai` provider
+identity and ordinary ChatGPT/Codex auth, and let OCX own only the managed proxy transport plus the
+canonical merged catalog at `$CODEX_HOME/opencodex-catalog.json`.
+
+Do not introduce alternate "native + OCX" catalog files or preserve a competing root
+`model_catalog_json` while OCX owns routing. Managed sync must write the canonical OCX catalog
+directly before injection and must never use the currently configured user catalog as an
+intermediate write target. Restore cleanup likewise targets only the managed catalog after the
+journal restores user config. Native bare OpenAI rows are authoritative live rows;
+preserve their capability fields unchanged so newly rolled-out models do not fall back to generic
+Codex metadata. When no managed catalog is available, or the selected bare native GPT/Codex slug is
+absent from it, remove the managed root catalog override and let native Codex metadata win.
+Restore/eject must recover the user's pre-OCX config through the journal.
+
 ## Tests and validation
 
 - Place focused regression coverage near the existing tests for the affected subsystem.
 - For focused behavior, run the relevant `bun test tests/<name>.test.ts` and `bun run typecheck`.
 - For shared routing, adapters, config, OAuth, or server behavior, also run `bun run test`.
 - For logging, requests, credentials, account data, or fixtures, also run `bun run privacy:scan`.
-- Update `docs-site/` when the change affects user-visible behavior or configuration.
+- Update `docs-site/` when the change affects user-visible behavior or configuration.

src/codex/catalog/metadata.ts

@@ -137,10 +137,18 @@ export function nativeModelRows(config: Pick<OcxConfig, "disabledModels">): Arra
   });
 }
 
-export function applyNativeVisibility(entries: RawEntry[], disabledNative: Set<string>): RawEntry[] {
+export function applyNativeVisibility(
+  entries: RawEntry[],
+  disabledNative: Set<string>,
+  authoritativeNativeSlugs: ReadonlySet<string> = new Set(),
+): RawEntry[] {
   for (const entry of entries) {
     const slug = typeof entry.slug === "string" ? entry.slug : "";
-    if (!slug || slug.includes("/") || !SUPPORTED_NATIVE_OPENAI_SLUGS.has(slug)) continue;
+    if (
+      !slug
+      || slug.includes("/")
+      || (!SUPPORTED_NATIVE_OPENAI_SLUGS.has(slug) && !authoritativeNativeSlugs.has(slug))
+    ) continue;
     entry.visibility = disabledNative.has(slug) ? "hide" : "list";
   }
   return entries;