Feature: let vision/web-search sidecar use any configured provider's model (drop openai|anthropic-only backend) #1276

Area

Proxy and routing · Dashboard · Catalog / models

What are you trying to accomplish?

I want to pick my own provider's model as the vision sidecar and web-search sidecar, from the same dropdowns where I select models — not be forced to use a native ChatGPT/Claude model. For example: route the main model through z.ai GLM / DeepSeek / Kimi / Qwen, and have the vision sidecar describe images with Qwen-VL (or whatever vision model I have quota on), and the web-search sidecar execute searches with a model from a provider I actually have available, instead of borrowing the ChatGPT or Anthropic account.

What prevents this today?

The sidecar backend is hardcoded to "openai" | "anthropic" and the model must be a native ChatGPT model:

• src/types.ts: OcxVisionSidecarConfig.backend?: "openai" | "anthropic" and OcxWebSearchSidecarConfig.backend?: "openai" | "anthropic" (same in OcxClaudeDesktopProfile.webSearchSidecar/visionSidecar).
• The management API rejects any other value with 400: src/server/management/config-routes.ts — "webSearch.backend must be openai, anthropic, or null" / "vision.backend must be openai, anthropic, or null".
• src/web-search/index.ts resolveSidecarBackend() and src/vision/index.ts resolveVisionBackend() only resolve to "openai" / "anthropic".

So in the UI dropdowns the sidecar model list only shows native ChatGPT models, and there is no provider: selector to route the sidecar through any other configured provider.

What should OpenCodex do?

Allow the sidecar backend to reference any configured provider (by provider id), with model set to any model that provider exposes:

• backend accepts a provider id (e.g. "zai", "deepseek", "moonshot", "qwen", ...) in addition to "openai" | "anthropic".
• The vision/search sidecar dropdowns in the UI list models from all configured providers (with a capability hint: vision-capable for the vision sidecar), not just ChatGPT natives.
• If the sidecar model is a text-only model, the existing graceful-degradation contract (e.g. [image omitted: …] marker) still applies.

Example usage or interface

// ocx config (sidecar settings)
{
  "visionSidecar": {
    "backend": "qwen",            // any configured provider id, not just openai|anthropic
    "model": "qwen3-vl-plus"      // model from that provider's catalog
  },
  "webSearchSidecar": {
    "backend": "zai",
    "model": "glm-5.2"
  }
}

In the Dashboard sidecar settings, the backend/model dropdown would show e.g. openai · anthropic · zai · deepseek · moonshot · qwen … and the model list follows the selected provider.

Alternatives or workarounds

• feat(sidecar): add Exa and other search providers as web-search sidecar backends #414 proposes adding third-party search APIs (Exa) as extra hardcoded backends — helpful, but it does not let me point the sidecar at a provider I already have configured and funded.
• Today I can only work around this by maintaining a ChatGPT/Claude account purely as the sidecar backend, even when my main provider has its own capable vision model.

Additional context

Related but not identical existing issues:

• [Bug] Web Search/Vision Sidecar Backend Fixed to OpenAI·Anthropic - Routing Turns Fail with 499/502 When Limits are Exhausted #398 — confirmed the openai|anthropic-only constraint (as a bug source), focused on 499/502 failures when those accounts are exhausted.
• feat:Enable DeepSeek and other text-only input models to perform image recognition and search functions. #88 — vision sidecar for text-only models was implemented, but the description model stays a native ChatGPT model (visionSidecar.model, default gpt-5.2).
• [Bug] Custom-provider vision capability is ambiguous when model metadata is missing #1024 — custom-provider vision capability ambiguity (registry metadata missing), different angle.
• [Feature]: expose remaining vision sidecar controls in Dashboard #1201 — exposing sidecar config in the web UI, related to the dropdown surface.