feat(providers): remap outbound model IDs for deployment-key endpoints - #900
feat(providers): remap outbound model IDs for deployment-key endpoints#900vignesh-chaturvedi wants to merge 1 commit into
Conversation
An OpenAI-compatible endpoint reached through a provider's *_BASE_URL may publish the catalog's models under its own names, and the deployment-key path had no way to rewrite them. ROUTER_<PROVIDER>_MODEL_ALIASES takes a JSON map of catalog model ID to upstream name and layers it over the catalog's per-binding UpstreamID. Only the outbound wire name changes: routing, pricing, and analytics stay keyed on the catalog ID. A BYOK key's model_aliases still wins, since it is applied after the catalog map in openaicompat.Client.Proxy. A malformed value aborts boot; an alias naming a model outside the deployed catalog is logged and skipped, so retiring a model cannot turn a stale alias into a failed start. OpenRouter and XAI now build with a model ID map where they previously passed nil. Both resolve to zero catalog entries, so an unconfigured deployment puts exactly the same bytes on the wire as it does today. upstreamIDsForProvider moves to the new file with it; its doc comment had drifted onto registerDeploymentKeyedProvider in main.go. Closes workweave#526 Closes workweave#541 Signed-off-by: Vignesh Chaturvedi <vigneshchaturvedi@gmail.com>
|
PR author is not in the allowed authors list. |
|
Worth flagging prior art I should have led with: #557 already targets #526 and The difference is scope. #557 adds If you'd rather land #557 and treat multi-provider support as a follow-up, I'm |
|
Thanks for this — it follows the repo conventions closely, so no changes needed from a conventions standpoint. What I checked:
Also verified the two claims the design rests on: The only open item is the one you already flagged yourself — the scope overlap with #557. That's a maintainer sequencing call rather than a code issue, and flagging it proactively was the right thing to do. |
What
Adds
ROUTER_<PROVIDER>_MODEL_ALIASES, a JSON map of catalog model ID → the model name a provider's endpoint publishes, applied to the outbound request body on the deployment-key path.Closes #526
Closes #541
Why
Pointing a provider at an OpenAI-compatible endpoint that isn't the vendor itself is already a supported, documented setup —
.env.exampledescribesOPENROUTER_BASE_URLas "override for vLLM/Together/etc." But if that endpoint publishes the catalog's models under different names, there is no way to rewrite them, and every turn 400s.#541 reproduces it end to end: the router sends
deepseek/deepseek-v4-flash, the gateway expectsdeepseek-v4-flash, and the response isModel ... is not supported.Rewriting exists, but only on two paths that don't cover this case:
UpstreamIDupstreamIDsForProvider, baked into the bindingExternalAPIKey.ModelAliases→proxy.ApplyModelAliasinternal/proxy/credentials.go:50documents it as "non-empty only on BYOK credentials"So a self-hosted deployment using
OPENROUTER_API_KEY+OPENROUTER_BASE_URL— exactly #541's repro — has no mechanism at all. This adds the third layer.How
resolveModelAliasesreadsROUTER_<PROVIDER>_MODEL_ALIASESfor each OpenAI-compatible provider and layers the entries over that provider's catalog-derivedUpstreamIDmap. The merged map goes to the existingNewClientWithModelIDMap, so the rewrite itself reusesrewriteModelFieldunchanged — no new code on the request path.Precedence, outermost wins: catalog
UpstreamID<ROUTER_<PROVIDER>_MODEL_ALIASES< BYOKmodel_aliases. The BYOK layer keeps winning for free, becauseClient.Proxyalready applies it after the catalog map:Only the outbound wire name changes. Routing, pricing, and analytics stay keyed on the catalog ID.
Failure modes. A malformed value aborts boot (
panic, consistent withconfig.MustGetandROUTER_DEPLOYMENT_MODE) — the error names the env var. An alias naming a model outside the deployed catalog is logged and skipped rather than fatal, so retiring a model (#896, #897) can't turn a stale alias into a failed start.Validation is against catalog IDs, not per-provider bindings, deliberately. An operator pointing
OPENROUTER_BASE_URLat a different gateway may well be served models the catalog binds elsewhere — #526's own list includesz-ai/glm-5.2, which the catalog binds to Together and Fireworks. Rejecting those would reject the use case.Placement. The logic sits in
cmd/routerrather thaninternal/configbecause it needsinternal/router/catalog, andAGENTS.mdrequiresinternal/configto stay a leaf that imports nothing else underinternal/. It is boot-time wiring, which is the composition root's job.Behavior change
None without configuration.
resolveModelAliaseswith no env vars set produces exactly the maps in use today:OpenRouter and XAI now construct via
NewClientWithModelIDMapwhere they previously calledNewClient. Both resolve to zero entries, so the map is nil andrewriteModelFieldreturns the body untouched — an unconfigured deployment puts the same bytes on the wire as before.Testing
make build,make vet,make test, andmake test-statuslineare green, andmake generate-statuslineleaves the tree clean. (I couldn't runsqlc generatelocally, but nodb/queries/or migration files are touched, sointernal/sqlc/is untouched.)model_aliases_wire_test.goreproduces #541 at the wire. Anhttpteststub stands in for a gateway publishing bare names, and the assertion is themodelfield it actually receives:deepseek/deepseek-v4-flash— what the gateway rejectsROUTER_OPENROUTER_MODEL_ALIASESsetdeepseek-v4-flash— what it acceptsxiaomi/mimo-v2.5-pro, unchangedI chose this over booting the compose stack because it pins the exact byte #541 is about, runs in CI on every change, and needs no upstream key. It also closes a gap: nothing currently asserts
modelIDMaprewriting at all.model_aliases_test.goadds 11 cases: parsing (empty, valid, malformed JSON, JSON array, empty key, empty value), env override application, override beating a catalogUpstreamID, unaliased bindings surviving the merge, unknown models skipped without error, and a malformed value erroring with the env var named. They discover a suitable model fromcatalog.Modelsat run time rather than pinning an ID, so catalog churn doesn't break them.Mutation-checked: dropping the
merged[id] = upstreamIDwrite failsTestResolveModelAliasesAppliesEnvOverride,TestResolveModelAliasesOverridesCatalogUpstreamID, and the wire test's alias case, while the two no-alias cases correctly keep passing.Not run: the full
docker composestack against a real re-named endpoint. The evidence above is the test suite plus the resolved-map parity table.Docs
docs/CONFIGURATION.md— new "Deployment-level model aliases" subsection plus a row in the provider table..env.example— commentedROUTER_OPENROUTER_MODEL_ALIASESexample next to the other provider vars.cmd/CLAUDE.mdandcmd/AGENTS.md—resolveModelAliasesadded to the composition-root helper list, same edit in both halves of the mirror.Incidental
upstreamIDsForProvidermoves into the new file alongside its only caller. Its doc comment had drifted ontoregisterDeploymentKeyedProviderinmain.go; moving the function reunites the two.