Repository navigation
feat: self-discover upstream model name for /v1/models - #4
Merged
Merged
Conversation
The CNB gateway exposes no model-list endpoint, so the only trustworthy source of the real model id is the `model` field echoed on every completion. Add a passive registry (src/models.mjs) that records the echoed name in first-seen order, atomically persists it to PROXY_MODELS_STATE (survives a workspace hot restart), and makes /v1/models advertise the real name. PROXY_MODELS becomes a fallback used only until the first response has been sniffed. Update the default model list to deepseek-v4.1-flash (the gateway now resolves every request to it) and mirror the docs (README x2, DESIGN, SETUP, .env.example). Tests: 61/61 (6 new models unit tests + 2 proxy E2E sniffing tests).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
/v1/modelscurrently advertises a hardcoded list. The CNB gateway has no model-list endpoint, so this drifts whenever the upstream model is renamed — which just happened: the gateway now resolves every request todeepseek-v4.1-flash(verified live: sendingglm-5.3-flash,kimi-k3, or even a made-up name all echodeepseek-v4.1-flash).How
src/models.mjs: a passive registry.proxy.mjscallsobserve(model)on streamed chunks and on the non-stream aggregate; the registry keeps first-seen order, dedups, ignores junk, and persists atomically (.tmp+renameSync) toPROXY_MODELS_STATE./v1/modelsreturns the sniffed name(s);PROXY_MODELSis only a fallback until the first response is sniffed.deepseek-v4.1-flash; docs (README ×2, DESIGN, SETUP, .env.example) updated.Verification
node --test→ 61 pass / 0 fail (6 new models unit tests incl. cross-process reload, 2 new proxy E2E sniffing tests).