feat: add bedrock-mantle vendor for Bedrock's OpenAI-compatible endpoint - #20
Merged
Merged
Conversation
One vendor covering Bedrock's resold open-weight catalog (GLM, Kimi, Grok, DeepSeek, MiniMax, Qwen) behind a single endpoint and Bedrock API key, so a newly-launched model is usable by id with no library release. knownModels is a convenience seed only — the live /v1/models listing is authoritative. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
bedrock-mantlevendor for Amazon Bedrock's OpenAI-compatible surface — the endpoint that resells third-party open-weight models (Z.ai GLM, Moonshot Kimi, xAI Grok, DeepSeek, MiniMax, Qwen, …) behind one endpoint and one Bedrock API key.Why
The jambonz hosted-LLM catalog resells these models. Today each provider needs its own alias (
zai,moonshot,xai) that differs from the others only in its defaultbaseURL— so onboarding a newly-launched Bedrock model requires a library release even though the wire protocol, endpoint, and credential are all identical.Because the model id carries the provider (
zai.glm-5,moonshotai.kimi-k2.5), one vendor covers the whole catalog. A new Bedrock model becomes a data change.This is not the existing
bedrockadapter, which speaks the native Converse API via the AWS SDK and supports SigV4/IAM. That one stays the right choice for Anthropic and Nova models.Notes
knownModelsis a deliberate convenience seed, not a source of truth —supportsModelListingis true and the live/v1/modelslisting is authoritative. Uncurated ids stream fine and surface fromlistAvailableModels()without metadata./v1/modelsnorGetFoundationModelreturns it), so the limits inknownModelsare carried over from the native-vendor manifests for the same underlying models. Vision follows Bedrock'''sinputModalities.prompt_cache_keyis not forwarded: Mantle fronts many third-party runtimes with differing param strictness and none are verified. Added to the gating test'''s deny list.normalizeAuthpassesapi_urlthrough, which the hosted path relies on — xAI models are served only on/openai/v1, and a401 "Berm is not enabled"means the wrong path rather than a missing entitlement.Verification
getManifest()includes the vendor, so api-server'''s/LlmVendors/manifestand the webapp add-LLM form pick it up with no changeVersion intentionally not bumped.
🤖 Generated with Claude Code