feat(agents): point an agent at an OpenAI-compatible endpoint you alr… - #92
Merged
Conversation
…eady run
MATE's control layer applied only to agents built inside MATE, so trying it
meant rebuilding your agents in it. That is the wrong price for a control
layer, and it is the likeliest reason the project collects stars without
collecting users.
An agent reached over /v1/chat/completions is, on the wire, the same shape
as a model, and create_model already routes openai/* through LiteLLM with a
custom base_url — it is how LM Studio and llama.cpp are already supported.
The only thing missing was somewhere to put the endpoint: create_model was
called with the model name alone, so base_url and api_key came from provider
env vars and every agent shared one endpoint per provider.
So this adds two columns rather than a proxy agent class. An external agent
is an ordinary agent, which means RBAC, guardrails, evals, versioning, audit,
the widget and cost tracking apply to it with no new code paths to keep in
sync. Both runtimes resolve the endpoint through one function so a config
means the same thing under either.
Two refusals are deliberate, and each is covered by a test that fails when
the behaviour is mutated away:
A ${VAR} that is not set refuses to build the agent instead of falling back.
Falling back would have LiteLLM read the provider's own key and send it to
whatever third-party host base_url names.
An endpoint configured with no key is given a placeholder rather than left
unset, for the same reason — keyless endpoints are normal on local servers
and private networks, and OPENAI_API_KEY should not follow them out.
Keys are meant to be written as ${VAR}. A literal one is stored in the row
and captured in version history, so the dashboard never sends it to the
browser: responses carry a sentinel, and saving the form with the sentinel
unchanged leaves the stored value alone. The copy form clears it rather than
showing the sentinel, which would have created a keyless agent silently.
A template carries the endpoint but never the credential, since a template
is a shared artifact. A config export round-trips both.
The ${VAR} resolver moved from mcp_tools to shared utils now that it has a
second consumer.
814 tests, OK. Migration V030 verified against a real database: applied,
columns present, and idempotent on a second run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ToCUNy2SqwvfTq4a6xfSk1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…eady run
MATE's control layer applied only to agents built inside MATE, so trying it meant rebuilding your agents in it. That is the wrong price for a control layer, and it is the likeliest reason the project collects stars without collecting users.
An agent reached over /v1/chat/completions is, on the wire, the same shape as a model, and create_model already routes openai/* through LiteLLM with a custom base_url — it is how LM Studio and llama.cpp are already supported. The only thing missing was somewhere to put the endpoint: create_model was called with the model name alone, so base_url and api_key came from provider env vars and every agent shared one endpoint per provider.
So this adds two columns rather than a proxy agent class. An external agent is an ordinary agent, which means RBAC, guardrails, evals, versioning, audit, the widget and cost tracking apply to it with no new code paths to keep in sync. Both runtimes resolve the endpoint through one function so a config means the same thing under either.
Two refusals are deliberate, and each is covered by a test that fails when the behaviour is mutated away:
A ${VAR} that is not set refuses to build the agent instead of falling back. Falling back would have LiteLLM read the provider's own key and send it to whatever third-party host base_url names.
An endpoint configured with no key is given a placeholder rather than left unset, for the same reason — keyless endpoints are normal on local servers and private networks, and OPENAI_API_KEY should not follow them out.
Keys are meant to be written as ${VAR}. A literal one is stored in the row and captured in version history, so the dashboard never sends it to the browser: responses carry a sentinel, and saving the form with the sentinel unchanged leaves the stored value alone. The copy form clears it rather than showing the sentinel, which would have created a keyless agent silently.
A template carries the endpoint but never the credential, since a template is a shared artifact. A config export round-trips both.
The ${VAR} resolver moved from mcp_tools to shared utils now that it has a second consumer.
814 tests, OK. Migration V030 verified against a real database: applied, columns present, and idempotent on a second run.
Claude-Session: https://claude.ai/code/session_01ToCUNy2SqwvfTq4a6xfSk1