Repository navigation
Fix the gateway's 404, route by caller protocol, upgrade Bifrost, and bound history memory - #51
Merged
Merged
Conversation
The gateway advertised that it accepts Anthropic Messages, OpenAI Chat Completions and OpenAI Responses from clients and passes each through to a matching upstream, converting only when it has to. With more than one provider configured it did not: provider selection was left entirely to Bifrost's model catalog, which has no idea which protocol the caller spoke. Every provider advertises the same `["*"]` model list, so that resolution collapsed onto whichever provider the catalog indexed first, and a Codex client was answered by an Anthropic upstream even when a Responses upstream was sitting in the queue. Reproduced against the real helper with three distinct upstreams: all three callers reached the Anthropic one, and the other two received nothing. GatewayProtocolRouter now makes that decision in CC Buddy and pins it by naming the provider in the request model, which Bifrost honours ahead of catalog resolution. Configure all three protocols and every caller passes through untouched; configure one or two and a caller whose protocol is missing is converted and served by the head of the queue. Pinning takes provider choice away from Bifrost's virtual-key balancer, which was also what provided failover, so the rest of the queue travels with each request as `fallbacks` and still takes over on failure. Model mapping now follows the upstream that actually serves the request rather than whichever provider happened to be marked active. Two smaller gateway defects went with it. Chat -> Responses conversion was gated on a non-empty generated model catalog, silently refusing Chat clients against any Responses provider whose models were not enumerated in advance; it now follows the configured protocol. An unversioned `/models` request reached Bifrost's management UI and came back as HTML instead of a catalog, and Bifrost's internal provider prefixes leaked into model lists. Bifrost moves from v1.6.11 to v2.1.1. Its Intel slice still declares macOS SDK 10.4 and still has to be corrected before notarization, but the correction no longer runs through `xcrun vtool`, whose whole-file re-emit could only be pinned by a digest produced on a Mac. normalize-macho-sdk.py edits the one SDK field in place: two bytes change, the digest is reproducible anywhere, and verify-bifrost.sh still confirms the result independently. A provider that publishes `/v1/models` or `/models` can now fill in its own model bindings from the provider editor. Because that endpoint is a convention rather than a contract, discovery distinguishes "this host has no listing endpoint" from "the request failed", and only the former greys the refresh control out. Memory: a large history library grew the app until it degraded the whole machine. The catalog kept one record per session for the life of the process, including per-message spans for every block of every document — hundreds of megabytes on a big library, never returned. Those spans are split into a bounded LRU that only search reads, while the light header rows the listing walks need stay resident. The usage record cache and the Qoder file cache now hand their contents back under system memory pressure, which nothing in the app previously listened for at all. The usage scan's deduplication table is released before the Codex pass instead of at the end of the scan, and the scope snapshot walks the catalog once rather than twice per refresh. CI: the Ubuntu job no longer builds and tests the legacy Tauri stack that 2.x does not ship, and no longer gates the macOS job that covers the app that does. The twelve-thousand-message preflight ran the most expensive UI test a second time for no added coverage. The legacy 1.3.9 updater contract scripts and their suites are removed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx
Shedding exactly one entry per insert meant paying for recency ordering on every insert once the cache was full — which is precisely what the first walk over a whole library does. Shedding down to a low water mark instead spreads that cost across many inserts. A cache small enough that a tenth rounds to nothing still sheds only what it must, so the bound stays exact for small limits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx
This is the 404. Bifrost owns the whole upstream path and inserts its own
version segment: its Anthropic transport posts to `<base>/v1/messages`, its
OpenAI transport to `<base>/v1/chat/completions`. Every provider, meanwhile,
documents a base URL that already ends where its own version begins, and the
preset catalog stores them that way — 71 of 72 presets end in `/v1`.
Handed over untouched, that asked every upstream for `/v1/v1/messages`.
Reproduced against the real helper with a GLM-shaped host: the upstream
received `/api/anthropic/v1/v1/messages` and answered 404 `{"detail":"Not
Found"}`, which is exactly what a user reported. The connection test did not
catch it because it computed its own URL and happened to compute the right
one, so a provider could test healthy and then fail every real request.
GatewayUpstreamURL reconciles the two in one place, used by both the
configuration builder and the probe so they cannot drift again:
* A base ending in exactly `/v1` has it removed, because Bifrost puts it
back. That is the complete fix for this shape, including the Responses
routes whose path carries a response id.
* A base carrying a version Bifrost cannot reproduce, such as `/v4` or
`/v1beta/openai`, is preserved and Bifrost is given
`request_path_overrides` so it does not prepend a second one.
* A base with no version segment keeps Bifrost's defaults, which are
already correct.
Verified end to end against bifrost-http. A GLM Anthropic base ending in
`/v1` goes from 404 to 200 with the upstream seeing `/api/anthropic/v1/
messages`; a `/api/paas/v4` OpenAI base goes from 404 to 200 with
`/api/paas/v4/chat/completions`; a base already without `/v1` is unchanged
and still 200. All 72 presets now resolve to a URL containing their version
segment exactly once, and to the same URL for the gateway and the probe.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx
CI caught three tests that encoded the behaviour the previous commit fixed. Two of them are the bug itself written down as an expectation. The probe used to try a second spelling of the endpoint and, when that answered, silently rewrite the user's stored base URL. That is the machinery that let the probe and the gateway disagree about whether a base already carried its version segment — the disagreement that made a provider test healthy and then answer every real request with 404. Now that each base URL maps to exactly one upstream URL, there is no second spelling to try, so the fallback and the base-URL migration come out entirely, along with `migratedBaseURL` and its two call sites. That changes what a refused endpoint means: it is reported as a failure instead of quietly succeeding somewhere else. Tests updated accordingly, and the integration test's expected `base_url` follows the same correction as the unit test in the previous commit. Model discovery kept its own slightly different answer to "does this base already carry a version segment", which is exactly the duplication that caused this. It now asks `GatewayUpstreamURL`, which also gets it right for a base like `/v1beta/openai` that discovery's copy classified wrongly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The 404: every upstream URL had its version segment twice
Bifrost builds the whole upstream path and inserts its own version segment. Its Anthropic transport posts to
<base>/v1/messages, its OpenAI transport to<base>/v1/chat/completions. Every provider, meanwhile, documents a base URL that already ends where its own version begins, and the preset catalog stores them that way: 71 of the 72 presets end in/v1.Handed over untouched, that asked every upstream for
/v1/v1/messages. Reproduced against the real helper with a GLM-shaped host, which answered{"detail":"Not Found"}— exactly what a user hit in practice.The connection test did not catch it, because it computed its own URL and happened to compute the right one. So a provider could test healthy and then fail every real request.
GatewayUpstreamURLreconciles the two in one place, used by the configuration builder, the probe and model discovery so they cannot drift again:/v1has it removed, because Bifrost puts it back. That is the complete fix for this shape, including the Responses routes whose path carries a response id./v4or/v1beta/openai, is preserved and Bifrost is givenrequest_path_overridesso it does not prepend a second one.Verified end to end against
bifrost-http:…/api/anthropic/v1/v1/v1/messages/api/anthropic/v1/messages…/api/anthropic…/api/paas/v4/v4/v1/chat/completions/api/paas/v4/chat/completionsThe probe now issues exactly one request, to exactly the address the gateway will call. Its second-spelling fallback and the silent base-URL rewriting that came with it are gone: that machinery is what let the two drift apart.
Provider selection ignored the caller's protocol
Separately, the gateway advertises that it accepts Anthropic Messages, OpenAI Chat Completions and OpenAI Responses and hands each to a matching upstream. With more than one provider configured it did not. Selection was left to Bifrost's model catalog, which has no idea which protocol the caller spoke, and every provider advertises the same
["*"]model list, so resolution collapsed onto whichever provider was indexed first.With three upstreams on three ports, one per protocol:
GatewayProtocolRoutermakes the decision in CC Buddy and pins it by naming the provider in the request model, which Bifrost honours ahead of catalog resolution. Configure all three protocols and every caller passes straight through, unconverted. Configure one or two and a caller whose protocol is present still passes through, while one whose protocol is missing is converted and served by the head of the queue. Pinning bypasses Bifrost's virtual-key balancer, which also provided failover, so the rest of the queue travels with each request asfallbacks; verified that a pinned provider returning 500 falls through to the next.Multi-provider routing follows the existing provider queue, which stays an explicit opt-in. The provider list is otherwise a one-at-a-time switcher, so three-of-three passthrough means three providers in the queue; the settings copy now says so.
Two smaller defects went with it. Chat to Responses conversion was gated on a non-empty generated model catalog, silently refusing Chat clients against any Responses provider whose models were not enumerated in advance. An unversioned
/modelsrequest reached Bifrost's management UI and came back as HTML, and Bifrost's internal provider names leaked into model lists.Bifrost v1.6.11 to v2.1.1
v2.1.1 is the newest published release. Its Intel slice still declares macOS SDK 10.4 and still has to be corrected before notarization, but the correction no longer runs through
xcrun vtool, whose whole-file re-emit could only ever be pinned by a digest produced on a Mac.normalize-macho-sdk.pyedits the one SDK field in place: two bytes change, the digest is reproducible on any platform, andverify-bifrost.shstill confirms the outcome independently.Model discovery
A provider that publishes
/v1/modelsor/modelscan now fill in its own model bindings from the provider editor. That endpoint is a convention rather than a contract, so discovery distinguishes "this host has no listing endpoint" from "the request failed", and only the former greys the refresh control out. Bindings the user wrote are never disturbed.Memory
A large history library grew the app until it degraded the whole machine. The catalog kept one record per session for the life of the process, including per-message spans for every block of every document.
CI
The Ubuntu job no longer builds and tests the legacy Tauri stack that 2.x does not ship, and no longer gates the macOS job that covers the app that does. It now finishes in about twenty seconds and runs alongside. The twelve-thousand-message preflight ran the most expensive UI test a second time for no added coverage. The legacy 1.3.9 updater contract scripts and their suites are removed.
Verification
CI is green: the native unit suite with every real end-to-end test, the UI suite, the Bifrost helper validator, and the unsigned universal package build with its packaged self-check, all against v2.1.1.
Because Swift cannot be built for macOS in the authoring environment, the routing, URL, cache and discovery logic was also compiled and executed against swift-corelibs-xctest on Linux: 31 assertions across four new suites, including one that checks all 72 presets resolve to a URL carrying their version exactly once, identically for the gateway and the probe. The protocol matrix, provider pinning, request fallbacks and the doubled-version 404 were each reproduced against the real
bifrost-httpbinary with strict mock upstreams that assert on the exact path.🤖 Generated with Claude Code
https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx