Skip to content

Fix the gateway's 404, route by caller protocol, upgrade Bifrost, and bound history memory - #51

Merged
loadchange merged 4 commits into
mainfrom
claude/happy-bohr-m85n21
Sep 11, 2026
Merged

loadchange merged 4 commits into
mainfrom
claude/happy-bohr-m85n21

Conversation

@loadchange

@loadchange loadchange commented Sep 11, 2026 •

Copy link
Copy Markdown
Collaborator

The 404: every upstream URL had its version segment twice

Bifrost builds the whole upstream path and inserts its own version segment. Its Anthropic transport posts to <base>/v1/messages, its OpenAI transport to <base>/v1/chat/completions. Every provider, meanwhile, documents a base URL that already ends where its own version begins, and the preset catalog stores them that way: 71 of the 72 presets end in /v1.

Handed over untouched, that asked every upstream for /v1/v1/messages. Reproduced against the real helper with a GLM-shaped host, which answered {"detail":"Not Found"} — exactly what a user hit in practice.

The connection test did not catch it, because it computed its own URL and happened to compute the right one. So a provider could test healthy and then fail every real request.

GatewayUpstreamURL reconciles the two in one place, used by the configuration builder, the probe and model discovery so they cannot drift again:

  • A base ending in exactly /v1 has it removed, because Bifrost puts it back. That is the complete fix for this shape, including the Responses routes whose path carries a response id.
  • A base carrying a version Bifrost cannot reproduce, such as /v4 or /v1beta/openai, is preserved and Bifrost is given request_path_overrides so it does not prepend a second one.
  • A base with no version segment keeps Bifrost's defaults, which are already correct.

Verified end to end against bifrost-http:

base URL before after
…/api/anthropic/v1 404, upstream saw /v1/v1/messages 200, /api/anthropic/v1/messages
…/api/anthropic 200 200, unchanged
…/api/paas/v4 404, upstream saw /v4/v1/chat/completions 200, /api/paas/v4/chat/completions

The probe now issues exactly one request, to exactly the address the gateway will call. Its second-spelling fallback and the silent base-URL rewriting that came with it are gone: that machinery is what let the two drift apart.

Provider selection ignored the caller's protocol

Separately, the gateway advertises that it accepts Anthropic Messages, OpenAI Chat Completions and OpenAI Responses and hands each to a matching upstream. With more than one provider configured it did not. Selection was left to Bifrost's model catalog, which has no idea which protocol the caller spoke, and every provider advertises the same ["*"] model list, so resolution collapsed onto whichever provider was indexed first.

With three upstreams on three ports, one per protocol:

caller upstream reached, before upstream reached, after
Anthropic Messages Anthropic Anthropic
OpenAI Chat Completions Anthropic OpenAI Chat
OpenAI Responses Anthropic OpenAI Responses

GatewayProtocolRouter makes the decision in CC Buddy and pins it by naming the provider in the request model, which Bifrost honours ahead of catalog resolution. Configure all three protocols and every caller passes straight through, unconverted. Configure one or two and a caller whose protocol is present still passes through, while one whose protocol is missing is converted and served by the head of the queue. Pinning bypasses Bifrost's virtual-key balancer, which also provided failover, so the rest of the queue travels with each request as fallbacks; verified that a pinned provider returning 500 falls through to the next.

Multi-provider routing follows the existing provider queue, which stays an explicit opt-in. The provider list is otherwise a one-at-a-time switcher, so three-of-three passthrough means three providers in the queue; the settings copy now says so.

Two smaller defects went with it. Chat to Responses conversion was gated on a non-empty generated model catalog, silently refusing Chat clients against any Responses provider whose models were not enumerated in advance. An unversioned /models request reached Bifrost's management UI and came back as HTML, and Bifrost's internal provider names leaked into model lists.

Bifrost v1.6.11 to v2.1.1

v2.1.1 is the newest published release. Its Intel slice still declares macOS SDK 10.4 and still has to be corrected before notarization, but the correction no longer runs through xcrun vtool, whose whole-file re-emit could only ever be pinned by a digest produced on a Mac. normalize-macho-sdk.py edits the one SDK field in place: two bytes change, the digest is reproducible on any platform, and verify-bifrost.sh still confirms the outcome independently.

Model discovery

A provider that publishes /v1/models or /models can now fill in its own model bindings from the provider editor. That endpoint is a convention rather than a contract, so discovery distinguishes "this host has no listing endpoint" from "the request failed", and only the former greys the refresh control out. Bindings the user wrote are never disturbed.

Memory

A large history library grew the app until it degraded the whole machine. The catalog kept one record per session for the life of the process, including per-message spans for every block of every document.

  • Those spans move into a bounded LRU that only search reads. The light header rows the listing walks need stay resident.
  • The usage record cache and the Qoder file cache hand their contents back under system memory pressure, which nothing in the app previously listened for at all.
  • The usage scan's deduplication table is released before the Codex pass rather than at the end of the scan.
  • The scope snapshot walks the catalog once per refresh instead of twice.

CI

The Ubuntu job no longer builds and tests the legacy Tauri stack that 2.x does not ship, and no longer gates the macOS job that covers the app that does. It now finishes in about twenty seconds and runs alongside. The twelve-thousand-message preflight ran the most expensive UI test a second time for no added coverage. The legacy 1.3.9 updater contract scripts and their suites are removed.

Verification

CI is green: the native unit suite with every real end-to-end test, the UI suite, the Bifrost helper validator, and the unsigned universal package build with its packaged self-check, all against v2.1.1.

Because Swift cannot be built for macOS in the authoring environment, the routing, URL, cache and discovery logic was also compiled and executed against swift-corelibs-xctest on Linux: 31 assertions across four new suites, including one that checks all 72 presets resolve to a URL carrying their version exactly once, identically for the gateway and the probe. The protocol matrix, provider pinning, request fallbacks and the doubled-version 404 were each reproduced against the real bifrost-http binary with strict mock upstreams that assert on the exact path.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx

The gateway advertised that it accepts Anthropic Messages, OpenAI Chat
Completions and OpenAI Responses from clients and passes each through to a
matching upstream, converting only when it has to. With more than one
provider configured it did not: provider selection was left entirely to
Bifrost's model catalog, which has no idea which protocol the caller spoke.
Every provider advertises the same `["*"]` model list, so that resolution
collapsed onto whichever provider the catalog indexed first, and a Codex
client was answered by an Anthropic upstream even when a Responses upstream
was sitting in the queue. Reproduced against the real helper with three
distinct upstreams: all three callers reached the Anthropic one, and the
other two received nothing.

GatewayProtocolRouter now makes that decision in CC Buddy and pins it by
naming the provider in the request model, which Bifrost honours ahead of
catalog resolution. Configure all three protocols and every caller passes
through untouched; configure one or two and a caller whose protocol is
missing is converted and served by the head of the queue. Pinning takes
provider choice away from Bifrost's virtual-key balancer, which was also
what provided failover, so the rest of the queue travels with each request
as `fallbacks` and still takes over on failure. Model mapping now follows
the upstream that actually serves the request rather than whichever
provider happened to be marked active.

Two smaller gateway defects went with it. Chat -> Responses conversion was
gated on a non-empty generated model catalog, silently refusing Chat
clients against any Responses provider whose models were not enumerated in
advance; it now follows the configured protocol. An unversioned `/models`
request reached Bifrost's management UI and came back as HTML instead of a
catalog, and Bifrost's internal provider prefixes leaked into model lists.

Bifrost moves from v1.6.11 to v2.1.1. Its Intel slice still declares macOS
SDK 10.4 and still has to be corrected before notarization, but the
correction no longer runs through `xcrun vtool`, whose whole-file re-emit
could only be pinned by a digest produced on a Mac. normalize-macho-sdk.py
edits the one SDK field in place: two bytes change, the digest is
reproducible anywhere, and verify-bifrost.sh still confirms the result
independently.

A provider that publishes `/v1/models` or `/models` can now fill in its own
model bindings from the provider editor. Because that endpoint is a
convention rather than a contract, discovery distinguishes "this host has
no listing endpoint" from "the request failed", and only the former greys
the refresh control out.

Memory: a large history library grew the app until it degraded the whole
machine. The catalog kept one record per session for the life of the
process, including per-message spans for every block of every document —
hundreds of megabytes on a big library, never returned. Those spans are
split into a bounded LRU that only search reads, while the light header
rows the listing walks need stay resident. The usage record cache and the
Qoder file cache now hand their contents back under system memory
pressure, which nothing in the app previously listened for at all. The
usage scan's deduplication table is released before the Codex pass instead
of at the end of the scan, and the scope snapshot walks the catalog once
rather than twice per refresh.

CI: the Ubuntu job no longer builds and tests the legacy Tauri stack that
2.x does not ship, and no longer gates the macOS job that covers the app
that does. The twelve-thousand-message preflight ran the most expensive UI
test a second time for no added coverage. The legacy 1.3.9 updater contract
scripts and their suites are removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx
Shedding exactly one entry per insert meant paying for recency ordering on
every insert once the cache was full — which is precisely what the first
walk over a whole library does. Shedding down to a low water mark instead
spreads that cost across many inserts. A cache small enough that a tenth
rounds to nothing still sheds only what it must, so the bound stays exact
for small limits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx
This is the 404. Bifrost owns the whole upstream path and inserts its own
version segment: its Anthropic transport posts to `<base>/v1/messages`, its
OpenAI transport to `<base>/v1/chat/completions`. Every provider, meanwhile,
documents a base URL that already ends where its own version begins, and the
preset catalog stores them that way — 71 of 72 presets end in `/v1`.

Handed over untouched, that asked every upstream for `/v1/v1/messages`.
Reproduced against the real helper with a GLM-shaped host: the upstream
received `/api/anthropic/v1/v1/messages` and answered 404 `{"detail":"Not
Found"}`, which is exactly what a user reported. The connection test did not
catch it because it computed its own URL and happened to compute the right
one, so a provider could test healthy and then fail every real request.

GatewayUpstreamURL reconciles the two in one place, used by both the
configuration builder and the probe so they cannot drift again:

  * A base ending in exactly `/v1` has it removed, because Bifrost puts it
    back. That is the complete fix for this shape, including the Responses
    routes whose path carries a response id.
  * A base carrying a version Bifrost cannot reproduce, such as `/v4` or
    `/v1beta/openai`, is preserved and Bifrost is given
    `request_path_overrides` so it does not prepend a second one.
  * A base with no version segment keeps Bifrost's defaults, which are
    already correct.

Verified end to end against bifrost-http. A GLM Anthropic base ending in
`/v1` goes from 404 to 200 with the upstream seeing `/api/anthropic/v1/
messages`; a `/api/paas/v4` OpenAI base goes from 404 to 200 with
`/api/paas/v4/chat/completions`; a base already without `/v1` is unchanged
and still 200. All 72 presets now resolve to a URL containing their version
segment exactly once, and to the same URL for the gateway and the probe.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx
CI caught three tests that encoded the behaviour the previous commit fixed.
Two of them are the bug itself written down as an expectation.

The probe used to try a second spelling of the endpoint and, when that
answered, silently rewrite the user's stored base URL. That is the machinery
that let the probe and the gateway disagree about whether a base already
carried its version segment — the disagreement that made a provider test
healthy and then answer every real request with 404. Now that each base URL
maps to exactly one upstream URL, there is no second spelling to try, so the
fallback and the base-URL migration come out entirely, along with
`migratedBaseURL` and its two call sites.

That changes what a refused endpoint means: it is reported as a failure
instead of quietly succeeding somewhere else. Tests updated accordingly, and
the integration test's expected `base_url` follows the same correction as
the unit test in the previous commit.

Model discovery kept its own slightly different answer to "does this base
already carry a version segment", which is exactly the duplication that
caused this. It now asks `GatewayUpstreamURL`, which also gets it right for
a base like `/v1beta/openai` that discovery's copy classified wrongly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CKKXBtwRphz6HboCRWQTkx
@loadchange loadchange changed the title Route gateway traffic by the caller's protocol, upgrade Bifrost, and bound history memory Fix the gateway's 404, route by caller protocol, upgrade Bifrost, and bound history memory Sep 11, 2026
@loadchange
loadchange merged commit 3a987b2 into main Sep 11, 2026
2 checks passed
@loadchange
loadchange deleted the claude/happy-bohr-m85n21 branch September 11, 2026 07:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants