[sachalmalick]

Summary

Adds the Global Frontend for the Kiro Global Router: an OpenAI-compatible HTTP service that forwards each request to a pool frontend picked by the Global Router. It reuses Dynamo's HttpService unchanged and needs no changes to dynamo-llm.

dynamo-global-ingress (lib/global-ingress, binary global-ingress)

• PoolForwardingEngine implements the chat and completions engines. It asks a PoolSelector for a pool, sends the request to that pool frontend's /v1/... route, and decodes the pool's SSE with the existing SseLineCodec. One instance serves every model, because it reads the model from each request. The Responses and Anthropic APIs translate to chat before the engine, so they forward too.
• ModelSync registers the engine through ModelManager's in-process path for each model in the pool catalog. It removes models no pool serves and turns endpoints on or off.
• main.rs takes pools as --pool <site>/<namespace>/<DGD name>=<model>@<frontend URL>[,...].

dynamo-global-router (lib/global-router, no HTTP code)

• PoolCatalog publishes pool snapshots through a watch channel. StaticPoolCatalog backs it for now; a catalog backed by the Global Discovery Service and Pool Relay state can implement the same trait.
• PoolSelector with FirstPoolSelector, a stand-in until round-robin.

Forwarding behaviour

• No tokenization at the Global Frontend. The pool frontend preprocesses as usual.
• The HTTP handler always consumes the engine as a stream and folds it into one response for non-streaming clients. So the engine always asks the pool to stream, and adds stream_options.include_usage for non-streaming clients so the folded response keeps its usage.
• The handler moves x-dynamo-session-id into the request context, so the engine puts the header back on the forwarded request.
• An error before the stream starts comes back as HttpError, so the client gets the pool's status (404 stays 404). An unreachable pool returns 503. An {"error": ...} frame mid-stream ends the client's stream with an error.
• SseLineCodec consumes [DONE] without emitting it, so a pool that closes its body cleanly before [DONE] looks finished. Connection resets still surface as errors.

Ownership: claims lib/global-ingress, lib/global-router, lib/global-discovery-service and lib/global-router-shared for @ai-dynamo/dynamo-global-router-codeowners. The last two were unclaimed, which failed the strict coverage check.

Base branch: this PR targets sachalmalick/global-router-v2-base, a push of the local global-router-v2 stack (Global Discovery Service and client, GlobalViewReceiver, global-router-shared). That stack isn't on origin/global-router-v2 yet. Once it is, switch this PR's base to global-router-v2.

Not in this PR: the catalog backed by the Global Discovery Service (needs GlobalDiscoveryClient::list_pools), round-robin selection, retrying the next pool before the first byte (TODO in forward.rs), per-model metric series, x-request-id/trace propagation, and the non-chat APIs.

Validation

• cargo test -p dynamo-global-ingress -p dynamo-global-router: 15 passed (6 unit, 9 end-to-end). The end-to-end tests run a Global Frontend in front of a real Dynamo pool frontend serving the echo engine, and in front of a scripted axum pool. They cover:
  • chat in streaming and non-streaming mode, completions, and /v1/responses;
  • models following catalog changes;
  • a pool's 404 passed through, and an unreachable pool returning 503;
  • session header forwarding, dropping pool keep-alives, and a mid-stream pool error.
• cargo clippy -p dynamo-global-ingress -p dynamo-global-router --all-targets: no warnings in these crates. cargo fmt applied.
• Ran the binary by hand:
  • --pool with a model name containing / parses;
  • /v1/models lists the model;
  • chat to an unreachable pool returns 503;
  • SIGTERM exits 0.
• build_codeowners.py --strict: 100% coverage. CODEOWNERS regenerated with emit_codeowners.py.
• Built on macOS with dynamo-llm default-features = false (no block manager).

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.