[sachalmalick]

Summary

Round-robin routing for the Global Frontend, over pools read from the Global Discovery Service. Stacked on #17272.

RoundRobinSelector (lib/global-router/src/selector.rs)

• Rotates through the pools serving a model, then through the chosen pool's frontends, so every pool gets an equal share however many frontends it has.
• Reuses the runtime's dynamo_runtime::pipeline::BuiltinRoutePicker unchanged, the same way dynamo-llm's kv_router/routing_host/builtin.rs does. There is one cursor per model and one per pool. Candidate positions stand in for worker IDs, since round robin only counts.
• The load-aware task can use the same picker's least_loaded() and power_of_two_choices(), passing a load function.
• Replaces FirstPoolSelector.

DiscoveryPoolCatalog (lib/global-router/src/discovery.rs)

• Polls GET /pools once at startup, then on an interval.
• Skips pools without a model (the Kubernetes stores report only names) and pools with invalid keys.
• Sorts pools by ID, so an unchanged list compares equal and round robin keeps its place.
• Publishes only when the pools change, and keeps the last pools through a discovery outage.

GlobalDiscoveryClient::pools() (lib/global-discovery-client)

• The Global Router's call. It returns a Snapshot<PoolListing> and maps the service's complete flag to Completeness.
• Like routers(), one bad entry fails the whole call. Both now share one body-decoding helper.

global-ingress

• Takes --discovery-url (with --discovery-interval-secs) as the alternative to --pool. A clap argument group requires exactly one of the two.
• Routes round robin.
• The shared test helpers moved to tests/common/mod.rs.

Not in this PR: a pool counts as routable whenever the discovery service lists it. Gating on live Pool Relay state streams (GlobalViewReceiver) waits for a Relay-to-router stream transport, which doesn't exist yet. PoolSelector doesn't change when it lands.

Validation

• cargo test -p dynamo-global-ingress -p dynamo-global-router -p dynamo-global-discovery-client: 29 passed. The new ones:
  • Selector unit tests: pools rotate, then their frontends; models rotate independently; pools without frontends are skipped.
  • Catalog unit tests: filtering and ordering; following a fake service through changes and an outage without publishing an unchanged list.
  • Client tests: decoding GET /pools, including a Kubernetes-style entry with no model; an estimated (partial) list; one bad frontend URL failing the whole call.
  • End-to-end test against the real dynamo-global-discovery-service (memory store, in-process). Two registered pools each get 2 of 4 requests. A third pool registered later joins the rotation without a restart, giving 3/3/1 after 3 more requests.
• cargo clippy --all-targets on the three crates: no warnings in them. cargo fmt applied.
• Ran the binaries by hand:
  • started global-discovery-service, registered a pool with curl, and started global-ingress --discovery-url;
  • /v1/models listed the pool's model, and chat reached the pool (503, because its frontend was a closed port);
  • SIGTERM exited 0;
  • giving neither --discovery-url nor --pool, or both, is rejected with the argument group's error.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.