Skip to content

docs(architecture): state the key-selection order the router actually uses - #431

Merged
argszero merged 1 commit into
mainfrom
docs/architecture-key-selection-order
Oct 23, 2026
Merged

argszero merged 1 commit into
mainfrom
docs/architecture-key-selection-order

Conversation

@argszero

Copy link
Copy Markdown
Owner

Summary

docs/architecture.md §4 (the "one call" data-flow block) listed the gateway's key-selection step
as healthy first → random → sticky reuse, which puts a fresh random pick ahead of the sticky
reuse. The router does the opposite: a sticky hit wins over a random draw.

This PR rewrites that one parenthetical to healthy first → sticky reuse → otherwise random, so
the documented order matches the order the code executes. No code changes, no behaviour changes,
no other line touched.

What the code actually does

src/router.rs::pick():

  1. filters the candidate keys down to the healthy ones (status on and not in cooldown);
  2. looks up the sticky entry for (user_id, model) and, when that key is still inside the healthy
    set, returns it immediately;
  3. only when there is no usable sticky entry does it draw healthy.choose(&mut rng).

The early return Some(*kid) sits above let mut rng = rand::thread_rng(), so a sticky hit
short-circuits the random draw rather than following it.

Carriers for the corrected order

Three independent readings agree, so this is not asserted on a single reading:

  1. The function body itself — the early return is above the RNG construction; the precedence
    cannot be reversed without deleting that branch.
  2. pick()'s own doc comment (Chinese in the source; translated here): "select key: sticky
    first (while still healthy), otherwise random from the healthy set." The function documents the
    precedence that the §4 sentence denied.
  3. router::tests::pick_prefers_sticky_key — marks a sticky key and then asserts that the
    very next pick returns it (assert_eq!(p1, 2)).

Honest note on the strength of that test: it demonstrates sticky reuse deterministically, but it
separates "sticky" from "random" only probabilistically. With two candidate keys, an implementation
ordered random-then-sticky would return the other key on the first pick about half the time, so one
run in two would pass by luck. The body and the doc comment are what make the precedence a fact
rather than a probability; the test is what turns a green suite red when someone reorders the
branches.

Why the sentence is load-bearing

A reader uses §4 to predict how calls are spread across the keys behind one model. As written, two
calls from the same user to the same model appear to land on independently random keys. In fact
they reuse one key until it becomes unhealthy — which is precisely what to check when one key
starts rate-limiting while its siblings sit idle.

This defect is in what ships

git show v0.7.33:docs/architecture.md (line 45) still carries the inverted order, so the released
artifact ships the wrong order. The correction matches code that has been unchanged since P0-B.

Scope

  • Only the parenthetical on the selection line changed (1 file, +1/−1).
  • The neighbouring steps in the same block were re-read against the code and stand as written:
    auth resolves the Bearer API key to a user; the balance pre-check is
    available = gift + permanent, rejected with 402 when ≤ 0 (src/gateway.rs:266-277); settle
    credits the sharer 90% and writes transactions + usage_records + keys.used.
  • The router.rs row in §3's module table and ui/README.md:337 both name sticky without
    asserting an order, so neither is touched.
  • No gate reads docs/architecture.md: the only include_str! of a docs file is
    docs/prototype/aitokenpool-console.html.

Related Issue

None. The repository has no open issue for this, and a one-line documentation correction does not
warrant opening one.

Changes

  • Documentation fix described above
  • Config/data-structure changes — none

Tests

  • cargo test — 579 passed, 0 failed
  • cargo fmt --check — clean
  • cargo clippy --all-targets — no warnings
  • New unit tests — not applicable, this PR changes no behaviour

Checklist

  • Branch name follows the convention (docs/…)
  • Commit message uses Conventional Commits (docs(architecture): …)
  • Single responsibility, minimal diff

@argszero
argszero merged commit 259c497 into main Oct 23, 2026
2 checks passed
@argszero
argszero deleted the docs/architecture-key-selection-order branch October 23, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant