Skip to content

Honor stated retry times (add-retry-after) - #115

Merged
gschaetz merged 2 commits into
mainfrom
feat/retry-after
Oct 16, 2026
Merged

gschaetz merged 2 commits into
mainfrom
feat/retry-after

Conversation

@gschaetz

Copy link
Copy Markdown
Owner

Implements the spec merged in #114 (add-retry-after), step 1 of Replacing ModelRelay. No release is cut here.

Behavior

  • A 429 or 503 whose error body states a retry time ("Please try again in 7.66s", retryDelay, retry_after, ...) cools the endpoint at once for exactly that long, clamped to 1 second .. [health] max_retry_after_secs (new, default 300, validated 1 to 86400). It does not wait for failure_threshold, does not double, is not held to max_cooldown_secs, and does not count as a trip, so a later unannounced failure streak is not escalated by it.
  • Anything unreadable, or any other status, behaves exactly as before. Only the first 4 KiB of the body is read, a bounded duration is the only output, and nothing from the body is logged or stored.
  • One seam, retry_after::from_body, where the Retry-After header joins once Switchyard exposes it (fix(server): preserve safe headers on terminal upstream errors NVIDIA-NeMo/Switchyard#904, which I added a use case to).
  • Visible: cooldown_source (retry_after or failures) in GET /v1/health and route/explain; retry_after_ms on decision attempts (inside the existing attempts column, no schema change); counter humpyard_retry_after_honored_total{provider,source}.
  • Health still never denies service: a retry time only cools endpoints, and the fail-open pass is unchanged (covered by a test).

Tests

Parser unit tests (phrasings, JSON fields, units and combinations, bare numbers, junk, zero and negative, a day, a body past 4 KiB, a multibyte character at the limit), breaker unit tests (exact duration, no doubling or cap, late failures, failed probe, no escalation, disabled breaker), and 9 end-to-end tests with a controllable clock: cooled at once for the stated time and probed exactly then, the cap, the one-second floor, 503, hint-less failures unchanged, a 500 mentioning a time ignored, metrics and decision fields, fail-open, config validation. The architecture-rules test gains the new module. Local gates: fmt, clippy -D warnings, 452 tests, cargo deny.

🤖 Generated with Claude Code

gschaetz and others added 2 commits October 15, 2026 22:53
A 429 or 503 whose error body states when the endpoint will serve again cools
the endpoint for exactly that long, at once, clamped to 1 s to
[health] max_retry_after_secs (default 300). Read from the body for a few
well-known phrasings until Switchyard exposes Retry-After (Switchyard#904).
Cooldown source in /v1/health and explain, retry_after_ms on decision
attempts, humpyard_retry_after_honored_total. Amends ADR 0018.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gary Schaetz <gary@schaetzkc.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gary Schaetz <gary@schaetzkc.com>
@gschaetz
gschaetz merged commit a33c5e9 into main Oct 16, 2026
9 checks passed
@gschaetz
gschaetz deleted the feat/retry-after branch October 16, 2026 03:59
@gschaetz gschaetz mentioned this pull request Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant