Skip to content

fix(guard): stop retrying requests after the response start timeout - #5857

Open
eersnington wants to merge 1 commit into
mainfrom
stack/fix-guard-stop-retrying-after-response-start-timeout-kymtrxox
Open

eersnington wants to merge 1 commit into
mainfrom
stack/fix-guard-stop-retrying-after-response-start-timeout-kymtrxox

Conversation

@eersnington

Copy link
Copy Markdown
Member

Stop retrying a request after guard.gateway_response_start_timeout. The actor already received the request, so a retry runs the action a second time.

sequenceDiagram
	participant C as Client
	participant G as Guard
	participant A as Actor
	C->>G: POST /action/prompt
	G->>A: ToEnvoyRequestStart
	Note over G: response start timeout (5 min)
	G--xC: 504 gateway_response_start_timeout
	Note over C,A: Before: guard and the client resent the request, so prompt ran twice
Loading
  • Both gateways start the response start timer after ToEnvoyRequestStart is handed to the tunnel. This timeout is always after delivery, so the result is unknown.
  • Removed gateway_response_start_timeout from the retryable codes in guard-core/src/utils.rs and in the client's lifecycle-errors.ts.
  • The caller now gets the existing 504. No new error code: a 504 already means "the result is unknown". guard.request_delivery_unconfirmed is not reused because gateway3 uses it with a different meaning.
  • The test engine accepts a gatewayResponseStartTimeoutMs option, so a driver test can use a 2 second timeout and check that the action runs once.
  • Not changed: tunnel_request_aborted, tunnel_message_timeout, tunnel_response_closed, and actor_stopped_while_waiting can also arrive after delivery. Fixing them needs guard to know the request phase.

@railway-app

railway-app Bot commented Oct 11, 2026

Copy link
Copy Markdown

This PR was not deployed automatically as @eersnington does not have access to the Railway project.

In order to get automatic PR deploys, please add @eersnington to your workspace on Railway.

@claude

claude Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Review

The change is small and correct: a response-start timeout happens after ToEnvoyRequestStart is delivered, so retrying can run a non-idempotent action twice. Dropping the code from both the guard and client retry lists is the right fix. The driver test checks the behavior end to end (runs === 1), and the PR description scopes out the other post-delivery codes.

Issues

  1. Stale docs. docs/actors/content/docs/clients/javascript.mdx:135 lists guard.gateway_response_start_timeout under "Requests can still return transient lifecycle or gateway errors. Retry once the actor is available again." That is no longer true, and the docs now tell users to do the retry this PR removes. Move it out of the retry list and say the result is unknown, so the action may have run. Check .claude/reference/docs-sync.md for other places that need updating.
  2. Remaining post-delivery retry codes. tunnel_request_aborted, tunnel_message_timeout, tunnel_response_closed and actor_stopped_while_waiting have the same double-execution risk. Please track a follow-up issue or todo.
  3. Coverage. The driver test runs only with native, bare and remote SQLite on the default gateway. pegboard-gateway3/src/http_stream/request.rs:317 also emits this error but is not exercised. No Rust unit test pins is_retryable_guard_http_error, so the code could be re-added unnoticed.

Minor

  • shared-engine.ts: pegboard is now always emitted, so confirm the engine config accepts pegboard: {}.
  • lifecycle-errors.test.ts now has no negative case. Asserting that gateway_response_start_timeout is not retryable would document the intent.

No security or performance concerns.

@eersnington
eersnington force-pushed the stack/fix-guard-stop-retrying-after-response-start-timeout-kymtrxox branch from c98adf5 to 8578dd6 Compare October 11, 2026 20:26
@eersnington
eersnington force-pushed the stack/fix-guard-stop-retrying-after-response-start-timeout-kymtrxox branch from 8578dd6 to 3794005 Compare October 12, 2026 11:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant