Skip to content

Fix concurrency timeouts: add request throttle and retry backoff - #40

Open
hamsterkacke wants to merge 1 commit into
mandatoryprogrammer:mainfrom
hamsterkacke:fix/concurrency-throttle-and-retry-backoff
Open

Fix concurrency timeouts: add request throttle and retry backoff#40
hamsterkacke wants to merge 1 commit into
mandatoryprogrammer:mainfrom
hamsterkacke:fix/concurrency-throttle-and-retry-backoff

Conversation

@hamsterkacke

Copy link
Copy Markdown

Addresses #33 — concurrent requests causing timeouts, especially for basic page navigation GETs.

Root Cause

Every concurrent proxy request independently creates a CDP browser session and Chrome tab with no concurrency control. Under load, Chrome's DevTools server gets overwhelmed:

  • CDP WebSocket calls slow down across all connections
  • The fixed 30s timeout becomes too tight
  • All requests timeout nearly simultaneously
  • Retries fire immediately with zero delay, adding more load

This is worst for manual_browser_visit (simple GET navigation) because it has the tightest flow — one shot to intercept the response, with no room for CDP latency spikes.

Fix

Three changes, each targeting a different part of the cascade:

1. Concurrency semaphore (server.js)

Limits how many requests can use Chrome simultaneously (default: 3, configurable via MAX_CONCURRENT_REQUESTS env var). Excess requests queue instead of all fighting for Chrome at once. Actual throughput improves because Chrome can handle 3 requests efficiently rather than 10 requests poorly.

2. Retry backoff (cdp.js)

Replaces zero-delay retries with exponential backoff (1s → 2s → 4s, capped at 5s). When Chrome is struggling, immediate retries just make it worse. The backoff gives Chrome breathing room to recover.

3. Timeout race guard in manual_browser_visit (cdp.js)

The Fetch.requestPaused handler now checks the settled flag before making CDP calls. Previously, if the 30s timeout fired while the handler was mid-execution, the timeout would call Fetch.disable() and the handler would then try Fetch.getResponseBody() / Fetch.fulfillRequest() on a disabled domain, causing spurious errors.

Under concurrent load, every request independently creates a browser
session and tab, overwhelming Chrome's DevTools server and causing
cascading timeouts. Retries with zero delay amplify the problem by
immediately adding more load.

Three fixes:
- Add concurrency semaphore (default 3, configurable via
  MAX_CONCURRENT_REQUESTS) so requests queue instead of all
  fighting for Chrome simultaneously
- Add exponential backoff to CDP retries (1s, 2s, 4s capped at 5s)
  instead of retrying immediately, giving Chrome time to recover
- Guard Fetch.requestPaused handler in manual_browser_visit against
  timeout race where the handler runs CDP calls after Fetch.disable()
  has already been called by the timeout path

Ref: mandatoryprogrammer#33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant