Fix concurrency timeouts: add request throttle and retry backoff - #40
Open
hamsterkacke wants to merge 1 commit into
Open
Conversation
Under concurrent load, every request independently creates a browser session and tab, overwhelming Chrome's DevTools server and causing cascading timeouts. Retries with zero delay amplify the problem by immediately adding more load. Three fixes: - Add concurrency semaphore (default 3, configurable via MAX_CONCURRENT_REQUESTS) so requests queue instead of all fighting for Chrome simultaneously - Add exponential backoff to CDP retries (1s, 2s, 4s capped at 5s) instead of retrying immediately, giving Chrome time to recover - Guard Fetch.requestPaused handler in manual_browser_visit against timeout race where the handler runs CDP calls after Fetch.disable() has already been called by the timeout path Ref: mandatoryprogrammer#33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #33 — concurrent requests causing timeouts, especially for basic page navigation GETs.
Root Cause
Every concurrent proxy request independently creates a CDP browser session and Chrome tab with no concurrency control. Under load, Chrome's DevTools server gets overwhelmed:
This is worst for
manual_browser_visit(simple GET navigation) because it has the tightest flow — one shot to intercept the response, with no room for CDP latency spikes.Fix
Three changes, each targeting a different part of the cascade:
1. Concurrency semaphore (
server.js)Limits how many requests can use Chrome simultaneously (default: 3, configurable via
MAX_CONCURRENT_REQUESTSenv var). Excess requests queue instead of all fighting for Chrome at once. Actual throughput improves because Chrome can handle 3 requests efficiently rather than 10 requests poorly.2. Retry backoff (
cdp.js)Replaces zero-delay retries with exponential backoff (1s → 2s → 4s, capped at 5s). When Chrome is struggling, immediate retries just make it worse. The backoff gives Chrome breathing room to recover.
3. Timeout race guard in
manual_browser_visit(cdp.js)The
Fetch.requestPausedhandler now checks thesettledflag before making CDP calls. Previously, if the 30s timeout fired while the handler was mid-execution, the timeout would callFetch.disable()and the handler would then tryFetch.getResponseBody()/Fetch.fulfillRequest()on a disabled domain, causing spurious errors.