Three inherited defects, found while auditing a new sibling repo - #7
Merged
Merged
Conversation
None of these is Etsy-specific; all three came from the shared core and
were found by auditing tokopedia-scraper against the family notes, then
checked here before being patched.
1. --fingerprint FAILED ON EVERY INVOCATION. The engines' --fp-tags
default was "Windows,Chrome,Desktop", and the fingerprint API rejects
it with HTTP 400 ("Request parameters are invalid") — while
fingerprint_client.py's own --tags help has always said ONE OS-family
tag, not a list, and that Chrome/Desktop/Mobile are each rejected. The
engines' default contradicted the client's documentation.
Measured against the live API on 2026-09-10:
Windows OK
Windows,Chrome,Desktop 400
Chrome 400
Desktop 400
This is the same root cause as the "documented --tags example returns
400" defect the family notes already record — that fix reached the
client's docstring and not the engines' default. Present in all four
repos in this family; each is being fixed on its own branch.
2. A COPIED .env.example READ AS CONFIGURED. The placeholder check was a
literal set, so it caught `your_2captcha_api_key_here` and missed the
two credentialled URLs, which .env.example documents the way the vendor
does — with the parts you fill in in braces:
ws://{login}-zone-scraping_browser-…-pid-{profileId}:{password}@…
http://{user}:{password}@na.proxy.2captcha.com:2334
So `cp .env.example .env` and a run connected to cb.2captcha.com with
the string `{login}-zone-…` as its username and got a 401 — a confusing
failure a long way from its cause, which is exactly what the credential
rule exists to prevent. Any `{…}` left in a value now reads as unset,
while ETSY_URL (no credential, a working default) stays usable.
mediamarkt-scraper has the same hole. amazon-scraper and
farfetch-scraper do not use braced placeholders and are unaffected —
checked rather than patched blind.
3. RETRY_ON_BLOCKED WAS A POLICY NOTHING ENFORCED. A constant with a
paragraph of justification that no engine read: all three computed
their budget from BLOCK_RETRIES_WITHOUT_POOL alone, so setting it False
changed nothing. Same defect as dead code that looks load-bearing. Now
consulted by all three.
Each of the three is pinned by a check, so none can come back quietly.
581 offline checks, all green; the credential grep is clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0193w6TZpMqmbfERNZAGdtv3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by auditing
tokopedia-scraperagainst the family notes, then verified here before patching.--fingerprintfailed on every invocation. The engines'--fp-tagsdefault wasWindows,Chrome,Desktop, which the fingerprint API rejects with HTTP 400 — whilefingerprint_client.py's own--tagshelp has always said ONE OS-family tag. Measured against the live API:Windowssucceeds;Windows,Chrome,Desktop,ChromeandDesktopeach 400. Present in all four repos in this family; each gets its own branch..env.exampleread as configured. The placeholder check was a literal set, so it missed the two credentialled URLs documented with{braces}.cp .env.example .envthen connected to cb.2captcha.com with{login}-zone-…as its username and got a 401.mediamarkt-scraperhas the same hole;amazonandfarfetchdo not use braced placeholders and are unaffected — checked, not patched blind.RETRY_ON_BLOCKEDwas a policy nothing enforced. No engine read it; all three computed their budget fromBLOCK_RETRIES_WITHOUT_POOLalone, so setting it False changed nothing.All three are pinned by checks. 581 offline checks green, credential grep clean.
🤖 Generated with Claude Code
https://claude.ai/code/session_0193w6TZpMqmbfERNZAGdtv3