Scraper API: send waitFor as an object, read the target status from http_code

[jehrr]

Two defects in the Scraper API client

Measured 2026-09-23 against https://scraper.2captcha.com/tasks/sync.

1. waitFor was sent as a JSON-encoded string. _build_wait_for returned json.dumps({...}), following a docstring that said the API wanted a double-encoded string. The live API refuses that with HTTP 422 ScrapeParser: params.waitFor must be an object — and still bills the call ($0.0005). The same request with an object answers HTTP 200. So every --wait-text / --wait-element / --wait-state run was a paid exit 5. _build_wait_for now returns a dict (Optional[dict]), logged through json.dumps, and the docstring says what was measured.
2. The target status was read from status. The response is {"status": "success", "http_code": <int>, "headers": …, "body": …}: status is the API's own verdict string, http_code is the target site's HTTP code. The client passed "success" into detect_page_state, so a target 403/503 was never seen (a live rakuten run logged Upstream page status success). It now reads http_code, falling back to status only if that is an int.

Regression check

test_scraper_api_waitfor_is_an_object in smoke_test.py: drives the real parse_args() + fetch_html() with --wait-text, requests.post monkeypatched to capture the payload and return {"status":"success","http_code":403,"body":"<html></html>"}. Asserts waitFor is a dict and the status handed onward is the int 403.

Control: with scraper_api_client.py restored from origin/main (edit asserted to change the file) the suite went red on exactly those two checks and nothing else; with the fix it is green.

Live result

One call with the fixed client, key from the environment:

scraper_api_client.py --url "https://www.spinny.com/used-cars-in-delhi-ncr/s/" --wait-text Maruti --retries 0

→ API HTTP 200 (no 422), Upstream page status 200 (target HTTP code, from http_code), 1,875,638 bytes, real page title ("1553 Used Cars in Delhi …").

But 0 products parsed, exit 4. The HTML carries no /buy-used-cars/ links at all — the car grid had not been painted into what came back (the wait text "Maruti" occurs once in the document, outside any grid). So the waitFor 422 is fixed; whether this client can read Spinny's grid at all is a separate open question, not addressed here. Worth a follow-up with --wait-element on a grid selector.

Not changed

• No version bump; the CHANGELOG entry is under [Unreleased].
• The browser engines, the parser and detect_page_state are untouched — they already take an int status; they were just never given one.
• The 0-product result above is not addressed here.

🤖 Generated with Claude Code