feat(site): site analyze, site verify, and OpenCLI commands through its own runtime

[leeguooooo]

Borrowed from OpenCLI (jackwener/OpenCLI), and reusing its adapters directly rather than copying them.

site analyze [url]

Scans the page for what an adapter should read: same-site API calls the page made (Resource Timing, scored; analytics/ads dropped), state it embeds (__NEXT_DATA__, __INITIAL_STATE__, JSON script tags; devtools-extension globals dropped), anti-bot vendors (reuses the humanize markers), webpack bundles, and the adapters already installed. Recommends a strategy in breakage order: page fetch → page state → DOM, with next steps.

site verify <name>/<cmd> [args] [--write-fixture]

Runs the adapter, then compares the result's shape with ~/.chrome-use/site-fixtures/<name>/<cmd>.json: a missing field, a type change, or a list that came back empty fails with exit 1. --write-fixture records the shape (types only, no values). Works for our adapters and OpenCLI ones.

OpenCLI commands

When Node.js 20+ is on PATH, site update installs a pinned @jackwener/opencli (1.8.8; npm install --ignore-scripts, no token) into ~/.chrome-use/opencli. A name/cmd neither of our packs has runs through opencli_runner.mjs, which loads OpenCLI's own registry, argument coercion, pipeline executor and BasePage, with a page subclass whose transport is chrome-use (open, eval, cookies get, screenshot, tab), so it drives the caller's session and logins. site list marks them (opencli), site info shows their args, and the domain hint lists them after ours (separate .index-opencli.json, so ours win regardless of how each pack spells the domain). Ours win on a shared name. AGENT_BROWSER_SITES_NO_OPENCLI=1 turns it off; AGENT_BROWSER_OPENCLI_VERSION overrides the pin.

Verified (release build, real Chrome)

• OpenCLI: hackernews/best (public), zhihu/hot pipeline and bilibili/feed / bilibili/dynamic (logged-in, WBI signing via Node crypto) return data; zhihu/hot resolves to our adapter when both exist; site verify hackernews/best --write-fixture then site verify pass.
• site analyze https://www.v2ex.com finds /api/nodes/list.json → page-fetch.
• v2ex hint lists v2ex/hot, v2ex/latest, v2ex/topic before OpenCLI's.
• cargo test --release: 1481 passed.

Summary by CodeRabbit

• New Features
  • Added site analyze to inspect a page’s data sources and detected anti-bot vendors, with recommended next steps.
  • Added support for OpenCLI site commands, including command discovery and details through site list and site info.
  • Added site verify to check adapter results and optionally save a structural fixture for future checks.
• Documentation
  • Updated site-adapter guides with instructions for analysis, verification, and OpenCLI command support.

[coderabbitai]

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 1316dec5-8c0e-45c1-97e2-471b757363b6

📥 Commits

Reviewing files that changed from the base of the PR and between 31a2817 and ce18c2a.

📒 Files selected for processing (11)

• cli/src/commands.rs
• cli/src/main.rs
• cli/src/native/actions.rs
• cli/src/native/humanize.rs
• cli/src/opencli.rs
• cli/src/opencli_runner.mjs
• cli/src/output.rs
• cli/src/site.rs
• docs/en/site-adapters.html
• docs/site-adapters.html
• skill-data/core/references/site-adapters.md

 ________________________________________________________________________
< Brb...inventing the time machine to fix your code before you wrote it. >
 ------------------------------------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}