web-radar

Live web data pipelines built on Bright Data + Claude, vibe-coded at the WeAreDevelopers Berlin 2026 Bright Data session.

Two commands, one skeleton: scrape the live web → let Claude read it → ship a report. No CSS selectors, no per-site parsers — pages are scraped as markdown through Bright Data's Web Unlocker (which handles Cloudflare, bot detection, and JS rendering), and Claude extracts structured data with a strict JSON schema, so every target page can look completely different.

spreads — FX broker spread comparison

broker spread pages + aggregators (targets.ts)
        │
        ▼
Bright Data Web Unlocker ── scrapeUrl(urls, {dataFormat: "markdown"})
        │
        ▼
Claude structured extraction ── {broker, symbol, spread, type, unit}
        │
        ▼
per-symbol comparison matrix ── output/spreads-<ts>.md + .json

pnpm radar spreads                          # default symbol set
pnpm radar spreads --symbols EURUSD,XAUUSD  # your symbols
pnpm radar spreads --dry                    # scrape only, save raw markdown (no Claude)

prospects — local-business agent-readiness scanner

Bright Data Discover API ── semantic search: "<niche> in <city>", intent-filtered
        │                    (directories & listicles excluded by intent)
        ▼
Web Unlocker ── scrape each business site as markdown
        │
        ▼
Claude scoring ── agent-readiness 0-100: booking? pricing? chat? stale content?
        │
        ▼
ranked prospect dossiers ── output/prospects-<city>-<date>/report.md

pnpm radar prospects --niche "fitness studio" --city Berlin --country de --top 8

serve — live scan console

pnpm radar serve   # → http://localhost:4600, press SCAN NOW

Local presentation surface: the latest styled report on the right, and on-demand pipeline reruns with the log streamed to the browser over SSE on the left.

The story

• docs/story.html — the one-page story of this build (open locally, or via the hosted artifact)
• docs/sample-report.html — a real spread scan from 2026-07-09 (hosted artifact)

Setup

pnpm install
cp .env.example .env.local   # add BRIGHTDATA_API_KEY + ANTHROPIC_API_KEY
pnpm radar spreads --dry     # smoke-test the Bright Data half alone

Requires Node ≥ 20. Built with @brightdata/sdk and @anthropic-ai/sdk (Claude Opus 4.8, structured outputs via Zod).

Notes

• Spread targets live in src/targets.ts — edit freely; extraction adapts to whatever the page publishes.
• Spreads are the published/typical values from broker pages, not streaming tick data — the standard source for competitor benchmarking.
• Discover + Unlocker calls are metered by Bright Data credits; a full run of either command costs a handful of requests.

License

MIT