PageLode turns web pages into clean Markdown.
Give it a URL and it returns the useful content, page title, internal links, and a record of how the page was loaded. It begins with a fast direct request and only opens a browser when the page actually needs one.
Many web pages can be downloaded directly. Others need JavaScript, and some place a browser check in front of their content. Using a full browser for every request is slow and expensive, so PageLode uses a simple waterfall:
- Load the page with a browser-like HTTP client.
- If JavaScript is required, render it with Rod and Chromium.
- If the page appears blocked, retry it with Patchright.
- Remove navigation, advertising, and other clutter.
- Return readable Markdown and useful metadata.
PageLode keeps the main service in Go. A small TypeScript worker exists only for Patchright, whose browser tooling is built for the JavaScript ecosystem.
You need Go, Bun, and Chromium installed.
make setup
make build
./bin/pagelode example.comPageLode prints the cleaned page as Markdown.
For the complete result, including the loader attempts:
./bin/pagelode --json example.com./bin/pagelode serveThe API starts at http://localhost:8083. Send it a page:
Send it a page:
curl -sS http://localhost:8083/extract \
-H 'content-type: application/json' \
-d '{"url":"https://example.com"}'The response contains:
content: the page as clean Markdowntitle: the page titlelinks: same-site links found on the pageprovider: the loader that succeededattempts: what PageLode tried and why it escalated
docker compose up --buildThe container starts PageLode in service mode.
PageLode currently supports HTML and text pages. It detects common block pages and JavaScript-only shells, limits response sizes and concurrency, and remembers the best loader for recently visited domains.
It does not guarantee access to protected websites. Some sites require suitable proxies, authenticated sessions, or permission from the site owner. Use PageLode responsibly and follow applicable terms, robots policies, and laws.
make check
make smokemake check runs the Go tests, race detector, vet, TypeScript checks, and browser-worker tests. make smoke verifies all three loading paths against local fixtures.
MIT