Repository navigation
Count every page view and crawler fetch without cookies; audit Cloudflare - #58
Merged
Merged
Conversation
…lare GA4 only sees visitors who accept cookies, and nothing recorded whether Googlebot, Bingbot or AI crawlers were fetching pages while both engines show one page indexed. The middleware now adds one to a daily D1 counter per page, status, client kind (named crawler or bucket), referring host and country, in waitUntil, never throwing. No cookie, IP or User-Agent stored, so no consent is needed; /privacy/ says so. npm run traffic reports crawlers per page, human referrers (AI assistants flagged), and views vs affiliate clicks per page. npm run cf:audit fetches pages as each crawler and, with a read-only token, reads Bot Fight Mode, Block AI bots, managed robots.txt, security level, Crawler Hints and every zone rule, then ranks what could block a crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CBB8KT6Paf7tCUjTvaanBB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cookieless D1 page/crawler counter in the middleware (schema/006, apply once), npm run traffic report, npm run cf:audit for dashboard-only settings that block crawlers. Privacy page updated. Build green, npm test 384/384.
🤖 Generated with Claude Code
https://claude.ai/code/session_01CBB8KT6Paf7tCUjTvaanBB
Generated by Claude Code