BrandSCOPE – Brand Search Content Optimization & Publishing Engine

A 4-engine content pipeline that extracts a brand's DNA, tracks live industry and brand news, builds an SEO/AEO/GEO-structured brief, writes the article in the brand's voice, and delivers it — on a daily schedule with no human in the loop.

Free end to end: GitHub Actions, Gemini free tier, Groq free tier, Gmail SMTP, SQLite.

---

Table of Contents

• Overview
• The Four Engines
• Automation (GitHub Actions)
• Tech Stack
• Installation
• Configuration
• Running the Project
• Known Limits

---

Overview

BrandSCOPE runs as two layers over the same engine code:

Layer	What it does	Where it runs
Interactive UI	React app that runs each engine on demand, with review at every step	Locally (FastAPI + React)
Scheduled pipeline	backend/run_pipeline.py runs Engine 02 → 03 → 04 → email → commit, unattended	GitHub Actions cron, daily 09:00 IST

Every article is structured for:

• SEO — traditional search ranking signals
• AEO — Answer Engine Optimization (featured snippets, AI Overviews)
• GEO — Generative Engine Optimization (citation by ChatGPT, Perplexity, Claude)

---

The Four Engines

Engine 01 – Brand DNA Extractor
Crawls up to 60 pages with sitemap parsing + httpx BFS (depth 3), mines JSON-LD, renders with Playwright (accordion/tab expansion, shadow DOM, iframes, raw-HTML fallback), runs web searches, and synthesizes a structured brand profile with Gemini. Runs locally; its output is committed as the pipeline's input.

Engine 02 – Live Trend Research
Pulls Google News RSS and DuckDuckGo for industry and brand-specific news, then uses Gemini to classify each item into brand_news, brand_future, industry_trend, or competitive, write a brand trajectory summary, and generate 10–14 article angles. DuckDuckGo failures degrade gracefully to RSS only.

Engine 03 – Article Brief Builder
Rule-based (no LLM call): picks the primary keyword from brand keywords that overlap the angle, sets secondary keywords, audience, an outline per article type, CTA, and brand-reference rules.

Engine 04 – Article Writer
Writes the article in the brand's voice from the brief + DNA. Model routing with fallback: Gemini 3.5 Flash → 3.5 Flash Lite → Flash (latest alias) → Groq Llama 3.3 70B, with retry on 429/503. Post-processing repairs missing structure, a banned-phrase filter catches AI clichés, and a deterministic rubric scores SEO/AEO/GEO (0–100). Output: Markdown + publish-ready HTML with Article and FAQPage JSON-LD.

---

Automation (GitHub Actions)

Workflow: .github/workflows/daily-article.yml

cron 03:30 UTC (09:00 IST)  or  manual "Run workflow"
  → 01 load committed company_dna.json
  → 02 trend research        (Google News RSS + DDGS + Gemini)
  → 03 article brief         (rule-based)
  → 04 article writer        (Gemini → Groq fallback, scored)
  → 05 email delivery        (Gmail SMTP, HTML body + .md attachment)
  → commit article, brief, trends and run log back to the repo

• No browser in CI. Engines 02–04 need only backend/requirements-pipeline.txt. Engine 01 stays local.
• Run log. Every stage records run_id, stage, status, error, model_used, detail, started_at, finished_at to the SQLite runs table and to clients/{slug}/runs/run_log.jsonl, which is committed so history survives the fresh runner. Each run also writes a stage table to the Actions job summary.
• Failure handling. A failed stage marks later stages skipped, exits non-zero (red run), and the run log is still committed.
• Demo client. clients/sarvam-ai/ is the default client for scheduled runs. Other brands run on demand from the UI's Brand Autopilot (below); the commit step uses git add -f -- clients/<slug> so their outputs are committed even though clients/ is git-ignored.
• Run name. Manual runs show as BrandSCOPE run for <slug> in the Actions tab, which is also how the UI finds the run it dispatched.

Brand Autopilot (UI → Actions)

type "Zepto" in the UI
  → resolve website        (DuckDuckGo, Gemini fallback, verified by HTTP)
  → Engine 01 locally      (background job, live log in the UI)
  → Run Workflow           commits clients/<slug>/01_brand_dna/company_dna.json via the GitHub API
                           dispatches daily-article.yml with client=<slug>
  → live step status       polled from the Actions API
  → results                run-log stages + article read back from the repo

The UI never writes pipeline outputs to your local disk. Run git pull afterwards to get the committed files.

API calls per run: 2 LLM calls (1 in Engine 02, 1 in Engine 04) when the primary model answers, ~9 Google News RSS requests, ~8 DuckDuckGo queries, 1 SMTP send.

---

Tech Stack

Backend

Package	Purpose
FastAPI + Uvicorn	REST API server, also serves the built frontend
Playwright	Full-render browser scraping (Engine 01 only)
httpx	Async HTTP for BFS crawling, RSS, fallbacks
spaCy (en_core_web_sm)	Noun-chunk keyword extraction (Engine 01 only)
DDGS	DuckDuckGo web search, no API key
google-genai	Gemini — DNA synthesis, trend classification, article writing
Groq	Final LLM fallback
aiosqlite	Async SQLite for records and the run log
smtplib (stdlib)	Email delivery

Frontend: React 18, Vite 6, Framer Motion

Storage

Layer	What lives here
data/searchos.db	Clients, DNA, trend reports, briefs, articles + scores, runs
clients/{slug}/01_brand_dna/	company_dna.json
clients/{slug}/02_trend_research/	trends_{timestamp}.json
clients/{slug}/03_article_briefs/	brief_{timestamp}_{title}.json
clients/{slug}/04_articles/	{date}-{slug}.md and .html
clients/{slug}/runs/	run_log.jsonl

---

Installation

Requires Python 3.11+ and Node.js 18+.

git clone https://github.com/SupratikB23/BrandSCOPE.git
cd BrandSCOPE

pip install -r backend/requirements.txt
playwright install chromium
python -m spacy download en_core_web_sm

cd frontend
npm install
npm run build
cd ..

---

Configuration

cp backend/.env.example backend/.env

Variable	Required	Purpose
GOOGLE_API_KEY	Yes	Gemini — aistudio.google.com/app/apikey
GROQ_API_KEY	Recommended	Fallback LLM — console.groq.com
SMTP_USER / SMTP_PASS	For email	Gmail address + App Password
EMAIL_TO	No	Recipient(s), defaults to SMTP_USER
GEMINI_MODELS	No	Override the Gemini chain, comma-separated
DISABLE_DDGS	No	1 = skip DuckDuckGo, Google News RSS only
GITHUB_TOKEN	For Run Workflow	Fine-grained PAT for this repo: Contents Read and write + Actions Read and write. Local .env only, never a repo secret
GITHUB_REPO	No	owner/name, defaults to SupratikB23/BrandSCOPE

For GitHub Actions, add the LLM and SMTP values under Settings → Secrets and variables → Actions. Runs started from the UI use those secrets, including the email recipient.

---

Running the Project

UI (single process)

npm run server        # FastAPI on http://localhost:8000, serves frontend/dist

UI (hot reload) — cd backend && python run.py in one terminal, cd frontend && npm run dev in another (http://localhost:5173).

Headless pipeline (same as CI)

python backend/run_pipeline.py --client sarvam-ai            # 02 → 03 → 04 → email
python backend/run_pipeline.py --client sarvam-ai --no-email --type listicle

Run log: GET http://localhost:8000/api/runs

Brand Autopilot: open http://localhost:8000 → Open the Engine → type a brand name → Scrape brand → Run Workflow → git pull when the run finishes.

Endpoint	Purpose
POST /api/autopilot/scrape {query}	Start a brand-name scrape job
GET /api/autopilot/jobs/{id}	Job status + live log
GET /api/github/status	Token / repo / workflow check
POST /api/github/run-workflow {client_id, article_type}	Commit DNA + dispatch the workflow
GET /api/github/runs · GET /api/github/runs/{run_id}?client=<slug>	Recent runs · live status and outputs

---

Known Limits

• Engine 01 on the manual Brand DNA page still blocks its HTTP request for the whole crawl. Brand Autopilot runs it as an in-process background job instead (one scrape at a time, lost on server restart).
• Some sites block headless browsers. Heavily bot-protected or app-only brands may yield thin DNA; enter the website URL directly if the name resolves to the wrong site.
• SEO/AEO/GEO scores are heuristic. A deterministic rubric (keyword placement, headings, FAQ, stats, attributions) — not validated against real ranking or citation data.
• Generated statistics must be fact-checked before publishing; the model is instructed to cite sources but can invent them.
• Scheduled runs are best effort. GitHub may delay cron runs at peak load and disables schedules after 60 days without repository activity.
• DuckDuckGo rate-limits datacenter IPs. On CI, Engine 02 may run on Google News RSS alone.
• Not deployed as a web app. Playwright needs Chromium, so serverless hosts cannot run Engine 01; it needs a container.

---

Documentation

Architecture, API reference, database schema and pipeline details: DOCS.md

Architecture Diagram