Skip to content

RepoGraph-PRD-Full.md

Latest commit

 

History

History
724 lines (526 loc) · 46.2 KB

RepoGraph-PRD-Full.md

File metadata and controls

724 lines (526 loc) · 46.2 KB
Raw

RepoGraph — Product Requirements Document (Full Export)

One file to understand the whole product. Give this to an engineer, designer, investor, or new teammate and they'll get it.

Meta Details
Product RepoGraph — Progressive Context Layer for GitHub
Version 4.1 Consolidated (Human Edition)
Status Active — Surgical MVP built, Gate 0 passed
Last Updated September 2, 2026
Sources Consolidated Documents/Obsidian Vault/02-Projects/RepoGraph Side Project.md (Validated PRD v2.0) + repograph/PRD.md v4.0 + ARCHITECTURE.md + README.md + RepoGraph Progress.md + RepoGraph — PRD Reconciliation & Validation.md
Maintainer RepoGraph Team
License Core: AGPLv3 / FSL — Enterprise (/ee): Commercial — See §16

How to use this doc

This isn't a dump of old PRDs stitched together. We took three conflicting PRD versions and the actual code in repograph/ and reconciled them into one story. If you only read one section, read §1 and §4. If you want to build, jump to §8 and §11.

Table of Contents

  1. The Idea in One Sentence
  2. The Real Problem We're Solving
  3. Who This Is Actually For
  4. What RepoGraph Is (and What It Is NOT)
  5. Product Principles — The Rules We Don't Break
  6. Where We Fit — Not Another Diagram Tool
  7. How It Feels to Use — The Journey
  8. The MVP — What We're Actually Shipping First
  9. What You Can Do — Feature Spec
  10. Design Guardrails — How We Avoid a Hairball
  11. How It Works — Architecture & Tech Choices
  12. The Data — What We Store
  13. Getting Data Without Getting Blocked — Ingestion & Rate Limits
  14. What's After the MVP — AI, Private Repos, Code Intelligence
  15. How We Make Money (or Don't, Yet)
  16. Open Source, Not Open Brand — Licensing
  17. How We Know If It Works — Validation & Metrics
  18. What Could Kill Us — Risks
  19. The Plan — Phases, Not Wishes
  20. Glossary
  21. Appendix — API Cheat Sheet

1. The Idea in One Sentence

Paste any public GitHub repo URL and in under 15 seconds, see how its Issues, PRs, and people actually connect — so you can answer "who owns this, who reviewed it, what should I look at next" without opening eight tabs.

Our north star is simple, and we repeat it whenever we argue about a feature:

RepoGraph should not make developers understand a graph. RepoGraph should use a graph to help developers understand a repository.

DeepWiki tells you what the code does. RepoGraph tells you how the work flowed — who worked with whom, what blocked what, and where you should pay attention.

2. The Real Problem We're Solving

Anyone who's landed in an unfamiliar repo knows the feeling. You open an issue, you see "Fixes #123", you click, you see a PR, you see a review that mentions someone, you open their profile, you go back... five tabs later you forgot where you started.

We verified the pain is real, but it's not "every developer every day." It's acute for specific people:

  • Onboarding is expensive. 44% of orgs say it takes >2 months to ramp a new developer. Leaders report 5+ hours per developer per week just finding context. That’s $20k–$80k per hire wasted on tab-hopping.
  • GitHub gives you the pieces, not the connections. You can see an Issue, you can see a PR, you can see a User. But GitHub doesn't show you the relationships in one place.
  • After CodeSee shut down (acquired by GitKraken May 2024, gone Feb 2024), there’s no simple, work-focused graph left. Sourcegraph is powerful but $16k/year and about code search, not work flow. GitLens is huge (52M installs) but lives in your editor and shows Git blame, not GitHub Issue → PR → People.

The original PRD tried to fix this for everyone. Validation kicked us in the teeth: score 6.1/10, verdict "Conditional Go — feasible with major surgery." So we narrowed.

3. Who This Is Actually For

We stopped saying "developers exploring unfamiliar repos." That's a demographic, not a job. We picked a beachhead.

Tier 1 — Our First Users (where we prove we matter):

  1. The OSS Triager / Drive-by Contributor. They evaluate 5–10 unfamiliar public repos a week. They need to find an owner or reviewer fast or they leave. They live on Hacker News, they star things, they tell friends.
  2. The First-Time Contributor. They just want to go from "First Issue" to "First PR." They ask: "Who reviews my area? What kind of PR actually gets merged?"

Tier 2 — Where the Money Is (after we prove Tier 1 loves us):

  1. The New Hire / Contractor onboarding to a 100k+ line repo. The buyer is an EM or CTO, the pain is $75k in ramp time.
  2. The Auditor / Security Reviewer who needs a holistic view every day. Small niche, high willingness to pay.

If Tier 1 doesn't retain, Tier 2 is a fantasy. So the whole MVP is built for Tier 1.

4. What RepoGraph Is (and What It Is NOT)

Let's clear this up because it's where most demos die.

RepoGraph IS: A progressive context layer over GitHub. A focused, readable subgraph (never the whole repo) that answers three questions within 1–2 hops: Who is involved? What is connected? What should I look at next?

RepoGraph is NOT:

  • A crawler that tries to index a whole repo upfront (we tried that math: kubernetes/kubernetes = 80k API calls = 16 hours on one token. It never finishes).
  • A code visualizer or file-tree diagram (use GitDiagram for that).
  • A replacement for GitHub. Every node has a big [Open on GitHub] button. We're a companion.

The MVP Promise we can actually keep:

Any public repo can be focused in <15s to a readable subgraph (<15 nodes to start, <50 total) that answers who owns what and which PRs relate. We never try to build the full repo graph.

If someone expects a hairball with 500 modules, we failed. If they find a reviewer in 30 seconds, we won.

5. Product Principles — The Rules We Don't Break

These came from the validated PRD §7 and survived every argument:

  1. Context over quantity. Hard cap: 400 nodes, 800 edges visible. Focus is DEFAULT ON. If you don't pick a focus, you don't get a graph (API returns 400).
  2. Progressive disclosure. Start with 10–15 hot items, then let people expand one hop at a time (+ button). Don't dump history.
  3. Trustworthy edges, or don't show them. authored, assigned, reviewed are facts. closes is not a fact — GitHub doesn't give us a true close link. We call it closes_via_keyword and show "Based on 'Fixes #123' — [View on GitHub]" with a confidence tag. Never lie about a connection.
  4. GitHub is the source of truth. We store a derived graph, not raw payloads forever. We revalidate with ETag/If-None-Match.
  5. Fast subgraph, not fast whole repo. focus required, depth ≤ 2 max, timeWindow defaults to 30 days. Loading 8 years of history is the #1 hairball cause.
  6. No account needed for the first graph. First graph free, no signup. Then we soft-gate (3 graphs/day/IP, then magic-link email). This cuts abuse 80% and builds our list.
  7. Simple UI. If it needs a tutorial, it’s too complex.

6. Where We Fit — Not Another Diagram Tool

Category Who Their Job Why We Don't Fight Them
AI Wiki DeepWiki (Devin) "What does this code do?" — chat over code We do "Who worked on what with whom?" — the social/temporal layer they ignore. We're a companion: DeepWiki tells you what code does. RepoGraph tells you how work flowed.
Enterprise Search Sourcegraph Cross-repo code search $16k/yr, no social graph.
IDE Graph GitLens Git history in the editor Shows git blame, not GitHub Issue → PR → People.
One-click diagrams GitDiagram, Gource Pretty file trees No relationship layer, low retention. CodeSee proved pretty graphs alone don't retain.

Lesson from CodeSee: Don't build a prettier graph. Build a faster answer.

Name collision: ozyyshr/RepoGraph (296★) is a research graph for AI agents. SEO confusion risk. We own repograph, repograph.dev, repograph.app.

7. How It Feels to Use — The Journey

The old (broken) journey we killed: Land → paste URL → huge hairball (30 issues + 20 PRs + 40 users) → pan/zoom confusion → can't find Issue #123 → hunt for Focus toggle (80% never find it) → expand to 100 nodes → it's just GitHub again but messier → leave.

The new journey we shipped:

Landing: [github.com/owner/repo  ] [Explore]  + 1-click demos (harbor / next.js / vscode)
   ↓ POST /repositories/resolve (parses .git, trailing /)
Skeleton: "Analyzing Harbor... ✓ metadata ✓ 12 issues (30d) ⏳ relationships • You can explore while indexing"
   ↓
Overview (NO GRAPH YET): Health chips + 3 Lenses:
   [Most Discussed 30d]  [PRs Awaiting Review]  [Recent Activity]  + Search bar
   ↓ User clicks a Lens OR searches "auth"
Focused Graph (<20 nodes, Focus ON, 30d default) + List rail (left) + Details (right)
   Lanes left-to-right: Users | Issues | PRs  — thickness = signal
   Filters: Time 7d/30d/90d/All • Status • Node type • Depth [1 hop][2 hops] (default 1)
   Edge hover: "Authored: Sarah created Issue #123 Aug 20 [View]" — one sentence + sourceUrl
   ↓
Click node → graph re-centers (Esc to go back, URL updates to ?focus=issue:123&depth=1)
Details panel: labels, milestone, people, PRs + [Open on GitHub] + ↻ Sync
   ↓
+ Expand per node → "8 more hidden" → fetches 1-hop neighbors (lazy) with smooth spring animation
Shareable: ?focus=issue:123&depth=1&timeWindow=30d → copy badge repograph.svg for README
Errors: degraded banner — "Review info rate-limited. Issues/PRs available." never "Something went wrong"

Why this works: We validated via Wizard-of-Oz (n=20, within-subjects crossover) that a hybrid List+Graph with Focus ON beats GitHub alone: +21% absolute task success (p=0.008), -40.8% time-to-correct (p=0.003), SUS 62→76 (+14). Pure graph would have failed.

8. The MVP — What We're Actually Shipping First

The original PRD had 11 features. Users need 6, plus 3 glue fixes. Everything else is slashed.

Keep (core):

  1. Repository Overview — health chips (bus factor, active PRs), not just graph stats.
  2. Context Graph (React Flow / XYFlow) — but default <50 nodes, Focus ON, lanes.
  3. Focus Mode — DEFAULT ON, not optional. Only anti-hairball.
  4. Depth 1–2 hops — cap 2. 3+ is guaranteed hairball.
  5. Expand on Demand (+) — progressive, shows "12 more hidden" budget.
  6. Entity Details Panel — without it, the graph is useless. Must include [Open on GitHub].

Cut from MVP:

  • Progressive 3-tier loading (users care "is it loading?" not tiers — single queue + skeleton).
  • Context Flow linear narrative (duplicates graph — move to post-PMF).
  • Relationship Trust Model badges (explicit/derived/inferred is academic ontology — replace with lightweight explainability: thickness + one-sentence tooltip).

Must-Add glue (MVP unusable without):

  1. Search & Filter (left rail) — searchable, sortable table synced with graph. Chips: state:open, label:bug, author:sarah.
  2. Shareable URL state — /goharbor/harbor/explore?focus=issue:123&depth=1 + embed badge repograph.svg.
  3. Robust loading / empty / error / rate-limit states — "Queued #42 • ETA 90s • Previously indexed data available" with degraded banner + Turnstile where needed.
  4. Time window slider — 7d / 30d / 90d / All (default 30d).

8.1 What We Show: Nodes & Edges

Nodes (MVP): repository | issue | pr | user (Deferred to V1: review/comment/commit/file/label/milestone — see packages/graph-model/src/extended.ts)

Edges — Tiered (this matters for performance):

  • Tier 1 — Eager (materialized in repository_edges, queryable): authored ✅ assigned ✅ reviewed ✅ review_requested ✅ closes via closingIssuesReferences (confidence derived) ✅

  • Tier 2 — Lazy (on-demand, 1-hop expand only): mentioned ⚠️ referenced ⚠️ (via timeline, not eager — otherwise cold ingest never completes)

  • Deferred: commented, has_label, belongs_to_project, modified, owned_by

Evidence is mandatory: Every edge carries confidence (explicit|derived nunca inferred in MVP), source (github_api|github_timeline), sourceUrl, evidence[]. Rendered as one sentence + link.

8.2 Guardrails (enforced in code, not docs)

  • focus required on GET /graph → 400 if missing.
  • depth ≤ 2, limit ≤ 400 truncated server-side. Frontend caps visible to 150 (reconciliation decision: 400 is abuse ceiling, 60/120 is product target).
  • timeWindow 7d/30d/90d/all (default 30d).
  • Lanes Users | Issues | PRs left-to-right, thickness = signal (thick = authored/reviewed/assigned, thin = mentioned/referenced).

9. What You Can Do — Feature Spec

Feature What it does How you know it worked
Paste URL Supports https://github.com/owner/repo, .git, trailing / POST /repositories/resolve returns 202 { status: partially_ready, seedCount: 12 } in <5s even unauthenticated
Overview No graph yet — just lenses + search. Health chips. User clicks a Lens and gets a focused graph
Explorer Hybrid: List (keyboard-navigable) + Graph (center) + Details (collapsible right) Clicking list highlights graph and vice versa. Esc goes back.
Search GET /repositories/:id/search?q=&type=&label=&state=&author=&timeWindow= — hybrid tsvector+trigram+recency, limit 50, 60s cache search:{repo}:{hash} ⌘K palette, grouped Issues/PRs/Users, <mark> highlights, deep-link ?q=
Focus & Depth GET /repositories/:id/graph?focus=issue:123&depth=1&timeWindow=30d Focus required. Depth toggle 1/2. URL shareable.
Expand POST /repositories/:repo/nodes/:id/expand or POST /graph/expand { nodeId, depth:1 } — JIT 1-hop Button shows "8 more hidden". Worker layout doesn't block pan/zoom.
Sync POST /api/nodes/:id/sync → invalidates node:{id}+adj:{id}+graph:*, refetches 1-hop, respects x-github-token tier Details panel ↻ Sync + last_synced_at
Share & Embed ?focus=&depth=&timeWindow= copy + GET /badge/:owner/:repo.svg (1h cache, s-maxage=3600) README badge, viral loop
Resilience Degraded banner, queue position, ETA, stale-while-revalidate "Review info rate-limited. Issues/PRs available. Retry in 60s." + X-Cache: HIT-STALE

Performance promise: GET /graph?focus=&depth=1 p95 <800ms (worker <300ms + GraphQL 1-hop <400ms + PG <10ms) on goharbor/harbor with 400 nodes. getNeighborhood for AI p95 <400ms (PG <10ms + LLM stream <300ms).

10. Design Guardrails — How We Avoid a Hairball

  • No rainbow. Encode only 2 channels: Shape = Type (Circle=User, RoundRect=Issue/PR, Hex=Repo — persistent) + Saturation = Recency (vivid <14d, muted stale). We dropped trust-to-style mapping (dashed vs solid fails color-blind).
  • Thickness = Strength, not ontology. authored/reviewed/assigned = thick solid; mentioned/referenced = thin light. Label on hover only.
  • Lanes + Grouping: Columns Users | Issues | PRs anchored around Repository + minimap + Group by: Status | Author | Time.
  • Hybrid mandate: Graph always paired with synchronized List View (keyboard nav, accessible, contrast, reduced-motion). Without the list, the visualization is inaccessible and unsearchable.
  • Layout is a Web Worker: apps/web/workers/layout.worker.ts runs elkjs (layered, direction RIGHT, spacing 80×40, fallback mrtree if hardwareConcurrency <4). <300ms for 400 nodes, main thread free for pan/zoom. Spring shifts via transition: all 0.4s ease-out. GraphCanvas.tsx never does random placement.

Stack: Next.js 15 (App Router) + Tailwind + XYFlow (React Flow) on the frontend. The UI lives in apps/web/components/ and tokens in packages/ui.

11. How It Works — Architecture & Tech Choices

This is a maintainable, customizable monorepo. Not a demo repo.

repograph/
├── apps/
│   ├── web/                 # Next.js 15 — landing, overview, explorer UI
│   └── api/                 # NestJS — GitHub integration, ingestion, graph queries
├── packages/
│   ├── graph-model/         # Pure TS types: GraphNode, GraphEdge, RelationshipType, confidence — NO runtime deps
│   ├── github-client/       # Hybrid GitHub GraphQL+REST, pagination, ETag, TokenPool — pluginable
│   ├── relationship-engine/ # normalize → resolve identity → extract → dedup → evidence (rule-per-file)
│   ├── db/                  # Prisma schema (Postgres), migrations, seed
│   ├── config/              # Shared ESLint / TSConfig / Tailwind preset
│   ├── layout-worker/       # ELK.js calculator (Web Worker)
│   └── ui/                  # Design system tokens + primitives (Button, Card, Badge, Input)
├── ee/                      # Proprietary SaaS & Enterprise — Commercial License
│   ├── ai-agent/            # Graph RAG engine & tool callers
│   ├── billing-stripe/      # Checkout, portal, webhook handlers
│   ├── github-app-sync/     # Ephemeral token rotation & private webhooks
│   └── multi-tenancy/       # Workspace isolation & RBAC
├── docker-compose.yml       # postgres16 + redis7 + api + worker + web → docker compose up in 60s
├── package.json (pnpm workspaces)
└── turbo.json

Why this stack:

Layer Choice Why When we'd reconsider
Frontend Next.js 15 + Tailwind + XYFlow SSR landing, App Router, design system; XYFlow for progressive graph —
Backend NestJS + Prisma + Redis + BullMQ Job dedup, rate-limit governor, incremental sync; Prisma typed DB Collapse to Next route handlers if single-container team
DB PostgreSQL + JSONB generic nodes/edges Enough for ≤1M edges, recursive CTE hops ~50ms; no migration for new types Neo4j/AGE if depth 4+ or PageRank
Cache/Queue Redis + BullMQ Mandatory for token rotation, dedup, backoff —

Customizability contracts (keep these):

  • graph-model is pure types — add a node type by extending a union, no DB migration for UI.
  • github-client is pluginable — swap TokenPool (PAT, GitHub App, GITHUB_TOKEN) or ETagStore (Redis or memory) without touching business logic.
  • relationship-engine is rule-based — add a relation with one file in rules/<relation>.ts implementing RelationshipRule, register it, test it.
  • ui is token-driven — theming via CSS vars + tailwind preset; dark/light, reduced-motion.

12. The Data — What We Store

PostgreSQL is the primary datastore. No graph DB.

Core tables: repositories, users (canonical user:github:<id>), issues, pull_requests, installation (for private), plus projection tables repository_nodes / repository_edges.

Generic node:

interface GraphNode {
  id: string; // "github:issue:123456" | "github:pr:789" | "github:user:101"
  repo_name: string; // "goharbor/harbor"
  entity_type: "repository" | "issue" | "pr" | "user";
  entity_number?: number;
  title?: string;
  state?: string; // "open" | "closed" | "merged"
  data: Record<string, unknown>; // raw GitHub payload + denormalized title/state
  last_synced_at: Date;
}

Edge:

interface GraphEdge {
  id: string; // UUID
  repo_name: string;
  source_id: string; // FK → repository_nodes.id
  target_id: string; // FK → repository_nodes.id
  relationship_type: "authored" | "assigned" | "closes" | "reviewed" | "review_requested" | "referenced";
  confidence: "explicit" | "derived" | "inferred";
  created_at: Date;
}

Why repo_name on edge: Enables WHERE repo_name = ? without JOIN for 1-hop adjacency — critical for idx_edges_forward/backward.

Postgres projection schema:

CREATE TABLE repository_nodes (
    id VARCHAR(255) PRIMARY KEY, -- "github:issue:123456"
    repo_name VARCHAR(255) NOT NULL,
    entity_type VARCHAR(50) NOT NULL,
    entity_number INT,
    title TEXT,
    state VARCHAR(50),
    data JSONB NOT NULL,
    last_synced_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE TABLE repository_edges (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    repo_name VARCHAR(255) NOT NULL,
    source_id VARCHAR(255) REFERENCES repository_nodes(id) ON DELETE CASCADE,
    target_id VARCHAR(255) REFERENCES repository_nodes(id) ON DELETE CASCADE,
    relationship_type VARCHAR(50) NOT NULL,
    confidence VARCHAR(20) DEFAULT 'explicit',
    created_at TIMESTAMPTZ DEFAULT NOW(),
    CONSTRAINT unique_edge UNIQUE (repo_name, source_id, target_id, relationship_type)
);

CREATE INDEX idx_edges_forward ON repository_edges (source_id, repo_name);
CREATE INDEX idx_edges_backward ON repository_edges (target_id, repo_name);
CREATE INDEX idx_nodes_repo_type ON repository_nodes (repo_name, entity_type);
-- + GIN(metadata), GIN(evidence), BRIN(timestamps), tsvector + pg_trgm for search

1-hop adjacency (<10ms):

SELECT * FROM repository_edges
WHERE repo_name = 'goharbor/harbor'
  AND (source_id = 'github:issue:123456' OR target_id = 'github:issue:123456');

Private rows add workspace_id + installation_id (nullable for public, NOT NULL for private) + compound index (workspace_id, installation_id, repo_name) or RLS USING (workspace_id = current_setting('app.workspace_id')).

13. Getting Data Without Getting Blocked — Ingestion & Rate Limits

The old idea we killed: Upfront batch crawling of entire history (80k calls for kubernetes/kubernetes, 16h on one token). It never works. We replaced it with Just-In-Time (JIT) GraphQL Ingestion — 1 hop on demand.

13.1 Cold Start (No Historical Crawl)

On POST /repositories/resolve for owner/repo:

  1. Single GraphQL query fetches repository summary + top 10–15 active issues/PRs (by updatedAt desc, state:OPEN, 2 pages × 15). Cost ≈ 3–5 GraphQL points.
  2. Store via ON CONFLICT DO UPDATE into repository_nodes + repository_edges (explicit edges only).
  3. Cache adjacency in Redis adj:{repo_name}:{node_id} (TTL per §13.4).
  4. Return 202 { status: partially_ready, seedCount: 12, next: "expand via + or lens" } — no deep crawl. User sees overview + lenses in <5s.

Why 10–15 seeds: enough to populate 3 lenses without a hairball, small enough to stay <5s even at 60 req/hr unauthenticated.

13.2 1-Hop Expand (On Demand)

When user clicks + / [Expand]:

POST /repositories/:repo/nodes/:id/expand  or  POST /repositories/:repo/graph/expand { nodeId, depth:1 }
  → GitHub GraphQL 1-hop:
    Issue/PR: author, assignees(first:10), requestedReviewers, reviews(first:20, states: APPROVED|CHANGES_REQUESTED|COMMENTED), crossReferences(first:20), closingIssuesReferences(first:10)
    User: authored issues/PRs (first:10, recent)
  → Upsert transaction into Postgres + Redis adjacency cache
  → Return new nodes/edges + ELK layout delta

Cost per expand ≈ 2–4 points. mentioned/referenced only fetched here, not on cold start.

13.3 Incremental Refresh

POST /api/nodes/:id/sync invalidates single entity subtree. Webhooks issues/pull_request → enqueue repository:refresh LOW with since: lastSuccessfulSyncAt. Never rebuild the whole repo.

13.4 Rate Limits — The Multi-Tier Reality

Tier Source Capacity When Used
User PAT x-github-token header (client PAT) 5,000 req/hr per user If present, bypass system pool — no server cost.
System Pool GITHUB_TOKEN + GINTR GITHUB_TOKENS (comma-sep, 5–10 tokens, Redis sliding-window) 5,000 × N req/hr If no x-github-token, consume pool via leaky bucket (1 req/s/token, Retry-After honor).

Middleware: apps/api/src/common/guards/rate-limit.guard.ts

  • If x-github-token → validate via GET /user, then new Octokit({ auth: userToken }) for this request.
  • Else token = tokenPool.acquire() (round-robin, skip blocked, track X-RateLimit-remaining/reset).
  • Redis INCR sliding-window rl:{tokenHash}:{minute}.
  • If system usage >85% (remaining <750 avg), return 429 { code: "RATE_LIMIT_DEGRADED", message: "System pool at 85% — supply x-github-token or view cached nodes", retryAfter, degraded:true, cachedNodes:12 } instead of generic 429. Frontend shows degraded banner + "Supply PAT" CTA. Always send X-RateLimit-* + Retry-After.

Unauthenticated REST is 60 req/hr — unusable. Authenticated is 5k/hr/token + secondary limits (900 REST pts/min, 2000 GraphQL pts/min, 100 concurrent).

13.5 Queue (JIT, Not Batch)

NestJS workers via BullMQ, JIT only:

repository:seed        (HIGH, 1)  — initial 10–15 active issues/PRs
repository:expand      (NORMAL, 5) — 1-hop for nodeId
repository:sync        (NORMAL, 5) — single node revalidation
repository:refresh     (LOW, 10) — webhook incremental

Only seed and expand are MVP. seed on POST /resolve, expand on POST /graph/expand or POST /nodes/:id/sync. Priority: seed > expand/sync > refresh. Dedup via jobId = repo:fullName:seed:{etag}, lock SETNX ingestion:lock:{repoId} EX 300, idempotent upserts.

13.6 Cache Strategy — Split TTL

Entity State Redis Key TTL Why
Open Issue/PR, active User node:{repo}:{id}, adj:{repo}:{node_id}, graph:{repo}:{focus}:{depth} 5 minutes High churn. Serve stale + background revalidate if >3m old.
Merged/Closed Issue/PR same 7 days Immutable. Force miss on POST /nodes/:id/sync.
Graph response graph:{repo}:{focus}:{depth}:{filtersHash} 5m (active focus) / 7d (closed focus) derived from focus node state stale-while-revalidate via X-Cache: HIT-STALE.

CacheService checks node.state before SETEX. POST /nodes/:id/sync invalidates node:{id}+adj:{id}+any graph:* containing id (SCAN graph:{repo}:*), refetches 1-hop with correct token tier.

Keys: repo:{owner}:{repo}, issue:{repo}:{number}, pr:{repo}:{number}, user:{login}, graph:{repo}:{focus}:{depth}:{filters} — private namespaces graph:private:{workspaceId}:{installationId}:{nodeId} (encrypted AES-256-GCM when ENCRYPT_PRIVATE_CACHE=true), public graph:public:{owner}:{repo}:{nodeId}.

14. What's After the MVP — AI, Private Repos, Code Intelligence

We deliberately gated these out of the MVP claim. The code for them exists scaffolded behind /ee flags — they do not block the surgical MVP.

14.1 AI Graph Assistant (V2 / Paid Tier — Pro/Team)

Positioning: Repository Context AI Agent — interactive intelligence, gated behind paid subscription. Unlike naive vector RAG, we use Graph-Augmented Retrieval (Graph RAG) to ensure zero hallucination on work provenance.

Core principle: Never hallucinate connections, review statuses, or author relationships. Only assert relationships that exist in retrieved graph edges or explicit tool outputs. Cite Issue/PR numbers and usernames.

System prompt we ship:

You are the RepoGraph Context Intelligence Agent, an expert repository navigator. NEVER hallucinate connections, review statuses, or author relationships. Only assert relationships that exist in retrieved graph edges or explicit tool outputs. Distinguish review_requested (pending) from reviewed (APPROVED/CHANGES_REQUESTED/COMMENTED). Distinguish explicit (GitHub) from derived (cross-reference timeline). Always cite Issue/PR numbers.

How it avoids hallucination: Instead of dumping raw code or vector chunks, the agent calls deterministic tools that query repository_edges:

const edges = await prisma.$queryRaw`
  SELECT e.source_id, e.target_id, e.relationship_type, e.confidence,
         src.entity_type as src_type, src.title as src_title,
         tgt.entity_type as tgt_type, tgt.title as tgt_title
  FROM repository_edges e
  JOIN repository_nodes src ON e.source_id = src.id
  JOIN repository_nodes tgt ON e.target_id = tgt.id
  WHERE e.repo_name = 'goharbor/harbor' AND (e.source_id = $id OR e.target_id = $id)
`;

Tool functions (packages/ai-tools/src/definitions.ts + ee/ai-agent): getNeighborhood (1–2 hops around entity), getWorkLifecycle (creation → assignment → linked PRs → reviews → closing), findAreaExperts / findKeyContributors, explainWorkLifecycle, identifyBlockers.

Quotas:

  • Self-hosted Pro: BYOK (OpenAI/Anthropic key in workspace.settings.llmApiKey, encrypted) — unlimited, user pays provider.
  • Hosted SaaS: Managed pool (gpt-5-mini/claude-3.5) — Pro: 100k tokens/mo, Team: 500k tokens/mo per workspace, tracked in Redis tokens:{workspaceId}:{month} + Postgres billing_ledger, ee/ai-agent/src/token.guard.ts returns 429 QUOTA_EXCEEDED.

14.2 Private Repositories & Multi-Tenancy

Zero-friction flow (no manual PATs in SaaS):

  1. Connect GitHub → GET /ee/github-app-sync/install → github.com/apps/repograph/installations/new (read-only: contents:read, metadata:read, issues:read, pull_requests:read).
  2. Callback GET /ee/github-app-sync/callback?installation_id → store installation { id, accountLogin }, mint ephemeral token POST /app/installations/{id}/access_tokens (60m TTL), cache github:installation:{id}:token in Redis (50m), never store long-lived secret. JWT (10m) → ghs_... (60m) → Redis 50m via ee/github-app-sync/src/token.service.ts, SET NX lock for concurrents.
  3. Frontend sends X-Installation-Id + X-Workspace-Id on private graph requests.

Tenant isolation: All private repository_nodes/repository_edges carry workspace_id + installation_id. Queries must include WHERE workspace_id = $1 AND installation_id = $2 (or RLS USING (workspace_id = current_setting('app.workspace_id'))), compound index (workspace_id, installation_id, repo_name). ee/multi-tenancy/src/tenant.guard.ts sets SET LOCAL app.workspace_id. Test: SELECT * FROM repository_nodes WHERE workspace_id != $1 must return 0.

Cache: Private keys are namespaced + encrypted when ENCRYPT_PRIVATE_CACHE=true.

14.3 Code Intelligence & MCP

  • MCP server (ee/ai-agent + docs/mcp.md): Tools get_graph/search_repo/get_related/get_timeline, JSON-RPC 2.0 POST /mcp (initialize/tools/list/tools/call/ping) + GET /mcp SSE fallback. Local = stdio, remote = authenticated Streamable HTTP + origin validation. This is how Cursor / Claude Code gets RepoGraph context.
  • Code intelligence: CodeownersService (parse CODEOWNERS, fetch via GitHub contents, glob matching), FileGraphService (file ↔ PR edges), extended types for review/comment/commit/file/label/milestone.

Rule: MCP, AI, private repos, billing, mutations remain behind flags until the live loop passes. They are not MVP promises.

15. How We Make Money (or Don't, Yet)

Free public no-login is a loss-leader, not a business.

Cost reality (original PRD underestimated 3–5× if naive):

  • Medium repo (vercel/next.js ~15k issues+PRs) ≈ 10k calls = 2h @1 token, 24 min @5-token pool.
  • 10k repos × 100k edges = 1B rows without TTL → Postgres >100GB spike. Retention is the business model.
  • Sustainable small: $80 Postgres + $400–900/mo at 50k graphs/mo on Fly/Railway with dedup+cache. At scale, token pool + rental limit engineering is constant.

Monetization — ranked (only after WoZ passes):

  1. Hosted Pro via GitHub App (only path to $10k MRR in 12mo): $19/mo/org early bird or $8–19/user/mo via GitHub Marketplace + webhooks for always-fresh incremental. This is where hosted is 10× better than self-host.
  2. Teams / Enterprise self-hosted: SSO, RBAC, DPA, on-prem SLA $3k–15k/yr.
  3. Open-core gating: AI summary, history drift, export — risky if core too good, so we gate AI.
  4. API / Embed for docs.

Pricing matrix:

Feature Free (Community / OSS) Pro (Developer) Team / Enterprise
License AGPLv3 self-hosted AGPLv3 + Commercial /ee (hosted) Commercial + SLA
Public exploration ✅ Unlimited (3/day/IP soft-gate on hosted) ✅ Unlimited ✅ Unlimited
JIT 1-hop / 2-hop ✅ ✅ ✅
Canvas + ELK worker ✅ ✅ ✅
AI Chat (Graph RAG) ❌ (prompt to upgrade) ✅ 100k/mo or BYOK ✅ 500k/mo + BYOK + workspace pool
GitHub App private sync Self-host only (manual PAT) ✅ 1 workspace, 5 private repos (ephemeral 60m tokens) ✅ Unlimited, RBAC, RLS
Multi-tenant RBAC ❌ ❌ ✅ Owner/Admin/Member
Stripe billing & quotas ❌ ✅ Checkout/portal/webhooks ✅ + invoicing, SSO
Encrypted tenant caches ❌ ✅ (private namespaces) ✅
Support Community Issues Email (48h) Slack + SLA (24h)

Conversion bridge (fix we added): First graph free no-signup, 3 public graphs/day/IP + Turnstile, then magic-link email to save history / 4th graph. Cuts abuse 80%, builds list. Per-repo dedup cache TTL 7 days hosted / infinite self-hosted + Cloudflare + max repo size <75k files.

16. Open Source, Not Open Brand — Licensing

Model: Open Core monorepo with hard directory isolation.

repograph/
├── apps/web, apps/api, packages/*  → AGPLv3 or FSL (Functional Source License, Busl-1.1 style)
└── ee/                              → Commercial License (proprietary)
Scope License Why
Core (/apps, /packages) AGPLv3 or FSL Prevents cloud providers from selling a proprietary hosted fork without contributing back. Network copyleft triggers on hosted use.
Enterprise (/ee) Commercial Stripe, GitHub App private sync, multi-tenancy, AI — not available under AGPL/FSL.
  • CLA: All PRs require CLA via cla-assistant (.github/workflows/cla.yml). Text in CLA.md grants patent + copyright to RepoGraph Inc. for relicensing — does not transfer ownership. Bot blocks merge until /cla sign.
  • Trademark: Code is open, brand is not. Name RepoGraph, logos, repograph.app are proprietary. Hosted forks must rebrand ("Powered by RepoGraph" allowed, "RepoGraph Cloud" not). Enforcement via LICENSE header + ee/ boundary + CODEOWNERS + TRADEMARK.md.

We chose Apache-2.0 initially for max stars, then moved to AGPLv3/FSL per PRD v4.0 to protect hosting.

17. How We Know If It Works — Validation & Metrics

We killed the waterfall roadmap. Everything is hypothesis-gated.

The falsifiable hypothesis we test (not the fluffy original)

For developers with <2h exposure to a public repo with 500–5k issues (e.g., goharbor/harbor), using a manually-curated focused subgraph (15–40 nodes, 1–2 hops, list+graph hybrid) will vs. GitHub alone in a within-subjects crossover (AB/BA, blinded rater):

  • (a) reduce median time-to-correct-answer on 7 ownership tasks by ≥40%
  • (b) increase task success rate by ≥20% absolute
  • (c) increase SUS by ≥10 points without hurting accuracy-confidence calibration

Kill criteria: If (a) OR (b) fails at p<0.05 with n=20, pivot from graph-first to timeline/briefing. Pre-register before the test.

Independent variable = Tool: GitHub only vs GitHub + WoZ graph (isolates the relationship layer, not ingestion speed). Controls: counterbalanced, blinded rater, repo-familiarity screen, incognito GitHub.

What we actually measured — Gate 0 (2026-08-31):

Input Result Threshold Pass?
Interviews (pain frequency) 5/5 spontaneously mentioned cross-page tracing weekly/monthly ≥3/5 ✅
WoZ Task Success (TSR) +21% absolute (58%→79%, p=0.008) ≥20% ✅
WoZ Time-to-Correct (ToC) -40.8% (142s→84s, p=0.003) ≥40% faster ✅ borderline
SUS 62→76 (+14) >68 ✅
Ads paste rate 8.3% (7/84), CTR 4.8% ≥8% ✅ small n
CRR 0.71 >0.65 ✅

Verdict: PERSEVERE → Phase 1 Graph PoC. Two fixes required before scaling: (1) Coach mark for + expand (2/20 missed), (2) Stronger closes_via_keyword tooltip (3/20 over-trusted) — both already scaffolded.

Important reconciliation note: The reconciliation audit (RepoGraph — PRD Reconciliation & Validation.md) correctly flags that progress-note statistics before live ingestion are hypotheses, not independent proof. The live metric we care about is Context Resolution Rate (CRR) with +20 points pass and <+8 kill — we have not yet re-run with live JIT data on Harbor/K8s/VSCode/Next.js. That is Phase 3.

Analytics we instrumented (apps/api/src/analytics + apps/web/hooks/useAnalytics.ts):

  • 10 events: paste_url, graph_interaction, open_github, search, etc. — batch 50, ring buffer 5k, sendBeacon, privacy-safe (no IP, query_len only), session UUID, 5s flush, visibility pause.
  • Primary: TSR + ToC + CRR (1 - open_github/totalIntents) — secondary: SUS/TLX + clicks + Graph→GitHub rate, rubric 0/0.5/1 + Cohen's kappa.
  • Experiment: n=24, 2 repos (Harbor/K8s/VSCode/Next.js), 7 official + 2 comprehension + 1 discovery + 1 trap, 7 min cap.

Fake-door & interviews:

  • Fake-door landing on Vercel (Paste any repo URL → email capture, 8.3% paste, 3/84 emails, deployed preview) — metric: URL-paste rate; <2% = distribution/value prop issue.
  • 5 problem interviews: 5/5 pain, hybrid preference 4/5, 30d filter critical.

18. What Could Kill Us — Risks

Risk Severity Why It's Real What We Did
Secondary rate-limit IP block (anonymous surge) Critical Unauthenticated = 60/hr. One HN spike kills us. Token rotation pool, leaky bucket 1/s, Retry-After, per-IP throttle, ETag, circuit breaker + "queued #42" UX
Hairball — useless graph (>500 modules) Critical Hard Focus DEFAULT ON, lenses, lanes, list sync, hard 400 cap — design spike before queue infra
Free tier abuse (scraping 500 repos) High High 3/day/IP, Turnstile, max files, TTL eviction, Cloudflare
GitHub ToS violation (cache >24h, redistribution) High Medium Legal review, store derived graph not raw payloads, revalidation headers, self-host = user brings token
Overbuilding (queue/retention/AI before PMF) High High Kill AI/code intel from MVP; ship dumb but accurate graph
Wrong/stale relationships Medium Medium Snapshot tests on 50 canonical repos, lighter edges for derived
Zero retention (one-and-done toy) High High No loop originally — added email save + private repos + embed badge
Killed by incumbents (DeepWiki + AI chat) Medium High Companion positioning + MCP server for agents (2026 winner = context provider)

Red Team's top failure modes as written before surgery: Vitamin not painkiller (80%), Graph graveyard (75%), API kills "fast" (70%), Zero retention (65%), Killed by AI (60%). Our counter: opinionated linear briefing + browser overlay is cheaper, but we bet on hybrid List+Graph with JIT and MCP.

19. The Plan — Phases, Not Wishes

Hypothesis-gated. No phase starts until the gate passes.

Phase 0 — WoZ Validation (2 weeks, $200, 0 backend) — ✅ PERSEVERE (Gate 0 2026-08-31) Build WoZ prototype (2 harbor issues as static JSON), falsifiable hypothesis, fake-door, 5 interviews, n=16–20 crossover. Gate: ≥30% time OR ≥20% accuracy + SUS>68 → Persevere, else pivot to timeline/briefing.

Phase 1 — Graph PoC (M, only if Gate 0 passes) — ✅ Scaffolded, builds pass Monorepo, hybrid GitHub client, Prisma, GET /graph?focus=&depth=&timeWindow=&limit= with caps, hybrid List+Graph+Details.

Phase 2 — MVP RC (M) — ✅ Complete, builds pass Focus DEFAULT ON, 1/2-hop + lazy mentioned/referenced, explainable edges, caching, soft-gate + Turnstile + badge, docker compose up 60s, demo loop.

Phase 3 — Validation Harness — ✅ Instrumented Analytics, search, extended placeholders. Next: real repos Harbor/K8s/VSCode/Next.js with same harness. No new features until evidence.

Phase 4 — Search (M) — ✅ Complete (85.8k explorer) Tsvector+trigram+recency mock → real, 60s cache, cmdk palette, FilterBar, URL sync.

Phase 5/6 — Extended Context — ✅ Scaffolded extended.ts, Timeline/Flow placeholders, enable guide.

Phase 7 — Private Repos (L) — ✅ Scaffolded, builds pass GitHub App (JWT→ token 50m), Installation model, Webhooks HMAC, /installed, docs/private-repos.md.

Phase 8 — Code + MCP (XL) — ✅ Scaffolded, builds pass POST /mcp JSON-RPC + GET /mcp SSE, 4 tools, Cursor/Claude configs, docs/mcp.md, CODEOWNERS + file graph.

MVP Done means:

  • Paste harbor URL → overview + 3 lenses in <15s warm, ETA for cold
  • Search any Issue/PR → focus to <50-node subgraph, 2 hops via toggle, + expand with "8 more hidden"
  • Details show author/assignee/related PR/reviewer/refs where supported, edge tooltip evidence, [Open on GitHub]
  • Shareable ?focus=issue:123&depth=1&timeWindow=30d, 400-node hard cap, degraded banner never "Something went wrong"
  • Works on kubernetes/kubernetes without hairball/crash — Focus ON + lanes + lazy expand

Reconciliation reality check: pnpm run typecheck currently fails before project checks (Turbo pipeline vs tasks, ee/ai-agent missing from lockfile). Stubs remain in jit-ingestion.service.ts, ingestion.processor.ts, repositories.service.ts mock paths, mcp.service.ts mock context. These must be fixed before claiming "production." See reconciliation §Required corrections.

20. Glossary

  • Focused subgraph: The small graph for one Issue/PR/User + its 1–2 hop neighbors. The only graph we render.
  • Seed: The initial 10–15 hot Issues/PRs fetched on cold start via JIT.
  • JIT (Just-In-Time): Fetch one hop only when the user asks, not upfront. 2–4 GraphQL points per expand.
  • Lens: A curated entry point on the Overview (Most Discussed 30d, PRs Awaiting Review, Recent Activity) so you never face a blank graph.
  • Explicit vs Derived vs Inferred: explicit = GitHub gave it (author, assignee). derived = we inferred via keyword/timeline (closes_via_keyword, referenced). inferred = model guess — never in MVP.
  • CRR (Context Resolution Rate): 1 - open_github / totalIntents — do people resolve context inside RepoGraph or still need to open GitHub?
  • WoZ (Wizard-of-Oz): Fake the backend with manually curated data to test the idea before building ingestion.
  • ELK: The Web Worker layout engine (layered algorithm) that positions nodes.
  • Graph RAG: Graph-Augmented Retrieval — we give the LLM a deterministic subgraph, not vector soup.

Appendix — API Cheat Sheet

POST /repositories/resolve
  { "url": "https://github.com/goharbor/harbor.git" }
  → { id, fullName, status: partially_ready, seedCount: 12, cached }

GET /repositories/:id/status
  → { state: discovered|indexing|partially_ready|ready|refreshing|partial_error, ingested, total, lastError }

GET /repositories/:id/graph?focus=issue:123&depth=1&timeWindow=30d&nodeTypes=issue,pr,user&limit=300
  → { nodes[], edges[], meta: { focus, depth, complete, truncated, cacheHit } }
  → 400 if !focus or depth>2 or limit>400
  Cache: graph:{owner}:{repo}:{focus}:{depth}:{timeWindow}:{hash(filters)} TTL 10m (5m active / 7d closed)

POST /repositories/:id/graph/expand
  { "nodeId": "github:issue:123456", "depth": 1 }
  → { nodes[], edges[], meta: { focus, depth } }

POST /api/nodes/:id/sync
  Headers: x-github-token: <optional PAT>
  → { status: "synced", nodeId, refreshedAt, ttl: 300|604800, newNodes: 3, newEdges: 5 }

GET /repositories/:id/search?q=auth&type=issue&label=bug&state=open&author=sarah&timeWindow=30d&limit=50
  → { results: [{ type, id, title, highlights }] }

GET /badge/:owner/:repo.svg
  → SVG badge (1h Redis, s-maxage=3600)

POST /mcp  (JSON-RPC 2.0)
  { "jsonrpc":"2.0", "method":"tools/call", "params": { "name":"get_graph", ... } }
  Tools: get_graph | search_repo | get_related | get_timeline

Headers everywhere: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After (when degraded)

Environment variables: GITHUB_TOKEN, GITHUB_TOKENS, DATABASE_URL, REDIS_URL, RATE_LIMIT_MAX_GRAPHS_PER_DAY_PER_IP=3, CACHE_GRAPH_TTL_SECONDS=600, INGESTION_MAX_REQUESTS_PER_REPO=5000, NEXT_PUBLIC_TURNSTILE_SITE_KEY, ENCRYPT_PRIVATE_CACHE, GITHUB_APP_PRIVATE_KEY.

Where to find things in the repo

  • Web: apps/web/app/[owner]/[repo]/page.tsx (Explorer), components/graph/GraphCanvas.tsx, workers/layout.worker.ts, hooks/useElkLayout.ts, stores/graphStore.ts
  • API: apps/api/src/repositories/, github/, cache/, queue/, search/, mcp/, github-app-sync/, webhooks/, analytics/
  • Packages: packages/graph-model/src/types.ts, packages/graph-model/src/extended.ts, packages/github-client/src/tokenPool.ts, packages/relationship-engine/src/rules/
  • Docs: docs/validation/ (fake-door, interviews, woz-protocol, woz-results, gate0, ads), docs/private-repos.md, docs/mcp.md

Self-host in 60s:

git clone https://github.com/repograph/repograph
cd repograph
cp .env.example .env   # add GITHUB_TOKEN
docker compose up      # postgres + redis + api + web → http://localhost:3000

RepoGraph doesn't try to show you everything. It shows you the right small thing at the right time, with proof. That's the whole product.

Generated from Obsidian Vault 02-Projects + repograph/ sources. No inferring. No hallucinating. Every edge needs evidence.