🛡️ Brototype Policy AI

An enterprise-grade, agentic AI assistant engineered to provide accurate, grounded, and verified answers to questions regarding official Brototype student policies, academic guidelines, attendance rules, fee refunds, and placement regulations.

---

📌 Table of Contents

• Overview
• System Architecture
• Key Features
• Tech Stack
• Project Structure
• Getting Started
  • Prerequisites
  • 1. Backend Setup
  • 2. Frontend Setup
• API Specification
• RAG & Agentic Workflow Deep Dive
  • Hybrid Search & Ranking (RRF)
  • LangGraph State Machine
  • Security Guardrails
  • LLM-as-a-Judge Evaluation
• Environment Variables
• License

---

📖 Overview

Brototype Policy AI resolves the challenge of policy ambiguity and handbook navigation for students and staff. Instead of manually sifting through lengthy policy documents, students can query the assistant in natural language.

The application couples Hybrid Retrieval-Augmented Generation (RAG) with a LangGraph cyclical state machine, a multi-tiered Security Filter, and an asynchronous LLM-as-a-Judge evaluator to ensure every answer is:

1. Accurate & Grounded: Derived strictly from official PDF documents without hallucinations.
2. Audited & Reviewed: Subject to automatic review loops prior to delivery.
3. Resilient against Attacks: Hardened against prompt injection, jailbreaks, and delimiter escapes.

---

🏗️ System Architecture

flowchart TD
    User([👤 Student / User]) -->|Natural Language Query| UI[💻 React 19 Frontend]
    UI -->|POST /api/chat| Server[⚡ Express 5 Server]

    subgraph Security Layer
        Server --> Guard[🛡️ Security Guardrails & Sanitizer]
        Guard -->|Malicious / Prompt Injection| Blocked[🚫 Security Block Response]
    end

    subgraph Agentic Pipeline ["LangGraph State Machine"]
        Guard -->|Sanitized Query| RetrieveNode[🔍 Hybrid Retriever Node]

        subgraph RAG Engine ["Hybrid Search Engine"]
            RetrieveNode --> Dense[🧠 Dense Search: Xenova all-MiniLM-L6-v2]
            RetrieveNode --> Sparse[📚 Sparse Search: BM25]
            Dense --> RRF[⚖️ Reciprocal Rank Fusion & Threshold Filter]
            Sparse --> RRF
        end

        RRF --> DrafterNode[✍️ Drafter Node: Groq LLM]
        DrafterNode --> ReviewerNode{🧐 Reviewer Node: Audit}
        ReviewerNode -->|Flagged / Leakage / Inconsistency| DrafterNode
        ReviewerNode -->|Passed Verification| FinalDraft[✅ Verified Answer]
    end

    FinalDraft --> Response[📤 JSON Response to Client]
    Response --> UI

    subgraph Asynchronous Quality Control
        FinalDraft -.-> Judge[⚖️ LLM-as-a-Judge: Faithfulness & Grounding Scorer]
    end

Loading

---

✨ Key Features

• Hybrid RAG (Dense + Sparse): Combines dense semantic embeddings (Xenova/all-MiniLM-L6-v2) with sparse keyword retrieval (BM25) using Reciprocal Rank Fusion (RRF).
• Self-Reflective Agentic Loop (LangGraph): Multi-node state graph with conditional retry edges ensuring prompt safety and context faithfulness before answers are returned.
• In-Memory & Persistent Vector Store: Automatically embeds and indexes students_policy.pdf on server startup and caches vector embeddings locally.
• Strict Security Guardrails: Proactive regex and heuristic filters to intercept delimiter injection (<context>, <prompt>), jailbreaks (DAN, developer mode), and malicious instructions.
• LLM-as-a-Judge (Background Eval): Automatically grades every response for factual faithfulness and grounding against source chunks.
• Modern, Polished UI: Interactive chat interface with real-time feedback, quick-prompt suggestion chips, markdown rendering, and reset capabilities.

---

🛠️ Tech Stack

Frontend

• Framework: React 19 + Vite 8
• Icons: Lucide React
• Markdown Support: React Markdown
• Styling: Custom CSS3 design with glassmorphism, responsive breakpoints, and micro-interactions

Backend

• Runtime: Node.js (ES Modules)
• Server Framework: Express 5
• Orchestration: LangChain / LangGraph
• LLM Inference: Groq SDK (using high-speed open-source models)
• Embeddings: @xenova/transformers (Xenova/all-MiniLM-L6-v2 runs locally on CPU)
• Document Processing: pdf-parse (v2) with recursive section and overlap chunking
• Search Algorithms: Custom BM25 implementation + Cosine Similarity Vector Store + RRF

---

📂 Project Structure

broPolicy AI/
├── .gitignore                   # Consolidated root Git ignore file
├── README.md                    # Project documentation
│
├── client/                      # Frontend Application (React + Vite)
│   ├── public/                  # Static assets
│   ├── src/
│   │   ├── assets/              # Client images & branding assets
│   │   ├── App.jsx              # Main chat interface component
│   │   ├── App.css              # Custom UI stylesheet
│   │   ├── index.css            # Base typography & resets
│   │   └── main.jsx             # React entry point
│   ├── index.html               # Web HTML entry
│   ├── package.json             # Frontend dependencies & scripts
│   └── vite.config.js           # Vite build configuration
│
└── server/                      # Backend Service (Node.js + Express)
    ├── app.js                   # Express server entry point & chat route
    ├── package.json             # Server dependencies & scripts
    ├── .env                     # Server environment variables (port, API keys)
    ├── data/
    │   ├── students_policy.pdf  # Official Brototype policy document source
    │   └── vectorStore.json     # Cached vector embeddings (auto-generated)
    └── src/
        ├── eval/
        │   └── judge.js         # LLM-as-a-Judge compliance & faithfulness evaluator
        ├── graph/
        │   └── policyGraph.js   # LangGraph workflow (retrieve -> draft -> review -> route)
        ├── guardrails/
        │   └── securityFilter.js# Input sanitization, injection & jailbreak prevention
        └── rag/
            ├── bm25.js          # BM25 sparse keyword search implementation
            ├── chunker.js       # Semantic section splitter & sliding-window chunker
            ├── embedder.js      # Xenova all-MiniLM-L6-v2 local feature extraction
            ├── hybridSearch.js  # RRF hybrid retrieval & score fusion engine
            └── vectorStore.js   # Persistent vector database & similarity search

---

🚀 Getting Started

Prerequisites

• Node.js (v18.0.0 or higher recommended)
• npm (bundled with Node.js)
• A Groq Cloud API Key

---

1. Backend Setup

1. Open your terminal and navigate to the server/ directory:

   cd server

2. Install dependencies:

   npm install

3. Create or verify your .env file inside server/:

   PORT=3001
   GROQ_API_KEY=your_groq_api_key_here

4. Start the backend development server:

   npm run dev

   Note: On the very first run, the server will parse data/students_policy.pdf, generate embeddings using the local Xenova model, and cache them in data/vectorStore.json.

---

2. Frontend Setup

1. Open a new terminal tab and navigate to the client/ directory:

   cd client

2. Install dependencies:

   npm install

3. (Optional) Configure environment variables: If your backend is running on a custom port, create a .env file in client/:

   VITE_API_URL=http://localhost:3001/api/chat

4. Launch the Vite development server:

   npm run dev

5. Open your browser and navigate to the URL provided by Vite (typically http://localhost:5173).

---

📡 API Specification

Chat Completion Endpoint

• Endpoint: POST /api/chat
• Headers: Content-Type: application/json

Request Body

{
  "message": "What are the qualifying criteria for the final exam?"
}

Response Body (200 OK)

{
  "reply": "According to Section 4 of the Brototype Student Guidelines, attendees must fulfill the following qualifying criteria...",
  "source": [
    {
      "id": "chunk_12",
      "content": "Section 4: Examinations and Reviews...",
      "confidence": 0.812
    }
  ],
  "status": "found_answer"
}

Error / Blocked Response

{
  "reply": "🛡️ Request blocked by security guardrails: Malicious or unsafe prompt detected."
}

---

🔬 RAG & Agentic Workflow Deep Dive

Hybrid Search & Ranking (RRF)

The RAG pipeline solves the limitations of relying purely on vector embeddings or keywords alone:

• Dense Vector Search: Generates 384-dimensional dense vectors using Xenova/all-MiniLM-L6-v2 to understand user intent and semantic similarity.
• Sparse BM25 Search: Matches precise domain terms (e.g., "Section 13", "leave request", "re-evaluation", "counselor").
• Reciprocal Rank Fusion (RRF): $$\text{RRF Score} = \sum \frac{1}{60 + \text{rank} + 1}$$ Scores from both retrieval systems are merged, deduplicated, and filtered by a similarity threshold before passing to the generator.

LangGraph State Machine

Queries are processed through a cyclic state graph defined in server/src/graph/policyGraph.js:

1. retrieve: Extracts relevant context chunks from the PDF via hybrid search.
2. drafter: Synthesizes an initial response grounded strictly in the retrieved chunks.
3. reviewer: Audits the draft for XML tag leakage, prompt instruction leakage, or false policy refusals.
4. routeDecision: Routes back to drafter with corrective feedback (up to 2 retries) or outputs the verified draft.

Security Guardrails

Input is vetted in server/src/guardrails/securityFilter.js before reaching the LLM:

• Delimiter Injection: Strips or rejects tags like <context>, <system>, <prompt>.
• Jailbreak Defense: Detects patterns such as "ignore previous instructions", "DAN mode", "system prompt", or "developer mode".
• Boundary Controls: 500-character length ceiling and non-printable control character removal.

LLM-as-a-Judge Evaluation

After returning the verified response, an asynchronous worker (server/src/eval/judge.js) grades the interaction:

• Grounding: Confirms zero hallucination outside the policy context.
• Correctness: Validates that valid answers were not erroneously refused.
• Safety: Logs output faithfulness scores for observability and compliance monitoring.

---

⚙️ Environment Variables

Variable	Scope	Description	Default
PORT	Server	Port on which the Express server listens	3001 (or 5001)
GROQ_API_KEY	Server	Groq API Key for LLM inference	Required
VITE_API_URL	Client	Endpoint of the backend chat route	http://localhost:3001/api/chat

---

📄 License

This project is developed for internal Brototype student policy navigation and guidance. All rights reserved.