---

📜 Smart Contract Assistant (RAG-Based)

A production-style Retrieval-Augmented Generation (RAG) system for contract analysis and question answering.

This project allows users to upload legal contracts (PDF), ask questions about them, generate summaries, and evaluate system performance using a structured evaluation pipeline with hallucination guardrails.

---

🚀 Features

• 📂 Upload contract PDFs

• 🔎 Semantic document chunking

• 🧠 Embedding-based retrieval (FAISS)

• 🤖 RAG answer generation

• 🛡 Guardrail-based grounding validation

• 📊 Full evaluation pipeline:

  • Retrieval Hit Rate
  • Precision@k
  • Grounding Rate
  • Semantic Similarity Accuracy
  • Out-of-Scope Rejection

• 📈 Detailed per-question diagnostics

• ⚡ FastAPI backend with streaming responses

---

🏗 Architecture

ingestion/
│── loader.py
│── chunker.py
│── embedder.py

retrieval/
│── retriever.py
│── rag_chain.py

guardrails/
│── validator.py

evaluation/
│── metrics.py
│── run_tests.py

backend/
│── app.py

---

🔹 Ingestion

• Loads PDF contracts
• Splits documents into semantic chunks
• Creates FAISS vector store

🔹 Retrieval

• Retrieves top-k relevant chunks
• Uses embedding similarity scoring

🔹 Generation

• Retrieval-Augmented Generation (RAG)
• Streaming response support

🔹 Guardrails

• Validates whether answers are grounded in retrieved context
• Reduces hallucinations
• Enforces strict boolean validation

🔹 Evaluation

• Modular evaluation framework
• Configurable scoring thresholds
• JSON-safe metrics for API responses
• Per-question diagnostics

---

⚙️ Installation

git clone https://github.com/yourusername/smart-contract-assistant.git
cd smart-contract-assistant

python -m venv venv
# Windows
venv\Scripts\activate

# Mac/Linux
# source venv/bin/activate

pip install -r requirements.txt

---

🔑 Environment Variables

Create a .env file:

OPENAI_API_KEY=your_api_key_here
OPENAI_BASE_URL=https://api.groq.com/openai/v1

⚠️ Never commit your .env file to version control.

---

▶️ Run the Server

uvicorn backend.app:app --reload

API will be available at:

http://127.0.0.1:8000

Interactive API documentation:

http://127.0.0.1:8000/docs

---

📌 API Endpoints

📤 Upload Contract

POST /upload

Response:

{
  "session_id": "..."
}

---

❓ Ask a Question

POST /ask

Request Body:

{
  "session_id": "...",
  "question": "When can the contract be terminated?"
}

Streaming response supported.

---

📄 Summarize Contract

POST /summarize

---

📊 Evaluate System

POST /evaluate

Returns structured evaluation metrics:

{
  "retrieval_hit_rate": 1.0,
  "grounding_rate": 0.92,
  "semantic_accuracy": 0.88,
  "out_of_scope_rejection_rate": 1.0,
  "details": [...]
}

---

📊 Evaluation Design

The evaluation pipeline is modular and production-oriented.

Metrics

• Retrieval Hit Rate
• Precision@k
• Grounding Rate
• Semantic Similarity Accuracy
• Out-of-Scope Rejection Accuracy

Configurable Thresholds

• RETRIEVAL_THRESHOLD
• SIMILARITY_THRESHOLD

Out-of-scope classification is based on retrieval confidence — not fragile keyword matching.

---

🛡 Hallucination Protection

• Retrieval confidence thresholding
• Answer grounding validation
• Strict boolean outputs
• No reliance on keyword-based rejection

---

🧠 Tech Stack

• FastAPI
• LangChain
• FAISS
• OpenAI / Groq-compatible OpenAI API
• Python 3.10+
• Scikit-learn

---

📈 Future Improvements

• Multi-document session support
• Persistent vector store
• Evaluation dashboard
• CI evaluation tests
• Docker containerization
• Role-based access control
• Caching layer
• Async embedding pipeline

---

🎯 Design Principles

• Clean architecture
• Separation of concerns
• JSON-safe API responses
• Modular evaluation logic
• Production-oriented guardrails
• Configurable scoring thresholds

---

👨‍💻 Author

Omar Hamza Biomedical Engineering Student AI Systems & RAG Enthusiast