Enhancing Educational AI (Kiran) for Educhat with fine-tuning open-source LLMS: a comparative study of PEFT and Chain-of-Thought Reasoning

Master's Thesis Project: Advanced Synthetic Dataset Generation and Multi-Stage Fine-tuning for Educational Language Models

🎯 Overview

EduChat is a thesis research project for my Masters degree in Tampere Univeristy focusing on synthetic educational dataset generation and multi-stage LLM fine-tuning for creating specialized academic assistance models. This thesis project explores advanced techniques for enhancing reasoning capabilities in language models through guided reward policy optimization (GRPO) and multi-dataset training approaches. 📖 Read the full thesis:https://typst.app/project/rjPZM69sfKZpyLR6VBEarp

Key Research Contributions

• Synthetic Dataset Generation: Novel approach for creating high-quality educational datasets combining university-specific content (Tampere University + MIT)
• Multi-Stage Fine-tuning: Sequential training pipeline using arXiv papers, synthetic datasets, and reasoning enhancement datasets
• GRPO Implementation: Guided Reward Policy Optimization for chain-of-thought reasoning enhancement
• Comprehensive Evaluation: Performance analysis across multiple model architectures (Phi-4, Gemma-3, Qwen)

✨ Features

🔬 Research Components

• Synthetic Data Pipeline: Automated generation of educational Q&A pairs
• Multi-Model Support: Phi-4, Gemma-3 (4B), and Qwen model variants
• Advanced Training: LoRA fine-tuning with gradient accumulation and mixed precision
• Reasoning Enhancement: GRPO-based training for improved logical reasoning
• Experiment Tracking: Comprehensive WandB integration for reproducible research

🛠️ Technical Features

• Memory-Efficient Training: Chunked data loading for large datasets
• Format Conversion: Robust CSV to chat format conversion with validation
• Multi-Stage Training: Sequential fine-tuning across specialized datasets
• Model Versioning: Automatic saving to HuggingFace Hub and local storage
• Data Validation: Comprehensive quality checks and format compliance

📊 Dataset Information

Primary Datasets

Dataset	Type	Size	Purpose	Format
Synthetic University Dataset	Educational Q&A	~ samples	Domain adaptation	Gemma-3 Chat
arXiv Papers Dataset	Academic Papers	~50K samples	Knowledge base training	Conversational
GSM8K	Mathematical Reasoning	8.5K samples	Reasoning enhancement	Chain-of-thought
OpenCodeReasoning	Code Logic	~15K samples	Programming reasoning	Step-by-step
Open-R1	General Reasoning	~20K samples	Advanced reasoning	Multi-turn

Synthetic Dataset Composition

• Tampere University: Course materials, syllabi, academic policies
• MIT OpenCourseWare: Lecture notes, assignments, educational content
• Quality Metrics: Automated validation, coherence scoring, format compliance

🏗️ Model Architecture & Training

Models Used

Phi-4

• Architecture: 14B parameters, optimized for reasoning
• Training: Two-stage fine-tuning (arXiv → Synthetic)
• Optimization: LoRA with rank 16, alpha 32
• Memory: Gradient checkpointing, mixed precision (FP16)

Gemma-3 (4B)

• Architecture: 4B parameters, efficient deployment
• Training: Comprehensive dataset conversion pipeline
• Optimization: QLoRA with 4-bit quantization
• Focus: Educational domain specialization

Qwen Series

• Architecture: Multiple variants (7B, 14B)
• Training: Comparative analysis baseline
• Optimization: Standard LoRA configuration

Training Configuration

# Core Training Parameters
{
    "learning_rate": 2e-4,
    "batch_size": 4,
    "gradient_accumulation_steps": 4,
    "max_length": 2048,
    "warmup_ratio": 0.03,
    "weight_decay": 0.001,
    "fp16": True,
    "dataloader_pin_memory": False
}

🚀 Installation

Prerequisites

• Python 3.8+
• CUDA-compatible GPU (recommended: 16GB+ VRAM)

Environment Setup

# Clone the repository
git clone https://github.com/yourusername/educhat.git
cd educhat

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Install additional packages for training
pip install transformers[torch] accelerate datasets wandb

WandB Configuration

# Login to Weights & Biases
wandb login

# Set project name
export WANDB_PROJECT="educhat-thesis"

⚡ Quick Start

1. Dataset Preparation

# Convert CSV to Gemma-3 format
from dataset_pipeline.converter import CSVToGemmaConverter

converter = CSVToGemmaConverter()
dataset = converter.convert_csv_to_gemma3("path/to/dataset.csv")
converter.save_dataset(dataset, "converted_dataset.json")

2. Model Training

# Load and fine-tune Phi-4
from training.phi4_trainer import Phi4Trainer

trainer = Phi4Trainer(
    model_name="microsoft/Phi-4",
    dataset_path="converted_dataset.json",
    output_dir="./phi4-educhat"
)

trainer.train()

3. Reasoning Enhancement (GRPO)

# Apply GRPO for reasoning improvement
from training.grpo_trainer import GRPOTrainer

grpo_trainer = GRPOTrainer(
    base_model="./phi4-educhat",
    reasoning_datasets=["GSM8K", "OpenCodeReasoning", "Open-R1"]
)

grpo_trainer.train_reasoning()

🔄 Training Pipeline

Stage 1: Base Fine-tuning

1. arXiv Dataset: Knowledge base establishment
2. Synthetic Dataset: Domain-specific adaptation
3. Validation: Performance benchmarking

Stage 2: Reasoning Enhancement

1. GSM8K: Mathematical reasoning foundation
2. OpenCodeReasoning: Programming logic enhancement
3. Open-R1: Advanced reasoning capabilities

Stage 3: Evaluation & Optimization

1. Performance Metrics: Accuracy, reasoning quality, response coherence
2. Model Comparison: Cross-architecture analysis
3. Hyperparameter Tuning: Optimal configuration discovery

Keywords: Synthetic Dataset Generation, Large Language Models, Fine-tuning, Educational AI, GRPO, Chain-of-thought Reasoning, Academic Assistance, Master's Thesis