AI · IDS — Threat Intelligence Platform

COM668 Final Year Project | Abdulbosit Abdurazzakov | B00979380

An end-to-end AI Intrusion Detection System trained on the CICIDS2017 dataset, combining a supervised Random Forest and a semi-supervised Isolation Forest into a hybrid OR-rule ensemble — served through a production-grade Flask REST API and an Angular 20 enterprise security dashboard.

---

Project structure

cyber_ids_project/
├── ids-frontend/                   # Angular 20 enterprise dashboard
│   ├── src/app/
│   │   ├── features/
│   │   │   ├── dashboard/          # Live threat map, KPI cards, sparkline
│   │   │   ├── live-detection/     # Real-time flow classifier (CICIDS2017 presets)
│   │   │   ├── alerts/             # Alert management — create, resolve, dismiss
│   │   │   ├── analytics/          # Model benchmark, attack distribution charts
│   │   │   ├── explainability/     # SHAP waterfall, PR curve, feature table
│   │   │   ├── audit-log/          # Full audit trail with time-ago pipe
│   │   │   ├── settings/           # API config, user management
│   │   │   └── api-docs/           # Interactive endpoint explorer with Try It
│   │   ├── core/                   # Auth0 + demo JWT, services, models
│   │   ├── shared/                 # IdsChart (ECharts), StatCard, SeverityBadge, Toast
│   │   └── layout/                 # Sidebar + Topbar shell
│   ├── public/presets.json         # Verified CICIDS2017 attack vectors (78 features)
│   └── proxy.conf.json             # Dev proxy -> Flask :5000
│
├── src/
│   ├── ids/                        # Python IDS package
│   │   ├── data/loader.py          # CSV loading & cleaning
│   │   ├── features/preprocessing.py  # StandardScaler, SMOTE, encoding
│   │   ├── models/train.py         # RF + IF training helpers
│   │   ├── models/predict.py       # classify_flow() -- single inference entry point
│   │   ├── models/evaluate.py      # metrics, confusion matrix, ROC/PR curves
│   │   └── explainability/shap_analysis.py  # SHAP feature importance
│   ├── api.py                      # Flask REST API
│   └── demo_request.py             # CLI demo with real CICIDS2017 flows
│
├── notebooks/                      # Jupyter notebooks 01–10 (training, evaluation, drift, SHAP)
├── tests/                          # pytest suite (52 tests)
├── data/processed/                 # scaler.pkl, feature_names.pkl, label_encoder.pkl
├── models/                         # random_forest.pkl, isolation_forest.pkl
├── results/                        # evaluation plots and CSV outputs
├── Dockerfile                      # Docker build for Flask API
├── extract_attack_presets.py       # Extracts real CICIDS2017 attack vectors for frontend
├── make_submission_zip.ps1         # Clean submission zip builder
├── AT3_video_script.md             # Vodcast script (15 min)
└── requirements.txt                # Python dependencies

---

Quick start

1 — Backend (Flask API)

pip install -r requirements.txt
python src/api.py
# http://localhost:5000

2 — Angular 20 frontend

cd ids-frontend
npm install --legacy-peer-deps
ng serve
# http://localhost:4200

Requires Node.js >= 18 and Angular CLI: npm install -g @angular/cli

3 — Docker (optional)

docker build -t ai-ids .
docker run -p 5000:5000 ai-ids
# Then run the Angular frontend separately as above

4 — Extract verified attack presets

# With Flask running and CICIDS2017 CSVs placed in data/
python extract_attack_presets.py
# Writes ids-frontend/public/presets.json with real 78-feature attack vectors

5 — Run tests

pytest tests/ -v

---

API endpoints

Method	Endpoint	Description
GET	/health	Liveness check -- confirms models loaded
GET	/features	Ordered list of all 78 feature names
POST	/predict	Classify a single network flow
POST	/predict/batch	Classify up to 1000 flows
GET	/api/feed	Last N predictions (detection feed)
GET	/api/stats	Aggregate counts + confidence averages
GET	/api/shap	Top-N features by SHAP / MDI importance

---

Models

Model	Type	Key metric
Random Forest	Supervised (150 trees, balanced weights)	Precision 99.2%
Isolation Forest	Semi-supervised (contamination = 0.1)	Recall 97.0% at PR-optimal threshold
Ensemble OR rule	Hybrid	F1 93.1% -- ATTACK if either model flags

---

Angular dashboard pages

Page	Description
Dashboard	Live threat map, KPI cards, 24h sparkline, alert summary
Live Detection	Classify real CICIDS2017 flows -- DDoS Flood, DoS Hulk, Benign HTTP
Alert Management	Create, investigate, resolve, dismiss incidents with severity triage
Analytics	Model benchmark, attack distribution donut, threshold vs precision/recall
Explainability	SHAP feature importance bar chart, PR curve with optimal threshold marker
Audit Log	Timestamped audit trail of all user actions (role-aware)
Settings	API config, demo user management, system information
API Docs	Interactive endpoint explorer with live Try It panel

---

Dataset

CICIDS2017 — Canadian Institute for Cybersecurity. 2.8M labelled network flows, 15 attack types including DDoS, PortScan, SSH/FTP Brute Force, Web Attacks, and DoS variants.

Raw CSVs are not included (~2 GB). Download MachineLearningCSV.zip from cicresearch.ca, extract into data/, then run python extract_attack_presets.py to generate verified attack vectors for the frontend.

---

Abdulbosit Abdurazzakov | B00979380 | Ulster University | COM668