Interpretable CNN for Butterfly Species Classification

Final project for IPCV 2025 (Image Processing & Computer Vision). A convolutional neural network is trained to tell visually similar butterfly species apart, and — the main focus — its decisions are made interpretable through feature-space visualizations and Class Activation Mapping (CAM) implemented from scratch.

First-layer kernels (direct RGB)	PCA-RGB feature maps of two Monarch samples
CAM heat map overlaid on the input	The model consistently attends to the wing tail

What this project does

1. Fine-grained classification — a 4-block CNN (32→64→128→256 filters, 3×3 kernels, max pooling) classifies butterfly photos. Two heads were explored: a Flatten + Dense(512) head for the 75-species Kaggle task, and a Global-Average-Pooling (GAP) head with 10 species, whose structure directly supports CAM:

   Conv(32) → Pool → Conv(64) → Pool → Conv(128) → Pool → Conv(256) → GAP → Dense(10) → softmax
   (150×150)   74     (72)        36     (34)         17     (15×15×256)   256      10
   

2. Interpretability — three complementary views of what the network learned:

   • Feature maps: per-layer activations, re-implemented in pure NumPy (strided-view convolution/pooling) and validated against Keras to the last decimal;
   • PCA: the 256-channel last-convolutional features compressed to RGB. Two samples of the same species produce matching color patterns — the model extracts consistent species-level features;
   • CAM from scratch: the class heat map L_c = ReLU(Σ_k w_k^c · A^k + b_c) computed from the last convolutional layer and the Dense weights. The heat maps show the model focuses on wing texture (Adonis), wing-tip spots (Monarch) and the wing tail (Mangrove skipper), consistently across samples of a species.

The full write-up is in report/IPCV.tex (sources in report/; compile with pdflatex + bibtex, see below).

Repository layout

├── notebooks/
│   ├── 01_train_cnn_75_classes.ipynb     # train the 75-species CNN (originally run on Kaggle)
│   ├── 02_evaluate_75_classes.ipynb      # qualitative evaluation on validation batches
│   ├── 03_evaluate_10_classes.ipynb      # evaluation of the shipped GAP model on the 10-species subset
│   ├── 04_numpy_forward_pass.ipynb       # the whole network re-implemented in NumPy, validated vs Keras
│   ├── 05_feature_maps_and_pca.ipynb     # feature maps + PCA of the third pooling stage
│   └── 06_pca_and_cam_visualization.ipynb# kernels, PCA of the last conv layer, CAM heat maps
├── scripts/
│   └── generate_feature_maps.py          # batch NumPy forward pass -> feature maps for every image
├── data/
│   ├── archive_10/                       # 10-species subset (888 images) - included in this repo
│   └── archive/                          # full Kaggle dataset - download separately (see below)
├── models/
│   └── model-gp.keras                    # trained GAP model (~4.7 MB), used by notebooks 03-06
├── report/                               # LaTeX sources of the project report + figures
└── assets/                               # images used in this README

Getting started

pip install -r requirements.txt   # Python 3.10/3.11 recommended (TensorFlow 2.x)

Data. data/archive_10/ (10 species, 888 images) ships with the repo and is all that notebooks 03–06 and the script need. For the 75-species notebooks (01–02), download the full dataset from Kaggle — Butterfly Image Classification (75 species, 6,499 labelled training images) — and extract it to:

data/archive/
├── Training_set.csv
├── Testing_set.csv
├── train/    # 6499 images
└── test/     # 2786 images

Models. models/model-gp.keras (the 10-species GAP model analyzed in the report) is included, so every interpretability notebook runs out of the box. The large checkpoints are not in the repository (GitHub's 100 MB file limit):

• model_75_classes.keras — produced by notebook 01, consumed by notebook 02;
• the earlier Flatten-head 10-species model used for the initial feature-map experiments (the shipped GAP model replaces it).

Notes on the notebooks

• All paths are relative to the notebooks/ folder; run scripts/generate_feature_maps.py from the repository root.
• Notebook 04 is the sanity check for everything else: the NumPy forward pass reproduces the Keras model's softmax output (max |difference| ~ 1e-7 on float32).
• CAM is computed in notebook 06 (getHeatMap): the heat map of class c is the ReLU'd weighted sum of the 15×15×256 last-convolutional feature maps with the Dense weights of class c.
• Batch export in notebooks 05/06 and the script writes to outputs/, which is git-ignored.

Building the report

cd report
pdflatex IPCV && bibtex IPCV && pdflatex IPCV && pdflatex IPCV

Credits

• Dataset: Butterfly Image Classification on Kaggle.
• CAM method: Zhou et al., Learning Deep Features for Discriminative Localization (2015) — see report/IPCV.bib.
• Course project for IPCV 2025 — shared for educational/portfolio purposes.