Burn Your Chip

Pure Rust workspace for training small vision models on a PC and running camera inference on a Seeed Studio XIAO ESP32S3 Sense.

This project currently includes these camera inference demos:

• burn_mnist: handwritten digit recognition, using a 28x28 grayscale model input.
• burn_cifar10: CIFAR-10 object classification, using a 32x32 RGB model input.
• burn_rps: rock / paper / scissors classification, using a 32x32 RGB model input.
• burn_gtsrb: 43-class GTSRB traffic sign classification, using a 32x32 RGB TinyMobileNet model.

The ESP32-S3 firmware captures frames from the OV3660 camera, preprocesses the camera crop on-device, runs the embedded model, and serves the live result in a browser. Training, export, firmware inference, and preprocessing are all Rust.

Project Layout

burn_mnist/                       MNIST trainer
burn_cifar10/                     CIFAR-10 trainer
burn_rps/                         Rock Paper Scissors trainer
burn_gtsrb/                       GTSRB traffic sign trainer
xiao_esp32s3_sense_ov3660/        ESP32-S3 camera firmware

Each trainer writes its model to artifacts/model.mpk. The ESP32-S3 firmware embeds that model at build time, so train first, then build the firmware.

WiFi Setup

Set your WiFi credentials in .cargo/config.toml before running any camera firmware:

WIFI_SSID = "your_wifi_name"
WIFI_PASSWORD = "your_wifi_password"

Train MNIST

The trainer uses cached files under burn_mnist/datasets/mnist if they already exist. Missing MNIST files are downloaded automatically.

cargo +stable run -p burn_mnist --profile train --target x86_64-pc-windows-msvc -- train --epochs 100 --train-samples 60000 --test-samples 10000 --batch-size 64 --lr 0.001 --camera-augment --tui

Output:

burn_mnist/artifacts/model.mpk

Train CIFAR-10

The trainer checks burn_cifar10/datasets first. It supports cached CIFAR-10 binary data at:

burn_cifar10/datasets/cifar-10-batches-bin/

Train with:

cargo +stable run -p burn_cifar10 --profile train --target x86_64-pc-windows-msvc -- train --epochs 60 --train-samples 50000 --test-samples 10000 --batch-size 32 --lr 0.0003 --workers 0 --grad-clip 1.0 --augment --tui

If WGPU runs out of memory on Windows, lower --batch-size to 16.

Output:

burn_cifar10/artifacts/model.mpk

Train Rock Paper Scissors

Prepare this dataset archive first:

C:\Users\Admin\Desktop\RockPaperScissorsDataset.zip

Then cache/extract it into the project:

cargo +stable run -p burn_rps --profile train --target x86_64-pc-windows-msvc -- download

Expected cached layout:

burn_rps/datasets/Rock-Paper-Scissors/train/rock
burn_rps/datasets/Rock-Paper-Scissors/train/paper
burn_rps/datasets/Rock-Paper-Scissors/train/scissors
burn_rps/datasets/Rock-Paper-Scissors/test/rock
burn_rps/datasets/Rock-Paper-Scissors/test/paper
burn_rps/datasets/Rock-Paper-Scissors/test/scissors

Train with:

cargo +stable run -p burn_rps --profile train --target x86_64-pc-windows-msvc -- train --epochs 50 --train-samples 2520 --test-samples 372 --batch-size 32 --lr 0.0005 --workers 0 --grad-clip 1.0 --augment --tui

Output:

burn_rps/artifacts/model.mpk

Train GTSRB Traffic Signs

This trainer uses the local raw GTSRB layout under the trainer crate:

burn_gtsrb/datasets/gtsrb/GTSRB/Training/
burn_gtsrb/datasets/gtsrb/GTSRB/Final_Test/Images/
burn_gtsrb/datasets/gtsrb/GT-final_test.csv

Check the dataset:

cargo +stable run -p burn_gtsrb --profile train --target x86_64-pc-windows-msvc -- download

Train the full 43-class TinyMobileNet model:

cargo +stable run -p burn_gtsrb --profile train --target x86_64-pc-windows-msvc -- train --epochs 80 --train-samples 26640 --test-samples 12630 --batch-size 32 --lr 0.001 --workers 0 --grad-clip 1.0 --augment --tui

Output:

burn_gtsrb/artifacts/model.mpk

Run On ESP32-S3

Build the firmware for the model you want to run:

cargo build --release -p xiao_esp32s3_sense_ov3660 --bin mnist_camera

cargo build --release -p xiao_esp32s3_sense_ov3660 --bin cifar10_camera

cargo build --release -p xiao_esp32s3_sense_ov3660 --bin rps_camera

cargo build --release -p xiao_esp32s3_sense_ov3660 --bin gtsrb_camera

Flash the matching binary. Replace COMx with your board port:

espflash flash --monitor --chip esp32s3 --port COMx target/xtensa-esp32s3-none-elf/release/mnist_camera

espflash flash --monitor --chip esp32s3 --port COMx target/xtensa-esp32s3-none-elf/release/cifar10_camera

espflash flash --monitor --chip esp32s3 --port COMx target/xtensa-esp32s3-none-elf/release/rps_camera

espflash flash --monitor --chip esp32s3 --port COMx target/xtensa-esp32s3-none-elf/release/gtsrb_camera

You can also build and flash in one step with cargo run:

cargo run --release -p xiao_esp32s3_sense_ov3660 --bin mnist_camera

cargo run --release -p xiao_esp32s3_sense_ov3660 --bin cifar10_camera

cargo run --release -p xiao_esp32s3_sense_ov3660 --bin rps_camera

cargo run --release -p xiao_esp32s3_sense_ov3660 --bin gtsrb_camera

Inference

After the board connects to WiFi, the serial log prints:

OPEN WEB: http://<board-ip>/

Open that URL in a browser on the same network.

The web page shows:

• live camera view
• cropped camera input
• model input
• predicted class
• confidence
• preprocessing and timing information

The JSON endpoint is:

http://<board-ip>/result

MNIST labels are digits 0 through 9.

CIFAR-10 labels are airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck.

Rock Paper Scissors labels are rock, paper, and scissors.

GTSRB labels are the official 43 traffic-sign classes, class IDs 0 through 42.

For best camera inference results, keep the object or hand gesture centered, use steady lighting, avoid strong exposure, and hold the target still for a moment.