3LC — Chihuahua vs Muffin Classifier

A ResNet-18 image classifier (PyTorch/torchvision) that distinguishes chihuahuas from muffins, built around the 3LC data-centric-AI SDK (import tlc) for dataset versioning, per-sample metrics/embeddings collection, and an interactive dashboard. Two standalone scripts — register_tables.py (builds the 3LC train/test tables from the image folders) and train.py (trains the classifier, logging metrics back to 3LC) — plus an equivalent starter_notebook.ipynb. No web/API layer.

Current blocker

The 3lc PyPI package (imported as tlc) is quarantined and cannot be installed, as of the last check on 2026-07-11:

$ pip install 3lc
ERROR: Could not find a version that satisfies the requirement 3lc (from versions: none)

$ curl -s https://pypi.org/simple/3lc/
<meta name="pypi:project-status" content="quarantined">

Every script in this repo does import tlc and cannot run end-to-end until this is resolved. This is not fixable from inside this repo — it needs one of:

• the PyPI quarantine on 3lc to lift, or
• an alternate index / vendored wheel from 3LC, or
• direct contact with 3LC support for an install artifact.

Both register_tables.py and train.py now import tlc lazily (not at module load time), so:

• python train.py --help / python register_tables.py --help work and print usage, exit code 0, with no tlc installed.
• Running either script for real without tlc installed exits with a clear message ("The 3lc package is quarantined on PyPI...") and exit code 1 — never a raw ModuleNotFoundError traceback.

To finish setup once 3lc becomes installable again:

1. Uncomment the 3lc==... line at the bottom of requirements.txt and pin it to the actual resolved version (re-run pip install 3lc in a scratch venv and record what it resolves to — the pin in this repo is a placeholder, unconfirmed against a real install).
2. Rebuild the image / rerun pip install --user -r requirements.txt in the devcontainer.
3. Run python register_tables.py once (creates the 3LC train/test tables — this overwrites any existing tables of the same name, see if_exists="overwrite" in the script).
4. Run python train.py (add --epochs N to override the default of 100).

No other part of this container is blocked — torch, torchvision, pillow, tqdm, and numpy all install and import cleanly.

Quick start (devcontainer — recommended)

1. Prereqs: Docker, VS Code + the "Dev Containers" extension.
2. Open this folder in VS Code → "Reopen in Container" when prompted. This installs every dependency except 3lc (see "Current blocker" above).
3. No env vars/secrets are required (see below).
4. python train.py --help and python register_tables.py --help both work inside the container. Actually training/registering data will exit with the "3lc is quarantined" message until 3lc is installable.

Manual setup (no devcontainer)

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt   # everything except 3lc (see blocker above)
python register_tables.py --help  # confirms the script loads
python train.py --help            # confirms the script loads

Environment variables

None required. register_tables.py and train.py take no env vars and no .env file exists or is needed. (train.py does call tlc.init()/publish results to https://dashboard.3lc.ai once tlc is installed — whether that requires a 3LC account/API key is unverified, since tlc cannot currently be installed to check.)

Variable	Required	Purpose	Where to get it
(none)	—	—	—

Dataset layout

Both train128/ and test128/ are checked into this repo already (not downloaded at build/run time):

• train128/<chihuahua\|muffin\|undefined>/*.jpg — 4,733 files, ~52 MB
• test128/<chihuahua\|muffin\|undefined>/*.jpg — 1,184 files, ~13 MB
• Total: ~65 MB of real JPEGs, counts verified against this README's original dataset description.

register_tables.py --data-dir / --train-dir / --test-dir let you point at a different location if needed; by default it uses the script's own directory.

metrics/ — git-lfs pointer caveat

metrics/runs.zip, metrics/tables.zip, and metrics/Embedding Screenshots.pdf are unresolved git-lfs pointer files in this checkout — file reports them as plain ASCII text (the LFS pointer, not the binary content), and unzip -l metrics/runs.zip fails with "End-of-central-directory signature not found". The real content (37.7 MB runs.zip, 145 KB tables.zip per the pointer size fields) was never smudged into this working tree. git lfs ls-files lists them, but you'll need git lfs pull against the real LFS remote (with working credentials) to materialize the actual files — this container does not do that for you.

Testing

No automated tests exist in this repo (find . -iname "*test*.py" is empty). The only available verification today is the startup probe:

python train.py --help            # exit 0, prints usage
python register_tables.py --help  # exit 0, prints usage
python train.py --epochs 1        # exit 1, clear "3lc is quarantined" message

Deployment

Build the image: docker build -t 3lc . Run it: docker run --rm 3lc python train.py --help

This is a best-effort image: it builds and the scripts parse --help cleanly, but neither script can actually train a model or register tables until the 3lc PyPI quarantine (see "Current blocker") is resolved — there is no way around this from inside the container. See DEPLOYMENT.md in the Projects root for the general container-to-host walkthrough; it does not change the 3lc blocker.

Known limitations

• Cannot actually train or register data end-to-end — 3lc/tlc is required by every script and is currently unpip-installable (PyPI quarantine). This is the single blocking issue for this project.
• No GPU requirement in code (torch.device('cuda' if torch.cuda.is_available() else 'cpu') falls back to CPU automatically) — the original 100-epoch training run documented below took ~4-5h on CPU. GPU would be a nice-to-have, not required, once 3lc is installable.
• 3LC Dashboard / cloud publishing (tlc.init(), run.set_status_completed(), https://dashboard.3lc.ai) may require a 3LC account/API key — unverified without keys, since tlc can't be installed to check.
• register_tables.py stores image paths as machine-absolute paths (str(img_path.absolute())); portability across environments depends on tlc's internal URL-alias substitution actually rewriting these on read. This is unverified — tlc cannot be installed to exercise it. Flag as a risk if you move the dataset to a different path/host after registering.
• metrics/*.zip are unresolved git-lfs pointers in this checkout, not the real binary content — see "Dataset layout" above.
• No fake success claims: this container has never successfully run register_tables.py or train.py to completion, because 3lc cannot be installed. Everything else (build, --help, dependency install, error messaging) has been verified to work.

---

Appendix: Original project writeup

The following is the original project writeup (methodology, 3LC-driven data cleaning workflow, and results from a completed 100-epoch training run), preserved as-is from before containerization. It describes a real prior run using a working 3lc install; it does not describe anything reproducible in this container today (see "Current blocker" above).

1. Methodology

Model Architecture

We implemented a ResNet-18 classifier trained from scratch (no pretrained weights) with a custom classification head. The architecture consists of:

• Backbone: ResNet-18 (PyTorch, weights=None)
• Classifier Head:
  • Linear(512 → 256) + BatchNorm1d + ReLU + Dropout(0.3)
  • Linear(256 → 128) + BatchNorm1d + ReLU + Dropout(0.3)
  • Linear(128 → 2) for binary classification

Training Configuration

• Optimizer: AdamW with weight decay (5e-4)
• Learning Rate: 0.0005 (optimized for from-scratch training)
• Scheduler: CosineAnnealingWarmRestarts (T_0=10, T_mult=2)
• Batch Size: 16
• Epochs: 100
• Regularization: Dropout (0.3), Weight Decay (5e-4), Gradient Clipping (max_norm=1.0)
• Device: CPU

Data Augmentation

Moderate augmentation strategy to balance generalization and feature preservation:

• Random resize crop (136→128)
• Random horizontal flip (p=0.5)
• Random rotation (±10°)
• Random affine (shear=8, scale=0.85-1.15)
• Color jitter (brightness/contrast/saturation=0.15)
• Random erasing (p=0.05)

Dataset

• Training: 4,328 labeled samples (after data cleaning)
  • Chihuahua: ~100 samples
  • Muffin: ~100 samples
  • Undefined: 4,133 samples (excluded from training via zero weights)
• Validation: 1,184 samples (640 chihuahua + 544 muffin)

---

2. Data-Centric Improvements Using 3LC

Initial Data Quality Issues

Using the 3LC Dashboard, we identified several critical data quality problems:

1. Mislabeled Samples: Found chihuahua images incorrectly labeled as muffins and vice versa
2. Poor Quality Images: Blurry, corrupted, or irrelevant images affecting model performance
3. Ambiguous Cases: Borderline cases that needed manual review

3LC Dashboard Analysis Workflow

1. Embeddings Visualization with Lasso Tool:

   • Used 3D PaCMAP embeddings to visualize data clusters in the 3LC Dashboard
   • Lasso Tool for Cluster Labeling: Selected clusters of similar images using the lasso tool
   • Auto-labeled entire clusters that clearly belonged to one class (chihuahua or muffin)
   • Identified mislabeled samples that appeared in wrong clusters
   • Found ambiguous or unrelated images that didn't fit any clear cluster

2. Manual Label Correction:

   • Went through sets of images systematically to verify and correct labels
   • Fixed mislabeled samples identified through cluster analysis
   • Corrected edge cases where auto-labeling might have been incorrect
   • Ensured high-quality labels for training data

3. Data Cleaning:

   • Removed ambiguous images that couldn't be clearly classified
   • Removed unrelated images that didn't belong to either class
   • Cleaned dataset from 4,733 → 4,328 samples
   • Created new table versions after each improvement iteration

Key Improvements Made

• Cluster-Based Auto-Labeling: Used 3LC lasso tool to identify and auto-label clusters of similar images
• Manual Label Corrections: Systematically reviewed and corrected ~50+ mislabeled samples
• Data Cleaning: Removed ~400 ambiguous or unrelated images that didn't fit clear class boundaries
• Table Versioning: Created multiple table versions tracking each improvement iteration

Workflow Efficiency: The lasso tool was particularly powerful for efficiently labeling large groups of similar images at once, significantly speeding up the data improvement process compared to manual labeling of individual samples.

---

3. Iteration Details (v1 → v2 → v3)

Version 1: Baseline (Initial Training)

Configuration:

• Epochs: 10
• Learning Rate: 0.0001
• Batch Size: 16
• No learning rate scheduling
• Basic data augmentation

Results:

• Best Validation Accuracy: 80.83%
• Training Progression:
  • Epoch 1: 56.50%
  • Epoch 2: 60.56%
  • Epoch 3: 71.88%
  • Epoch 4: 74.83%
  • Epoch 5: 80.83% (best)
  • Epochs 6-10: Declined (overfitting)

Issues Identified:

• Accuracy plateaued around 80-82%
• Model overfitting after epoch 5
• No learning rate adaptation

Version 2: Hyperparameter Optimization

Changes:

• Learning rate: 0.0001 → 0.0005
• Added CosineAnnealingWarmRestarts scheduler (T_0=10, T_mult=2)
• Added gradient clipping (max_norm=1.0)
• Enhanced data augmentation
• Added BatchNorm layers in classifier
• Increased weight decay to 5e-4

Results:

• Best Validation Accuracy: 88-90% (initial runs with 30 epochs)
• Improved stability with learning rate scheduling
• Better convergence with optimized LR
• Learning rate restarts helped escape plateaus

Remaining Issues:

• Still plateauing around 88-90% with limited epochs
• Needed extended training and data quality improvements

Version 3: Final Optimized Training (100 Epochs)

Changes:

• Extended training to 100 epochs for full convergence
• Fixed mislabeled samples using 3LC Dashboard
• Removed poor-quality images
• Strategic labeling of undefined samples
• Cleaned dataset: 4,733 → 4,328 samples
• Cosine annealing with warm restarts (T_0=10, T_mult=2)

Results:

• Best Validation Accuracy: 96.37% (Epoch 62)
• Training Progression:
  • Epoch 1: 84.63%
  • Epoch 4: 90.12%
  • Epoch 6: 91.47%
  • Epoch 9: 92.91%
  • Epoch 22: 93.41%
  • Epoch 24: 94.09%
  • Epoch 29: 95.27%
  • Epoch 48: 95.52%
  • Epoch 54: 95.69%
  • Epoch 57: 96.20%
  • Epoch 62: 96.37% (BEST)
  • Epoch 68: 96.28%
  • Epoch 100: 96.28%

Key Insight: Extended training with learning rate restarts enabled the model to escape local minima and achieve significantly higher accuracy. Data quality improvements combined with proper hyperparameter tuning and extended training yielded the best results.

---

4. Results + Screenshots

Final Performance Metrics

Metric	Value
Best Validation Accuracy	96.37%
Training Samples (after cleaning)	4,328
Validation Samples	1,184
Total Epochs	100
Best Model Epoch	62
Model Size	ResNet-18 (~11M parameters)
Device	CPU
Training Time	~4-5 hours (100 epochs)

Training Progression (Final Run - 100 Epochs)

Early Training (Epochs 1-10):

• Epoch 1: 84.63% (LR: 0.000488)
• Epoch 2: 86.06%
• Epoch 4: 90.12% (first major jump)
• Epoch 6: 91.47%
• Epoch 9: 92.91%
• Epoch 10: 91.81% (LR restart to 0.000500)

Mid Training (Epochs 11-30):

• Epoch 22: 93.41%
• Epoch 23: 93.50%
• Epoch 24: 94.09%
• Epoch 26: 94.76%
• Epoch 29: 95.27%
• Epoch 30: 94.76% (LR restart to 0.000500)

Late Training (Epochs 31-100):

• Epoch 43: 95.10%
• Epoch 48: 95.52%
• Epoch 54: 95.69%
• Epoch 57: 96.20%
• Epoch 62: 96.37% ⭐ BEST MODEL
• Epoch 68: 96.28%
• Epoch 70: 96.03% (LR restart to 0.000500)
• Epoch 89: 96.03%
• Epoch 98: 96.20%
• Epoch 100: 96.28%

Key Observations:

• Learning rate restarts at epochs 10, 30, 70 helped escape plateaus
• Steady improvement from 84% → 96% over 100 epochs
• Model maintained high accuracy (96%+) in later epochs

3LC Dashboard Visualizations

Note: Screenshots should be added showing:

1. Embeddings Visualization: 3D scatter plot showing class clusters and outliers
2. Metrics Dashboard: Per-sample loss, accuracy, and confidence distributions
3. Confusion Analysis: Misclassified samples with predictions
4. Training Curves: Validation accuracy and loss over epochs
5. Data Quality View: Before/after data cleaning comparison

Key Metrics from 3LC

• Per-Sample Loss Distribution: Identified high-loss samples for review
• Confidence Scores: Used to prioritize undefined sample labeling
• Embedding Clusters: Revealed data quality issues and class boundaries
• Prediction Accuracy: Tracked improvement across iterations

Actual Training Run Details

Final Training Execution:

• Command: python train.py
• Device: CPU
• Total Training Time: ~4-5 hours for 100 epochs
• Training Speed: ~3.2 it/s (iterations per second)
• Validation Speed: ~8.0 it/s
• Best Model Saved: resnet18_classifier_best.pth (Epoch 62)
• Metrics Collection: Completed successfully after training
• Embeddings Reduction: PaCMAP applied for visualization

Training Stability:

• Consistent improvement throughout 100 epochs
• No significant overfitting observed
• Learning rate restarts (epochs 10, 30, 70) helped maintain momentum
• Final epochs (90-100) maintained 96%+ accuracy

---

5. Challenges & Learnings

Technical Challenges

1. Parquet File Corruption:

   • Issue: Initial table registration caused Parquet corruption errors
   • Root Cause: NumPy type incompatibility with PyTorch/PyArrow
   • Solution: Explicit type conversion to Python native types (int, float, str)
   • Learning: Always use native Python types when writing to Parquet for compatibility

2. Accuracy Plateau:

   • Issue: Model stuck at 88-90% accuracy despite hyperparameter tuning
   • Root Cause: Data quality issues (mislabeled samples, poor images)
   • Solution: Used 3LC Dashboard to identify and fix data problems
   • Learning: Data quality > hyperparameter tuning for limited data scenarios

3. Training from Scratch:

   • Issue: Slow convergence and instability without pretrained weights
   • Solution: Lower learning rate (0.0005), gradient clipping, BatchNorm
   • Learning: Training from scratch requires more careful hyperparameter selection

4. NumPy/PyArrow Compatibility:

   • Issue: NumPy 2.x incompatibility with pandas/pyarrow
   • Solution: Upgraded pandas to 2.3.3+ which supports NumPy 2.x
   • Learning: Keep dependencies updated and check compatibility

Key Learnings

1. Data-Centric AI is Powerful:

   • Fixing 50 mislabeled samples improved accuracy more than extensive hyperparameter tuning
   • Model feedback (via 3LC) is the best guide for data improvement
   • Quality > Quantity: 200 clean samples > 2000 noisy samples

2. Hyperparameter Sensitivity:

   • Small LR changes (0.0001 → 0.0005) made significant differences
   • Learning rate scheduling (cosine annealing) helps escape plateaus
   • Batch size affects convergence speed and stability

3. Training from Scratch Requires:

   • Lower learning rates than transfer learning (0.0005 optimal)
   • Many more epochs (100 vs 10-30) for full convergence
   • Better regularization (gradient clipping, BatchNorm)
   • Learning rate scheduling with restarts to escape plateaus
   • Patience and careful monitoring

4. 3LC Dashboard Value:

   • Embeddings visualization revealed data issues not visible in metrics alone
   • Lasso tool enabled efficient cluster-based labeling - much faster than individual sample labeling
   • Cluster analysis helped identify mislabeled samples and ambiguous cases
   • Manual review complemented automated cluster labeling for accuracy
   • Iterative workflow (train → analyze → fix → retrain) is highly effective

---

6. Future Improvements

Short-Term Enhancements

1. Data Expansion:

   • Label more undefined samples using active learning
   • Target low-confidence predictions for manual labeling
   • Aim to reach 500+ labeled samples per class

2. Model Architecture:

   • Experiment with deeper ResNet variants (ResNet-34, ResNet-50)
   • Try EfficientNet or Vision Transformer architectures
   • Ensemble multiple models for improved robustness

3. Advanced Training Techniques:

   • Implement mixup or cutmix augmentation
   • Add label smoothing for better calibration
   • Experiment with focal loss for hard example mining

4. Hyperparameter Optimization:

   • Automated hyperparameter search (Optuna, Ray Tune)
   • Learning rate finder for optimal initial LR
   • Architecture search for optimal classifier head

Long-Term Improvements

1. Active Learning Pipeline:

   • Automate the train → analyze → label → retrain loop
   • Use uncertainty sampling to prioritize labeling
   • Implement semi-supervised learning techniques

2. Data Augmentation:

   • Advanced augmentations (AutoAugment, RandAugment)
   • Domain-specific augmentations for dogs/food
   • Test-time augmentation for inference

3. Model Interpretability:

   • Grad-CAM visualizations for decision explanation
   • Attention mechanisms to understand model focus
   • Feature importance analysis

4. Production Considerations:

   • Model quantization for deployment
   • ONNX export for cross-platform compatibility
   • API development for real-time inference

Expected Impact

• Data Expansion: Already achieved 96.37% - could potentially reach 97-98% with more labeled data
• Architecture Improvements: Deeper models (ResNet-34/50) could push to 97-98%
• Advanced Techniques: Ensemble methods could achieve 98%+ accuracy
• GPU Training: Would significantly reduce training time (currently ~4-5 hours on CPU)

---

Conclusion (original writeup)

This project successfully demonstrated the power of data-centric AI by achieving 96.37% accuracy on a challenging binary classification task with limited labeled data. The key success factors were:

1. ✅ Systematic hyperparameter optimization (LR=0.0005, AdamW, weight decay)
2. ✅ Data quality improvements using 3LC Dashboard (cleaned 4,328 samples)
3. ✅ Extended training with learning rate scheduling (100 epochs, cosine annealing)
4. ✅ Proper regularization (dropout, gradient clipping, BatchNorm)
5. ✅ Iterative train-fix-retrain workflow enabled by 3LC

The project highlights that combining data quality improvements with extended training and proper hyperparameter tuning can achieve excellent results even when training from scratch with limited data. The learning rate restarts were particularly effective in helping the model escape accuracy plateaus.

---

Project Repository: [GitHub Link] 3LC Dashboard: [Dashboard Link] Best Model Accuracy: 96.37% (Epoch 62) Training Time: ~4-5 hours (100 epochs, CPU) Final Validation Loss: 0.1634