micrograd

This repository contains a small, notebook-based implementation of a toy autograd engine and a simple neural network training example. It is designed as an educational project to help readers understand how automatic differentiation, backpropagation, and computation graphs work under the hood.

The notebook is inspired by the classic micrograd idea and walks through the core building blocks step by step.

What this project teaches

• How a simple function can be differentiated numerically
• How a custom Value object can represent data and gradients
• How operations like addition, multiplication, power, tanh, and exp can be tracked in a computation graph
• How backward() propagates gradients through the graph
• How a tiny neural network can be built and trained with gradient descent

Repository contents

• micrograd.ipynb — the main interactive notebook with the full tutorial
• graphviz-output/ — generated Graphviz visualizations from the notebook
• README.md — this overview file

Notebook flow

The notebook is organized in a beginner-friendly sequence:

1. Basic math and derivative intuition

   • simple functions
   • numerical differentiation with small steps

2. Building the autograd engine

   • the Value class
   • forward computation
   • gradient tracking
   • backward propagation

3. Visualizing computation graphs

   • Graphviz-based graph rendering
   • examples of simple expressions and branching graphs
   • visualizing activation and loss-like graphs

4. Building a tiny neural network

   • Neuron, Layer, and MLP classes
   • forward pass through a small multilayer perceptron

5. Training with gradient descent

   • computing predictions
   • measuring loss
   • clearing gradients
   • backpropagation
   • updating parameters

Requirements

To run the notebook locally, make sure you have:

• Python 3.8+
• Jupyter Notebook or VS Code with Jupyter support
• the following Python packages:
  • numpy
  • matplotlib
  • graphviz

You may also need the Graphviz system package installed on your machine so the .dot graphs render correctly.

Getting started

1. Open micrograd.ipynb in Jupyter or VS Code.
2. Run the cells in order from top to bottom.
3. Pay attention to the graph visualization cells, as they make the backpropagation flow much easier to understand.

Notes

This project is intentionally small and educational. It is not meant to be a production-ready deep learning framework, but rather a clear and approachable way to understand the fundamentals behind modern automatic differentiation systems.

Suggested learning path

If you are new to this topic, follow the notebook in order and focus on these ideas:

• data flows forward through the graph
• gradients flow backward
• each operation contributes to the gradients of its inputs
• training is just repeated forward and backward passes with parameter updates

Enjoy exploring the notebook and the mechanics behind neural network training!