This repository contains a collection of metaheuristic optimization algorithms implemented in Python. It includes implementations of Genetic Algorithms (GA), Particle Swarm Optimization (PSO) and Ant Colony Optimization (ACO), applied to standard benchmark functions.
Developed for the Spatial Optimization Using Metaheuristic Methods course of my MSc in GIS (K. N. Toosi University of Technology), 2024. The Hölder Table notebook was revised and re-run in 2026.
- Algorithm comparisons: compare the performance of different optimization algorithms on the same problem.
- Robust evaluation: results over many independent runs, not just a single lucky one.
- Interactive Jupyter notebooks: every implementation is a notebook, for easy experimentation and visualization.
| Notebook | Problem | What it shows |
|---|---|---|
Rosenbrock_Function_Optimization.ipynb |
Rosenbrock function (a narrow curved valley) | PSO, GA, ACO and MMAS ACO compared on the same problem |
Holder_Table_Enhanced_PSO.ipynb |
Hölder Table function (rugged, four equal global minima) | An enhanced PSO, evaluated over 30 runs, with a test of what each enhancement contributes |
- Genetic Algorithm (GA): a population-based search that mimics natural selection.
- Particle Swarm Optimization (PSO): a swarm of candidate solutions moves through the search space, each particle pulled toward its own best position and the swarm's best.
- Enhanced PSO: PSO with linearly decreasing inertia, a neighbourhood-best term (best of the 3 nearest particles) and an elite archive of the 5 best distinct solutions.
- Ant Colony Optimization (ACO): a probabilistic technique inspired by ants finding paths to food.
- Max-Min Ant System (MMAS ACO): an ACO variant that bounds the pheromone values, improving the balance between exploration and exploitation.
The Hölder Table function has many local minima and four equal global minima (f = −19.2085 at (±8.05502, ±9.66459)). Run 30 times from different random starts, the enhanced PSO reached a global minimum in 30 of 30 runs, and found all four minima about equally often (7, 8, 6 and 9 times).
Switching each enhancement off in turn gives a more nuanced picture:
| Variant (30 runs each) | Reached the global minimum | Median iterations |
|---|---|---|
| Enhanced PSO (c1 = c2 = c3 = 1, inertia 0.9 → 0.4) | 30 / 30 | 201 |
| Enhanced PSO, original 2024 coefficients (c1 = c2 = 1.5, c3 = 2, inertia 0.9 → 0.2) | 21 / 30 | 267 |
| … without the elite archive | 2 / 30 | 147 |
| … without the neighbourhood best | 30 / 30 | 100 |
| … with constant inertia | 16 / 30 | 77 |
| Basic PSO (c1 = c2 = 1.5, inertia 0.5) | 30 / 30 | 13 |
- With the original coefficients, the three pulls add up to 5, so the swarm overshoots and settles too late in about a third of the runs. Rebalancing them makes every run succeed.
- On this 2-D function, basic PSO is the fastest. The enhancements deliberately prolong exploration: a cost on an easy landscape, and an expected benefit on harder, higher-dimensional problems where premature convergence is the main risk.
The original 2024 version stored the swarm's best position as a live reference to a particle (so it kept moving after being found), started the personal and global bests at ∞ and (0, 0) instead of evaluating the initial swarm, and could fill the elite archive with copies of one point. These are fixed, the coefficients are rebalanced, runs are seeded, and the 30-run evaluation and comparison were added. Details are at the end of the notebook.
- Clone the repository:
git clone https://github.com/KimyaKheirkhah/optimization-algorithms.git cd optimization-algorithms - Install the dependencies:
pip install numpy scipy pandas matplotlib jupyter
- Open a notebook:
jupyter notebook