MQE

PyTorch implementation of Multistep Quasimetric Estimation (Zheng, Myers, Eysenbach, Levine; ICLR 2026).

Offline goal-conditioned RL: a quasimetric distance (MRN) trained with multistep backups, action invariance, and DDPG+BC policy extraction.

Setup

python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

Test

make test     # unit tests
make smoke    # 400-step point-maze run (~30s)
make demo     # longer point-maze run with eval

The point maze is built in. No dataset download required.

UI

make ui

Open http://127.0.0.1:8000. Train on the point maze, animate rollouts, evaluate a checkpoint, and view the learned distance heatmap.

python -m mqe.train --env pointmaze --steps 3000
python -m mqe.train --env pointmaze --head iqe --gamma 0.99
python -m mqe.evaluate --ckpt results/pointmaze_mrn_s0_demo/checkpoint.pt
python -m mqe.plotting

OGBench

pip install -e ".[ogbench]"
python -m mqe.train --dataset antmaze-medium-navigate-v0