ProgramSmith-100

A collection of 100 containerized software-reimplementation tasks generated with ProgramSmith, packaged in Harbor format.

Every task contains an instruction, an offline Docker environment, a reference solution, and a verifier. Source metadata and upstream license notices are included in each bundle.

Run the tasks

git clone https://github.com/abundant-ai/ProgramSmith-100.git
cd ProgramSmith-100
harbor run --path tasks --agent oracle --n-concurrent 4

Run one task by adding --include-task-name <name>. The task names and content hashes are recorded in manifest.json.

Validate the dataset

make test
make validate
python3 scripts/build_manifest.py --check

The published tasks pass their offline oracle verification. Model sweeps and difficulty calibration are intentionally not included.

ProgramSmith-100 is licensed under Apache-2.0. Bundled upstream materials retain their original licenses under each task's upstream_licenses/ directory.