Skip to content

About

Get started with LLM post-training on Modal.

Resources

Stars

71 stars

Watchers

0 watching

Forks

Repository files navigation

Modal Dojo

Modal Dojo is an open-source library to get started with production-grade LLM training on Modal.

To get started, you can go through the Quickstart section below, or paste the prompt below into your favorite agent:

Install the `modal-dojo` library. Then, install the skill bundle into the local project with `modal-dojo skills install`. Finally, tell the user that they can ask you to do something like: "Walk through tutorials/rl_basics.py and train Qwen3.5-4B to write 5-7-5 haikus from statworx/haiku."

Quickstart

Requirements:

  • Python 3.12

Install the package directly:

uv pip install modal-dojo

Or add it to your project's dependencies:

uv add modal-dojo

Set up Modal and the dashboard:

modal-dojo setup

And empower your agents with the Modal Dojo's skill bundle:

modal-dojo skills install

Then, it's as easy as:

import re

from modal_dojo import (
    HuggingFaceDataset,
    Qwen3_5_4B,
    Qwen3_5_4B_Recipe,
    TrainConfig,
)

model = Qwen3_5_4B()


async def gsm8k_rm(args, sample, **kwargs) -> float:
    text = model.parse_response(sample.response or "").content
    boxed = re.findall(r"\\boxed\{([^}]+)\}", text)
    pred = boxed[-1] if boxed else (re.findall(r"-?[\d,]+(?:\.\d+)?", text) or [""])[-1]
    try:
        return float(float(pred.replace(",", "")) == float(sample.label))
    except ValueError:
        return 0.0


config = TrainConfig(
    model=model,
    dataset=HuggingFaceDataset(
        hf_repo="skrishna/gsm8k_only_answer",
        hf_split="train[:120]",
        input_column="text",
        output_column="label",
        input_format="text",
    ),
    recipe=Qwen3_5_4B_Recipe(
        custom_rm_function=gsm8k_rm,
    ),
)

if __name__ == "__main__":
    run = config.launch()
    print(run.training_run_id)

For a step-by-step walkthrough, see the Getting started tutorial.

Supported models

Family Models
Beam
DeepSeek
GLM
Gemma
Inkling
Kimi
Moonlight
Muse
Qwen3
Qwen3.5
Qwen3.6
Qwen3.8

About

Get started with LLM post-training on Modal.

Resources

Stars

71 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages