The questions about the training process

[jinhangzhan]

Thank you for your source code and dataset.

I want to ask about the training budget you use in the script stage_one.sh, the training batch size is 128 with 32 samples on each, and your dataset(sparkle-reasoning/dsr40k) includes around 40k prompts. Does this mean you have trained the model with 40k * 30 = 1200k prompts for total_epochs=30? Since I am using the same dataset to train the same Qwen model, it seems the performance doesn't change that much after the 400 training steps with a batch size of 256. I was wondering if it is because the amount of training is not enough, should I continue to train in order to improve the in-domain performance? I have attached my training dynamics for the reward:

[jiayuww]

Hi @jinhangzhan, thanks for reaching out and for your interest!

It looks like you're currently training on Qwen2.5-7B-Instruct, I would recommend switching to the math variant instead. Also, in our setup we use the checkpoint at 2k steps. You should expect reward growth similar to what’s shown in the attached figure.

For any follow-up questions, please cc me so I don’t miss the notification.