Issues

New issue

[Tracking] Move off liger-kernel and make the memory-efficient loss the default
#7063 · qgallouedec opened on Sep 4, 2026
8

GRPO ignores the end-of-turn tokens a model declares in its generation config
huggingface/trl#7504 · opened on Oct 2, 2026
Async GRPO does not train Gemma 4's tool-calling turns
huggingface/trl#7503 · opened on Oct 2, 2026
Make the upkeep of stored chat templates more robust
huggingface/trl#7501 · opened on Oct 2, 2026
Revisit how the tiny models track chat template revisions
huggingface/trl#7496 · opened on Oct 2, 2026
Supported models are rejected when their chat template differs from TRL's stored copy
huggingface/trl#7492 · opened on Oct 2, 2026
Remove the slow tests
huggingface/trl#7484 · opened on Oct 2, 2026
AsyncGRPOTrainer and AsyncDistillationTrainer log the wrong epoch with the epoch-driven stop
huggingface/trl#7482 · opened on Oct 2, 2026
1
BEMACallback breaks when any parameter is frozen
huggingface/trl#7480 · opened on Oct 1, 2026
3
GRPO and RLOO crash at the first step when the model is split across devices with device_map
huggingface/trl#7478 · opened on Oct 1, 2026
SFT prompt-completion trains the wrong tokens with Gemma 4 12B/26B/31B
Bug
huggingface/trl#7449 · opened on Sep 29, 2026
2
cast_lm_head_to_fp32 is ignored during GRPO generations (both BF16 autocast and vLLM)
huggingface/trl#7430 · opened on Sep 28, 2026
Perf Issue: Colocated vLLM sleeps and reloads weights between the tool turns of one rollout
huggingface/trl#7428 · opened on Sep 28, 2026
Merged-LoRA export changes frozen base weights
huggingface/trl#7423 · opened on Sep 28, 2026
CI fails for slow tests on multiple GPUs: RuntimeError: Expected all tensors to be on the same device, but got index is on cuda:1, different from other tensors on cuda:0
huggingface/trl#7421 · opened on Sep 28, 2026
2
[Feature request] C-GRPO: conformal adaptive rollout budgets for GRPO (NeurIPS 2026)
huggingface/trl#7402 · opened on Sep 25, 2026
OpenReward's default GRPO reward function cannot distinguish "never scored" from "scored 0.0"
huggingface/trl#7364 · opened on Sep 24, 2026
GRPOTrainer tool discovery evaluates every property on an environment
huggingface/trl#7362 · opened on Sep 23, 2026
1
SFTTrainer drops dataset_text_field for vision datasets with default column filtering
huggingface/trl#7354 · opened on Sep 23, 2026
rollout_func docstring says prompts have "no duplication", but each prompt is already repeated num_generations times
huggingface/trl#7353 · opened on Sep 23, 2026
1
CI rerun filter does not retry GPU allocation failures raised in a launched subprocess
huggingface/trl#7326 · opened on Sep 21, 2026
2
GRPO/RLOO/Distillation trainers crash on iterable datasets when dataloader_multiprocessing_context is set (default on MPS since transformers 5.15)
huggingface/trl#7302 · opened on Sep 19, 2026
Proposing to remove MiniLLMTrainer
huggingface/trl#7295 · opened on Sep 18, 2026
1
Support custom/composable advantage computation across trainers
huggingface/trl#7291 · opened on Sep 18, 2026
2
AsyncGRPO: one stale-dropped group pins the saved resume position for the rest of the run
huggingface/trl#7274 · opened on Sep 18, 2026
DPO and KTO skip DDP gradient synchronization with use_liger_kernel=True
Bug
huggingface/trl#7245 · opened on Sep 16, 2026
1

Navigation Menu

Issues

Issues

GRPO ignores the end-of-turn tokens a model declares in its generation config

Async GRPO does not train Gemma 4's tool-calling turns

Make the upkeep of stored chat templates more robust

Revisit how the tiny models track chat template revisions

Supported models are rejected when their chat template differs from TRL's stored copy

Remove the slow tests

AsyncGRPOTrainer and AsyncDistillationTrainer log the wrong `epoch` with the epoch-driven stop

BEMACallback breaks when any parameter is frozen

GRPO and RLOO crash at the first step when the model is split across devices with device_map

SFT prompt-completion trains the wrong tokens with Gemma 4 12B/26B/31B

cast_lm_head_to_fp32 is ignored during GRPO generations (both BF16 autocast and vLLM)

Perf Issue: Colocated vLLM sleeps and reloads weights between the tool turns of one rollout

Merged-LoRA export changes frozen base weights

CI fails for slow tests on multiple GPUs: RuntimeError: Expected all tensors to be on the same device, but got index is on cuda:1, different from other tensors on cuda:0

[Feature request] C-GRPO: conformal adaptive rollout budgets for GRPO (NeurIPS 2026)

OpenReward's default GRPO reward function cannot distinguish "never scored" from "scored 0.0"

GRPOTrainer tool discovery evaluates every property on an environment

SFTTrainer drops dataset_text_field for vision datasets with default column filtering

`rollout_func` docstring says prompts have "no duplication", but each prompt is already repeated `num_generations` times

CI rerun filter does not retry GPU allocation failures raised in a launched subprocess

GRPO/RLOO/Distillation trainers crash on iterable datasets when dataloader_multiprocessing_context is set (default on MPS since transformers 5.15)

Proposing to remove MiniLLMTrainer

Support custom/composable advantage computation across trainers

AsyncGRPO: one stale-dropped group pins the saved resume position for the rest of the run

DPO and KTO skip DDP gradient synchronization with use_liger_kernel=True