Skip to content
BranchesTags

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
.gitignore
.gitignore
 
 
README.md
README.md
 
 

Repository files navigation

Awesome Rubric-Based Reinforcement Learning 📋 🎯 🤖

A curated collection of papers on Rubric-Based Reinforcement Learning (Rubric RL): training language models and agents with rewards derived from explicit, multi-criterion evaluation standards rather than a single verifiable answer or one holistic preference score.

Rubrics extend reinforcement learning to open-ended tasks, but a rubric only becomes a learning signal after a chain of operations constructs the criteria, scores an output or trajectory, assigns temporal credit, normalizes and aggregates the scores, and updates a policy. This list is organized along that chain.

rubric construction -> criterion execution -> temporal credit assignment
  -> normalization -> scalarization -> policy optimization -> rubric adaptation

Table of Contents

The collection currently contains 483 papers, covering work released through August 3, 2026. Entries are placed in the most specific section their method supports.

Related Surveys and Repositories

  1. [arXiv'26] The Rules of the Game: A Survey of Rubrics for Large Language Models [Repo]
  2. [arXiv'26] From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape [PDF] [Repo]
  3. [arXiv'26] Seeing the Forest and the Trees: A Survey of Analytic Rubrics for Holistic Reward Modeling [Repo]
  4. [arXiv'26] From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models [PDF] [Repo]
  5. [arXiv'26] A Survey of Process Reward Models: From Outcome Signals to Process Supervisions [PDF]

Paper Collection

Rubric Construction

Automated and Synthetic Generation

  1. [arXiv '26] ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning [PDF]
  2. [arXiv '26] Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria [PDF]
  3. [arXiv '26] Autorubric: Unifying Rubric-based LLM Evaluation [PDF]
  4. [arXiv '25] OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment [PDF]

Contrastive and Preference-Derived Criteria

  1. [arXiv '26] Preference-Aware Rubric Learning for Personalized Evaluation [PDF]
  2. [arXiv '26] Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubric [PDF]
  3. [ICML '26] Online Rubrics Elicitation from Pairwise Comparisons [PDF]

Refinement, Redundancy, and Compression

  1. [arXiv '25] Automated Refinement of Essay Scoring Rubrics for Language Models via Reflect-and-Revise [PDF]

Expert and Task-Derived Criteria

  1. [arXiv '26] Rubric-as-Experts: Case-Specific MQM Rubrics for Translation Quality Evaluation [PDF]
  2. [arXiv '26] ComplexConstraints and Beyond: Expert Rubrics for RLVR [PDF]
  3. [arXiv '26] Feedback-to-Rubrics: Can We Learn Expert Criteria from Inline Comments? [PDF]
  4. [arXiv'26] MERIT: Matching Expertise via Rubric-Informed Training for Reviewer Assignment [PDF]
  5. [arXiv '26] AesRM: Improving Video Aesthetics with Expert-Level Feedback [PDF]
  6. [arXiv '26] JADE: Expert-Grounded Dynamic Evaluation for Open-Ended Professional Tasks [PDF]
  7. [arXiv '25] Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses [PDF]
  8. [arXiv '25] Reinforcement Learning with Rubric Anchors [PDF]
  9. [ICLR '26] Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains [PDF]
  10. [arXiv'25] CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards [PDF]
  11. [arXiv'25] AutoRule: Reasoning Chain-of-thought Extracted Rule-based Rewards Improve Preference Learning [PDF]
  12. [NeurIPS '24] Rule Based Rewards for Language Model Safety [PDF]

Criterion Execution and Reward Representation

Rubric Reward Models

  1. [arXiv'26] Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling [PDF]
  2. [arXiv '26] Beyond Rubrics: Exploration-Guided Evaluation Skills for Reward Modeling [PDF]
  3. [arXiv'26] Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill [PDF]
  4. [arXiv'26] DyCoRM: Dynamic Criterion-Aware Reward Modeling for Text-to-Image Generation [PDF]
  5. [arXiv '26] AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment [PDF]
  6. [arXiv '26] DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification [PDF]
  7. [arXiv'26] Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring [PDF]
  8. [arXiv '26] C2: Scalable Rubric-Augmented Reward Modeling from Binary Preferences [PDF]
  9. [arXiv '26] Rationale Matters: Learning Transferable Rubrics via Proxy-Guided Critique for VLM Reward Models [PDF]
  10. [arXiv'26] Small Reward Models via Backward Inference [PDF]
  11. [arXiv'26] P-GenRM: Personalized Generative Reward Model with Test-time User-based Scaling [PDF]
  12. [arXiv '26] Generating Data-Driven Reasoning Rubrics for Domain-Adaptive Reward Modeling [PDF]
  13. [arXiv '26] Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks [PDF]
  14. [arXiv '26] Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models [PDF]
  15. [arXiv '26] Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis [PDF]
  16. [arXiv'26] CE-RM: A Pointwise Generative Reward Model Optimized via Two-Stage Rollout and Unified Criteria [PDF]
  17. [arXiv'26] P-Check: Advancing Personalized Reward Model via Learning to Generate Dynamic Checklist [PDF]
  18. [arXiv '25] Multidimensional Rubric-oriented Reward Model Learning via Geometric Projection Reference Constraints [PDF]
  19. [arXiv'25] PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling [PDF]
  20. [arXiv '25] Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modeling [PDF]
  21. [ICLR '26] Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Training [PDF]
  22. [arXiv'25] Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models [PDF]
  23. [arXiv '25] Robust Reward Modeling via Causal Rubrics [PDF]
  24. [arXiv '25] RM-R1: Reward Modeling as Reasoning [PDF]
  25. [arXiv'25] Inference-Time Scaling for Generalist Reward Modeling [PDF]
  26. [Findings of ACL '25] CARMO: Dynamic Criteria Generation for Context-Aware Reward Modelling [PDF]
  27. [arXiv'23] Direct Preference Optimization: Your Language Model Is Secretly a Reward Model
  28. [arXiv] SALMON: SELF-ALIGNMENT WITH INSTRUCTABLE REWARD MODELS [PDF]
  29. [ACL '25] Generative Reward Modeling via Synthetic Criteria Preference Learning [PDF]

Generative and Reasoning Judges

  1. [arXiv '26] Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers [PDF]
  2. [arXiv'26] PReMISE: Policy Rubrics as Measurement Specifications for LLM Judges [PDF]
  3. [arXiv '26] Generating and Refining Dynamic Evaluation Rubrics for LLM-as-a-Judge [PDF]
  4. [arXiv'26] CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging [PDF]
  5. [arXiv '26] Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning [PDF]
  6. [arXiv '26] Criterion Validity of LLM-as-Judge for Business Outcomes in Conversational Commerce [PDF]
  7. [arXiv'26] AutoChecklist: Composable Pipelines for Checklist Generation and Scoring with LLM-as-a-Judge [PDF]
  8. [arXiv '26] A Rubric-Supervised Critic from Sparse Real-World Outcomes [PDF]
  9. [arXiv '26] Learning to Judge: LLMs Designing and Applying Evaluation Rubrics [PDF]
  10. [arXiv '26] Agentic Rubrics as Contextual Verifiers for SWE Agents [PDF]
  11. [arXiv '25] Are We on the Right Way to Assessing LLM-as-a-Judge? [PDF]
  12. [arXiv '25] Kardia-R1: Unleashing LLMs to Reason toward Understanding and Empathy for Emotional Support via Rubric-as-Judge Reinforcement Learning [PDF]
  13. [arXiv'25] IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation [PDF]
  14. [arXiv'25] Approximating Human Preferences Using a Multi-Judge Learned System [PDF]
  15. [arXiv '25] An Efficient Rubric-based Generative Verifier for Search-Augmented LLMs [PDF]
  16. [arXiv '25] Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustness [PDF]
  17. [arXiv'25] Baichuan-M2: Scaling Medical Capability with Large Verifier System [PDF]
  18. [arXiv'25] RefCritic: Training Long Chain-of-Thought Critic Models with Refinement Feedback [PDF]
  19. [arXiv'25] Checklist Engineering Empowers Multilingual LLM Judges [PDF]
  20. [arXiv'25] Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performance [PDF]
  21. [arXiv'25] YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering [PDF]
  22. [arXiv'25] Think-J: Learning to Think for Generative LLM-as-a-Judge [PDF]
  23. [arXiv '25] Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgments [PDF]
  24. [arXiv'24] Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning [PDF]
  25. [ICLR '26] QuRL: Rubrics As Judge For Open-Ended Question Answering [PDF]
  26. [EMNLP '23] G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment [PDF]

Checklists and Binary Criteria

  1. [arXiv'26] EvoIdeator: Evolving Scientific Ideas through Checklist-Grounded Reinforcement Learning [PDF]
  2. [arXiv'26] RLBFF: Binary Flexible Feedback to Bridge Between Human Feedback and Verifiable Rewards
  3. [arXiv'25] RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards [PDF]
  4. [NeurIPS '25] Checklists Are Better Than Reward Models For Aligning Language Models [PDF]
  5. [arXiv '24] TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation [PDF]

Graded, Ordinal, and Distributional Scores

  1. [arXiv'26] Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions [PDF]

Temporal Credit Assignment

Belief and Potential-Based Shaping

  1. [arXiv'26] Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents [PDF]
  2. [arXiv'26] Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers [PDF]
  3. [arXiv'26] TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMs [PDF]
  4. [arXiv'26] Optimizing Agentic Reasoning with Retrieval via Synthetic Semantic Information Gain Reward [PDF]
  5. [arXiv'25] Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents [PDF]

Turn-Level Credit

  1. [arXiv'26] TRACE: Turn-Level Reward Assignment via Credit Estimation for Long-Horizon Agents [PDF]
  2. [arXiv'26] ECHO: Learning Epistemically Adaptive Language Agents with Turn-Level Credit [PDF]
  3. [arXiv'25] Reinforcing Multi-Turn Reasoning in LLM Agents via Turn-Level Reward Design [PDF]

Step-Level and Process Rewards

  1. [arXiv'26] The Weakest Link Tells It All: Outcome-Supervised Process Reward Modeling via Learnable Credit Assignment [PDF]
  2. [arXiv'26] VisCritic: Visual State Comparison as Process Reward for GUI Agents [PDF]
  3. [arXiv'26] VeriBound: PAC-Bayesian Generalization Bounds for Process Reward Models Trained with Formal Verification Tools [PDF]
  4. [arXiv '26] ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents [PDF]
  5. [arXiv'26] EST-PRM: Stress-Testing Process Reward Models Before They Become Load-Bearing [PDF]
  6. [arXiv '26] Rubric-Guided Process Reward for Stepwise Model Routing [PDF]
  7. [arXiv '26] Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents [PDF]
  8. [arXiv '26] Step-wise Rubric Rewards for LLM Reasoning [PDF]
  9. [arXiv'26] Verifiable Process Rewards for Agentic Reasoning [PDF]
  10. [arXiv'26] Unsupervised Process Reward Models [PDF]
  11. [arXiv'26] SCPRM: A Schema-aware Cumulative Process Reward Model for Knowledge Graph Question Answering [PDF]
  12. [arXiv'26] Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation [PDF]
  13. [arXiv'26] GR-Ben: A General Reasoning Benchmark for Evaluating Process Reward Models [PDF]
  14. [arXiv'26] Improving Vision-language Models with Perception-centric Process Reward Models [PDF]
  15. [arXiv'26] ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward [PDF]
  16. [arXiv'26] Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewards [PDF]
  17. [arXiv'26] MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning [PDF]
  18. [arXiv '26] SWE-TRACE: Optimizing Long-Horizon SWE Agents Through Rubric Process Reward Models and Heuristic Test-Time Scaling [PDF]
  19. [arXiv'26] Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation [PDF]
  20. [arXiv'26] HISR: Hindsight Information Modulated Segmental Process Rewards For Multi-turn Agentic Reinforcement Learning [PDF]
  21. [arXiv'26] Grounding the Score: Explicit Visual Premise Verification for Reliable Vision-Language Process Reward Models [PDF]
  22. [arXiv'26] Truncated Step-Level Sampling with Process Rewards for Retrieval-Augmented Reasoning [PDF]
  23. [arXiv'26] SecCodePRM: A Process Reward Model for Code Security [PDF]
  24. [arXiv'26] What, Whether and How? Unveiling Process Reward Models for Thinking with Images Reasoning [PDF]
  25. [arXiv'26] LLM Reasoning with Process Rewards for Outcome-Guided Steps [PDF]
  26. [arXiv'26] FunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation [PDF]
  27. [arXiv'26] Beyond Outcome Verification: Verifiable Process Reward Models for Structured Reasoning [PDF]
  28. [arXiv'26] Towards Robust Process Reward Modeling via Noise-aware Learning [PDF]
  29. [arXiv'26] ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agents [PDF]
  30. [arXiv'26] PRPO: Aligning Process Reward with Outcome Reward in Policy Optimization [PDF]
  31. [arXiv'26] SmartSearch: Process Reward-Guided Query Refinement for Search Agents [PDF]
  32. [arXiv'26] PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations [PDF]
  33. [arXiv'26] A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage
  34. [arXiv'25] DreamPRM-Code: Function-as-Step Process Reward Model with Label Correction for LLM Coding [PDF]
  35. [arXiv'25] TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards [PDF]
  36. [arXiv'25] RPM-MCTS: Knowledge-Retrieval as Process Reward Model with Monte Carlo Tree Search for Code Generation [PDF]
  37. [arXiv'25] DPRM: A Dual Implicit Process Reward Model in Multi-Hop Question Answering [PDF]
  38. [arXiv'25] AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress [PDF]
  39. [arXiv'25] Process Reward Models for Sentence-Level Verification of LVLM Radiology Reports [PDF]
  40. [arXiv'25] Exploring Generative Process Reward Modeling for Semi-Structured Data: A Case Study of Table Question Answering [PDF]
  41. [arXiv'25] Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards [PDF]
  42. [arXiv'25] GroundedPRM: Tree-Guided and Fidelity-Aware Process Reward Modeling for Step-Level Reasoning [PDF]
  43. [arXiv '25] Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards [PDF]
  44. [arXiv'25] Advancing Reasoning in Diffusion Language Models with Denoising Process Rewards [PDF]
  45. [arXiv'25] GUI-PRA: Process Reward Agent for GUI Tasks [PDF]
  46. [arXiv'25] Training Vision-Language Process Reward Models for Test-Time Scaling in Multimodal Reasoning: Key Insights and Lessons Learned [PDF]
  47. [arXiv'25] Orcust: Stepwise-Feedback Reinforcement Learning for GUI Agent [PDF]
  48. [arXiv'25] SCAN: Self-Denoising Monte Carlo Annotation for Robust Process Reward Learning [PDF]
  49. [arXiv'25] Fin-PRM: A Domain-Specialized Process Reward Model for Financial Reasoning in Large Language Models [PDF]
  50. [arXiv'25] GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning [PDF]
  51. [arXiv'25] VRPRM: Process Reward Modeling via Visual Reasoning [PDF]
  52. [arXiv'25] Uncertainty-Based Methods for Automated Process Reward Data Construction and Output Aggregation in Mathematical Reasoning [PDF]
  53. [arXiv'25] The Bidirectional Process Reward Model [PDF]
  54. [arXiv'25] Dynamic and Generalizable Process Reward Modeling [PDF]
  55. [arXiv'25] CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoning [PDF]
  56. [arXiv'25] Self-Guided Process Reward Optimization with Redefined Step-wise Advantage for Process Reinforcement Learning [PDF]
  57. [arXiv'25] Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards [PDF]
  58. [arXiv'25] Know What You Don't Know: Uncertainty Calibration of Process Reward Models [PDF]
  59. [arXiv'25] FreePRM: Training Process Reward Models Without Ground Truth Process Labels [PDF]
  60. [arXiv'25] DreamPRM: Domain-Reweighted Process Reward Model for Multimodal Reasoning [PDF]
  61. [arXiv'25] Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision [PDF]
  62. [arXiv'25] From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling [PDF]
  63. [arXiv'25] Guiding VLM Agents with Process Rewards at Inference Time for GUI Navigation [PDF]
  64. [arXiv'25] Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning [PDF]
  65. [arXiv'25] Efficient Process Reward Model Training via Active Learning [PDF]
  66. [arXiv'25] GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning [PDF]
  67. [arXiv'25] R-PRM: Reasoning-Driven Process Reward Modeling [PDF]
  68. [arXiv'25] ViLBench: A Suite for Vision-Language Process Reward Modeling [PDF]
  69. [arXiv'25] VisualPRM: An Effective Process Reward Model for Multimodal Reasoning [PDF]
  70. [arXiv'25] Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning [PDF]
  71. [arXiv'25] Demystifying Multilingual Chain-of-Thought in Process Reward Modeling [PDF]
  72. [arXiv'25] AURORA:Automated Training Framework of Universal Process Reward Models via Ensemble Prompting and Reverse Verification [PDF]
  73. [arXiv'25] VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data [PDF]
  74. [arXiv'25] Coarse-to-Fine Process Reward Modeling for Mathematical Reasoning [PDF]
  75. [arXiv'25] The Lessons of Developing Process Reward Models in Mathematical Reasoning [PDF]
  76. [arXiv'25] Unlocking Multimodal Mathematical Reasoning via Process Reward Model [PDF]
  77. [arXiv'24] Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning [PDF]
  78. [arXiv'24] Entropy-Regularized Process Reward Model [PDF]
  79. [arXiv'24] Process Reward Model with Q-Value Rankings [PDF]
  80. [arXiv'24] ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search [PDF]
  81. [arXiv'24] Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing [PDF]

Token-Level Credit

  1. [arXiv'26] CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization [PDF]
  2. [arXiv'26] Enhancing Rubric-based RL via Self-Distillation [PDF]
  3. [arXiv '26] Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks [PDF]
  4. [_] Direct Reasoning Optimization: Token-Level Reasoning Reflectivity Meets Rubric Gates for Unverifiable Tasks [PDF]
  5. [arXiv'25] KTAE: A Model-Free Algorithm to Key-Tokens Advantage Estimation in Mathematical Reasoning [PDF]
  6. [arXiv'25] RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution

Hindsight, Counterfactual, and Shapley Attribution

  1. [arXiv'26] Reducing Credit Assignment Variance via Counterfactual Reasoning Paths [PDF]
  2. [arXiv '26] Experience is the Best Teacher: Motivating Effective Exploration in Reinforcement Learning for LLMs [PDF]
  3. [arXiv'26] Listening to the Echo: User-Reaction Aware Policy Optimization via Scalar-Verbal Hybrid Reinforcement Learning [PDF]
  4. [arXiv'26] Writer-R1: Enhancing Generative Writing in LLMs via Memory-augmented Replay Policy Optimization [PDF]
  5. [arXiv'26] Hindsight Credit Assignment for Long-Horizon LLM Agents [PDF]
  6. [arXiv'25] SCAR: Shapley Credit Assignment for More Efficient RLHF [PDF]
  7. [arXiv'23] Would I Have Gotten That Reward? Long-Term Credit Assignment by Counterfactual Contribution Analysis
  8. [arXiv'19] Hindsight Credit Assignment

Return Redistribution and Value Models

  1. [arXiv'26] Beyond Uniform Credit Assignment: Selective Eligibility Traces for RLVR [PDF]
  2. [arXiv'24] Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning [PDF]
  3. [arXiv'18] RUDDER: Return Decomposition for Delayed Rewards [PDF]

Trajectory-Level Rewards

  1. [arXiv'26] STAMP: Provenance-Guided Credit Assignment for Deep Search Agents [PDF]
  2. [arXiv'26] Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning [PDF]
  3. [arXiv'26] Credit Assignment with Resets in Language Model Reasoning [PDF]
  4. [arXiv'26] From Reasoning Chains to Verifiable Subproblems: Curriculum Reinforcement Learning Enables Credit Assignment for LLM Reasoning [PDF]
  5. [arXiv'26] CAVE: A Structured Credit Assignment Approach for Fragmented Visual Evidence Reasoning [PDF]
  6. [arXiv'26] PiCA: Pivot-Based Credit Assignment for Search Agentic Reinforcement Learning [PDF]
  7. [arXiv'26] DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models [PDF]
  8. [arXiv'26] Reinforcing 3D Understanding in Point-VLMs via Geometric Reward Credit Assignment [PDF]
  9. [arXiv'26] APEX-Searcher: Refining Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation [PDF]
  10. [arXiv'26] Harmonizing Dense and Sparse Signals in Multi-turn RL: Dual-Horizon Credit Assignment for Industrial Sales Agents [PDF]
  11. [arXiv'26] HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents [PDF]
  12. [arXiv'26] Intrinsic Credit Assignment for Long Horizon Interaction [PDF]
  13. [arXiv'26] InT: Self-Proposed Interventions Enable Credit Assignment in LLM Reasoning [PDF]
  14. [arXiv'26] PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignment [PDF]
  15. [arXiv'26] Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modeling
  16. [arXiv'25] CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic [PDF]
  17. [arXiv'25] CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignment [PDF]
  18. [arXiv'25] Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models [PDF]
  19. [arXiv'25] QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning? [PDF]
  20. [arXiv'25] Leveraging Large Language Models for Effective and Explainable Multi-Agent Credit Assignment [PDF]
  21. [arXiv'25] Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning [PDF]
  22. [arXiv'25] GRPO-$\lambda$: Credit Assignment Improves LLM Reasoning [PDF]
  23. [arXiv'24] VinePPO: Refining Credit Assignment in RL Training of LLMs [PDF]

Scalarization and Multi-Objective Optimization

Normalization and Advantage Construction

  1. [arXiv'26] DecoEvo: Score-Decoupled Co-Evolution of Solver and Rubric-Generator Skills in Text Space [PDF]
  2. [arXiv'26] Prompt-Level Reward Specifications for Open-Ended Post-Training [PDF]
  3. [arXiv '26] Tournament-GRPO: Group-Wise Tournament Rewards for Reinforcement Learning in Open-Ended Long-Form Generation [PDF]
  4. [arXiv '26] Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges [PDF]
  5. [arXiv'26] RVPO: Risk-Sensitive Alignment via Variance Regularization [PDF]
  6. [arXiv'26] MGDA-Decoupled: Geometry-Aware Multi-Objective Optimisation for DPO-based LLM Alignment [PDF]
  7. [arXiv'26] Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO [PDF]
  8. [arXiv '26] Stabilizing Rubric Integration Training via Decoupled Advantage Normalization [PDF]
  9. [arXiv '26] Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces [PDF]
  10. [arXiv'26] Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning [PDF]
  11. [arXiv '26] PAPO: Stabilizing Rubric Integration Training via Decoupled Advantage Normalization [PDF]
  12. [arXiv'25] Conditional Advantage Estimation for Reinforcement Learning in Large Reasoning Models [PDF]
  13. [arXiv'25] ReDit: Reward Dithering for Improved LLM Policy Optimization [PDF]
  14. [arXiv'25] Accelerating RLHF Training with Reward Variance Increase [PDF]
  15. [arXiv'25] CPGD: Toward Stable Rule-based Reinforcement Learning for Language Models [PDF]
  16. [arXiv] Policy Invariance under Reward Transformations: Theory and Application to Reward Shaping [PDF]
  17. [arXiv] Policy Invariance under Reward Transformations for Multi-Objective Reinforcement Learning [PDF]

Nonlinear and Worst-Case Aggregation

  1. [arXiv '26] Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards [PDF]

Dependency-Aware Aggregation

  1. [arXiv '26] Mitigating False Credit Propagation: Probabilistic Graphical Reward Aggregation for Rubric-Based Reinforcement Learning [PDF]

Multi-Objective and Preference-Conditioned Policies

  1. [arXiv'26] Online Linear Programming for Multi-Objective Routing in LLM Serving [PDF]
  2. [arXiv'26] Multi-Objective Exploration and Preference Optimization via Mutual Information [PDF]
  3. [arXiv'26] MAPL: Multi-Objective Preference Learning for Robot Locomotion [PDF]
  4. [arXiv'26] Holistic Data Scheduler for LLM Pre-training via Multi-Objective Reinforcement Learning [PDF]
  5. [arXiv'26] SAW: Stage-Aware Dynamic Weighting for Multi-Objective Reinforcement Learning in Large Language Models [PDF]
  6. [arXiv'26] MATO: Multi-objective Personalized Alignment with Test-time Optimization for Large Language Models [PDF]
  7. [arXiv'26] ERFSL: An Efficient Reward Function Searcher via Language Models for Custom-Environment Multi-Objective Optimization (Student Abstract) [PDF]
  8. [arXiv'26] C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs [PDF]
  9. [arXiv'26] One Model for All: Multi-Objective Controllable Language Models [PDF]
  10. [arXiv'26] Provable Last-Iterate Convergence for Multi-Objective Safe LLM Alignment via Optimistic Primal-Dual [PDF]
  11. [arXiv'26] Back to Blackwell: Closing the Loop on Intransitivity in Multi-Objective Preference Fine-Tuning [PDF]
  12. [arXiv'26] Multi-Objective Alignment of Language Models for Personalized Psychotherapy [PDF]
  13. [arXiv'26] Uncovering Cross-Objective Interference in Multi-Objective Alignment [PDF]
  14. [arXiv'25] Multi-Objective Reward and Preference Optimization: Theory and Algorithms [PDF]
  15. [arXiv'25] MOA: Multi-Objective Alignment for Role-Playing Agents [PDF]
  16. [arXiv'25] FIRM: Federated In-client Regularized Multi-objective Alignment for Large Language Models [PDF]
  17. [arXiv'25] Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning [PDF]
  18. [arXiv'25] Preference Orchestrator: Prompt-Aware Multi-Objective Alignment for Large Language Models [PDF]
  19. [arXiv'25] Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards [PDF]
  20. [arXiv'25] OrthAlign: Orthogonal Subspace Decomposition for Non-Interfering Multi-Objective Alignment [PDF]
  21. [arXiv'25] VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shaping [PDF]
  22. [arXiv'25] Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting [PDF]
  23. [arXiv'25] AI-SearchPlanner: Modular Agentic Search via Pareto-Optimal Multi-Objective Reinforcement Learning [PDF]
  24. [arXiv'25] Pareto Multi-Objective Alignment for Language Models [PDF]
  25. [Findings of ACL: EMNLP '25] QA-LIGN: Aligning LLMs through Constitutionally Decomposed QA [PDF]
  26. [arXiv'25] AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models [PDF]
  27. [arXiv'25] PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model [PDF]
  28. [arXiv'25] EMORL: Ensemble Multi-Objective Reinforcement Learning for Efficient and Flexible LLM Fine-Tuning [PDF]
  29. [arXiv'25] REWARD CONSISTENCY: Improving Multi-Objective Alignment from a Data-Centric Perspective [PDF]
  30. [arXiv'25] Robust Multi-Objective Controlled Decoding of Large Language Models [PDF]
  31. [arXiv'25] UC-MOA: Utility-Conditioned Multi-Objective Alignment for Distributional Pareto-Optimality [PDF]
  32. [arXiv'25] Projection Optimization: A General Framework for Multi-Objective and Multi-Group RLHF [PDF]
  33. [arXiv'25] Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models
  34. [arXiv'24] Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement [PDF]
  35. [ENNLP (Findings) '24] Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts [PDF]
  36. [arXiv'24] Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards [PDF]
  37. [arXiv'24] Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment [PDF]
  38. [arXiv'24] Panacea: Pareto Alignment via Preference Adaptation for LLMs
  39. [arXiv'24] Multi-Objective Alignment of Large Language Models Through Hypervolume Maximization [PDF]
  40. [arXiv'24] Multi-Objective Reinforcement Learning: A Tool for Pluralistic Alignment [PDF]
  41. [arXiv'23] Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization [PDF]
  42. [arXiv'21] A Practical Guide to Multi-Objective Reinforcement Learning and Planning [PDF]
  43. [arXiv'14] A Survey of Multi-Objective Sequential Decision-Making [PDF]

Adaptive and Co-Evolving Rubrics

Self-Evolving and Adversarial Rubrics

  1. [arXiv'26] SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning [PDF]
  2. [arXiv'26] Co-Evolving LLM Evaluators and Policies via DynamicRubric [PDF]
  3. [arXiv'26] Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence [PDF]
  4. [arXiv'26] SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use [PDF]
  5. [arXiv '26] EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learning [PDF]
  6. [arXiv '26] ARCO: ADAPTIVE RUBRIC WITH CO-EVOLUTION FOR MULTI-STEP LLM-BASED AGENTS [PDF]
  7. [arXiv '26] Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubrics [PDF]
  8. [arXiv'26] Self-Evolving Deep Research via Joint Generation and Evaluation [PDF]
  9. [arXiv '26] EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generation [PDF]
  10. [arXiv'26] CODESKILL: Learning Self-Evolving Skills for Coding Agents [PDF]
  11. [arXiv'26] SHARP: A Self-Evolving Human-Auditable Rubric Policy for Financial Trading Agents [PDF]
  12. [arXiv '26] EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubrics [PDF]
  13. [arXiv'26] MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment [PDF]
  14. [arXiv '26] SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probing [PDF]
  15. [arXiv '26] Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics [PDF]
  16. [arXiv '26] Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification [PDF]
  17. [arXiv '26] CoReflect: Conversational Evaluation via Co-Evolutionary Simulation and Reflective Rubric Refinement [PDF]
  18. [arXiv'26] ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agents [PDF]
  19. [ICML '26] DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research [PDF]
  20. [ICLR '26] RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks [PDF]

Online and Alternating Optimization

  1. [arXiv'26] LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL [PDF]
  2. [arXiv '26] RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains [PDF]
  3. [arXiv '26] Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR [PDF]
  4. [arXiv '26] AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning [PDF]
  5. [arXiv'26] SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models [PDF]
  6. [arXiv '26] Alternating Reinforcement Learning with Contextual Rubric Rewards [PDF]
  7. [ICML '26] Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Training [PDF]
  8. [arXiv'26] Alternating Reinforcement Learning with Contextual Rubric Rewards: Beyond the Scalarization Strategy [PDF]

Reliability, Reward Hacking, and Security

Reward Hacking

  1. [arXiv'26] LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks [PDF]
  2. [arXiv '26] Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning [PDF]
  3. [arXiv '26] Reinforcement Learning with Robust Rubric Rewards [PDF]
  4. [arXiv'26] Reward Hacking in Rubric-Based Reinforcement Learning [PDF]
  5. [arXiv '26] Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Text [PDF]
  6. [ICML '26] InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training [PDF]
  7. [arXiv '25] R3: Robust Rubric-Agnostic Reward Models [PDF]

Judge Bias and Reliability

  1. [arXiv'26] When Rubrics Change: Cross-Rubric Generalization for Critical Thinking Essay Scoring [PDF]
  2. [arXiv'26] FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning [PDF]
  3. [arXiv '26] Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic Scenarios? [PDF]
  4. [arXiv'26] ExpRL: Exploratory RL for LLM Mid-Training [PDF]
  5. [arXiv '26] Reliable to Expressive: A Curriculum for Rubric-Following Safety Judges [PDF]
  6. [arXiv'26] EDIT: Evidence-Diagnosed Intervention Training for Rule-Faithful LLM Grading [PDF]
  7. [arXiv'26] ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization [PDF]
  8. [arXiv '26] RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards [PDF]
  9. [arXiv'26] The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs [PDF]
  10. [arXiv '26] Rubric-based On-policy Distillation [PDF]
  11. [arXiv '26] Quantifying the Statistical Effect of Rubric Modifications on Human-Autorater Agreement [PDF]
  12. [arXiv '26] LLM-as-a-Judge for Human-AI Co-Creation: A Reliability-Aware Evaluation Framework for Coding [PDF]
  13. [arXiv '26] WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning [PDF]
  14. [arXiv'26] Measurement Risk in Supervised Financial NLP: Rubric and Metric Sensitivity on JF-ICR [PDF]
  15. [arXiv '26] Case-Specific Rubrics for Clinical AI Evaluation: Methodology, Validation, and LLM-Clinician Agreement Across 823 Encounters [PDF]
  16. [arXiv '26] Self-Preference Bias in Rubric-Based Evaluation of Large Language Models [PDF]
  17. [arXiv'26] Pressure, What Pressure? Sycophancy Disentanglement in Language Models via Reward Decomposition [PDF]
  18. [ICLR '26] Optimsyn: Influence-Guided Rubrics Optimization for Synthetic Data Generation [PDF]
  19. [arXiv '26] Comparing Developer and LLM Biases in Code Evaluation [PDF]
  20. [arXiv '26] AdaRubric: Task-Adaptive Rubrics for LLM Agent Evaluation [PDF]
  21. [arXiv '26] RubricRAG: Towards Interpretable and Reliable LLM Evaluation via Domain Knowledge Retrieval for Rubric Generation [PDF]
  22. [arXiv '26] CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling [PDF]
  23. [arXiv '26] When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On [PDF]
  24. [arXiv '26] FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge [PDF]
  25. [arXiv '26] Am I More Pointwise or Pairwise? Revealing Position Bias in Rubric-Based LLM-as-a-Judge [PDF]
  26. [arXiv'26] RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions [PDF]
  27. [arXiv'26] From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judges [PDF]
  28. [arXiv '25] Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge [PDF]
  29. [arXiv '25] AutoRubric: Rubric-Based Generative Rewards for Faithful Multimodal Reasoning [PDF]
  30. [arXiv'25] MENLO: From Preferences to Proficiency -- Evaluating and Modeling Native-like Quality Across 47 Languages [PDF]
  31. [arXiv '25] Curse of Knowledge: When Complex Evaluation Context Benefits yet Biases LLM Judges [PDF]
  32. [arXiv'25] Are Checklists Really Useful for Automatic Evaluation of Generative Tasks? [PDF]
  33. [arXiv'25] Kimi K2: Open Agentic Intelligence [PDF]
  34. [arXiv '25] Evaluating Scoring Bias in LLM-as-a-Judge [PDF]
  35. [arXiv'25] Do We Need a Detailed Rubric for Automated Essay Scoring using Large Language Models? [PDF]
  36. [arXiv '25] EvalAgent: Discovering Implicit Evaluation Criteria from the Web [PDF]
  37. [arXiv'24] Finding Blind Spots in Evaluator LLMs with Interpretable Checklists [PDF]

Attack Surface and Safety

  1. [arXiv '26] RUBAS: Rubric-Based Reinforcement Learning for Agent Safety [PDF]
  2. [arXiv '26] Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges [PDF]
  3. [arXiv'17] Constrained Policy Optimization [PDF]

Benchmarks and Evaluation

Rubric Benchmarks and Datasets

  1. [arXiv'26] Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution [PDF]
  2. [arXiv'26] POLARIS: Guiding Small Models to Write Long Stories [PDF]
  3. [arXiv '26] QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards [PDF]
  4. [arXiv '26] AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following [PDF]
  5. [arXiv '26] MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection [PDF]
  6. [arXiv '26] Learnable Assessment Skills for LLM-based Automated Scoring: Rubric Construction via Iterative Optimization [PDF]
  7. [arXiv '26] JudgmentBench: Comparing Rubric and Preference Evaluation for Quality Assessment [PDF]
  8. [arXiv '26] Enhancing Judgment Document Generation via Agentic Legal Information Collection and Rubric-Guided Optimization [PDF]
  9. [arXiv '26] ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents [PDF]
  10. [arXiv'26] Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization [PDF]
  11. [arXiv '26] Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation [PDF]
  12. [arXiv '26] RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following [PDF]
  13. [arXiv '26] PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation [PDF]
  14. [arXiv '26] Qworld: Question-Specific Evaluation Criteria for LLMs [PDF]
  15. [arXiv '26] RubricBench: Aligning Model-Generated Rubrics with Human Standards [PDF]
  16. [arXiv '26] When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification [PDF]
  17. [arXiv '26] Mock Worlds, Real Skills: Building Small Agentic Language Models with Synthetic Tasks, Simulated Environments, and Rubric-Based Rewards [PDF]
  18. [arXiv '26] DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Report [PDF]
  19. [arXiv '26] RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generation [PDF]
  20. [arXiv'26] ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking [PDF]
  21. [arXiv '26] TechImage-Bench: Rubric-Based Evaluation for Technical Image Generation [PDF]
  22. [arXiv '25] PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning [PDF]
  23. [arXiv '25] AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following [PDF]
  24. [arXiv '25] ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents [PDF]
  25. [arXiv'25] Benchmarking and Learning Real-World Customer Service Dialogue [PDF]
  26. [arXiv '25] ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge [PDF]
  27. [arXiv'25] TripScore: Benchmarking and rewarding real-world travel planning with fine-grained evaluation [PDF]
  28. [arXiv '25] MTalk-Bench: Evaluating Speech-to-Speech Models in Multi-Turn Dialogues via Arena-style and Rubrics Protocols [PDF]
  29. [arXiv '25] Configurable Preference Tuning with Rubric-Guided Synthetic Data [PDF]
  30. [arXiv '25] ExpertLongBench: Benchmarking Language Models on Expert-Level Long-Form Generation Tasks with Structured Checklists [PDF]
  31. [arXiv '25] HealthBench: Evaluating Large Language Models Towards Improved Human Health [PDF]
  32. [arXiv '25] Concept-based Rubrics Improve LLM Formative Assessment and Data Synthesis [PDF]
  33. [Findings of EMNLP '24] SedarEval: Automated Evaluation using Self-Adaptive Rubrics [PDF]
  34. [arXiv'24] AgentBench: Evaluating LLMs as Agents
  35. [arXiv'24] WebArena: A Realistic Web Environment for Building Autonomous Agents
  36. [NeurIPS '23] Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
  37. [arXiv'22] Constitutional AI: Harmlessness from AI Feedback [PDF]
  38. [ICML '25] PaperBench: Evaluating AI's Ability to Replicate AI Research [PDF]

Evaluation Protocols for Dense Supervision

  1. [arXiv'26] QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents [PDF]

Applications

Deep Research and Search Agents

  1. [arXiv'26] Designing Reward Signals for Portable Query Generation: A Case Study in Industrial Semantic Job Search [PDF]
  2. [arXiv '26] DEEPRUBRIC: Evidence-Tree Rubric Supervision for Efficient Reinforcement Learning of Deep Research Agents [PDF]
  3. [arXiv '26] Deep Research as Rubric for Reinforcement Learning [PDF]
  4. [arXiv '26] LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards [PDF]
  5. [arXiv'26] QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks [PDF]
  6. [arXiv '26] Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation [PDF]
  7. [arXiv '26] Chaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric Rewards [PDF]
  8. [arXiv'26] Inference-Time Budget Control for LLM Search Agents
  9. [arXiv'25] Step-DeepResearch Technical Report [PDF]
  10. [arXiv'25] Beyond Query-Level Comparison: Fine-Grained Reinforcement Learning for Text-to-SQL with Automated Interpretable Critiques [PDF]
  11. [arXiv'25] ARISE: Agentic Rubric-Guided Iterative Survey Engine for Automated Scholarly Paper Generation [PDF]
  12. [arXiv'25] Rethinking the Design of Reinforcement Learning-Based Deep Research Agents [PDF]
  13. [arXiv'25] ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards [PDF]
  14. [arXiv'25] Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrieval [PDF]
  15. [arXiv '25] ResearchQA: Evaluating Scholarly Question Answering at Scale Across 75 Fields with Survey-Mined Questions and Rubrics [PDF]

Software Engineering and Code

  1. [arXiv'26] DobicVLM: Aligning Chest X-Ray Report Generation with Clinically-Grounded Programmatic Rewards via Group Relative Policy Optimization [PDF]
  2. [arXiv'26] SCOPE: Leveraging Subgoal Critiques for Code Generation [PDF]
  3. [arXiv'26] Incentivizing Vision Language Models to Search for Long Video Question Answering [PDF]
  4. [arXiv '26] Beyond Verifiable Rewards: Rubric-Based GRM for Reinforced Fine-Tuning SWE Agents [PDF]
  5. [arXiv'26] Thinking with Spatial Code for Physical-World Video Reasoning [PDF]
  6. [arXiv '26] StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning [PDF]
  7. [arXiv'24] SWE-Agent: Agent-Computer Interfaces Enable Automated Software Engineering

Medicine and Health

  1. [arXiv'26] RubricsTree: Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skills [PDF]
  2. [arXiv '26] CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubrics [PDF]
  3. [arXiv'26] ProMedical: Hierarchical Fine-Grained Criteria Modeling for Medical LLM Alignment via Explicit Injection [PDF]
  4. [ICLR '26 Workshop DATA-FM,] RIFT: A RubrIc Failure Mode Taxonomy and Automated Diagnostics [PDF]
  5. [arXiv'26] MIND: Unified Inquiry and Diagnosis RL with Criteria Grounded Clinical Supports for Psychiatric Consultation [PDF]
  6. [arXiv'26] Health-SCORE: Towards Scalable Rubrics for Improving Health-LLMs [PDF]
  7. [arXiv'25] Adaptive Reinforcement for Open-ended Medical Reasoning via Semantic-Guided Reward Collapse Mitigation [PDF]

Multimodal, Vision, and Generation

  1. [arXiv'26] Claim-Level Rubric Rewards for Video Caption Reinforcement Learning [PDF]
  2. [arXiv'26] Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation [PDF]
  3. [arXiv'26] Reward Design for Physical Reasoning in Vision-Language Models [PDF]
  4. [arXiv '26] Visual Preference Optimization with Rubric Rewards [PDF]
  5. [arXiv'26] RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning [PDF]
  6. [arXiv '26] OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis [PDF]
  7. [LREC '26,] Rubric-guided fine-tuning of speechllms for multi-aspect, multi-rater l2 reading-speech assessment [PDF]
  8. [arXiv'26] CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning [PDF]
  9. [ICML '26] RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning [PDF]
  10. [arXiv '26] iRULER: Intelligible Rubric-Based User-Defined LLM Evaluation for Revision [PDF]
  11. [arXiv '26] SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models [PDF]
  12. [arXiv '25] RubricRL: Simple Generalizable Rewards for Text-to-Image Generation [PDF]
  13. [arXiv'25] Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning [PDF]
  14. [arXiv'25] MOSS-ChatV: Reinforcement Learning with Process Reasoning Reward for Video Temporal Reasoning [PDF]
  15. [arXiv '25] Compute as Teacher: Turning Inference Compute Into Reference-Free Supervision [PDF]

Writing, Social, and Dialogue

  1. [arXiv'26] Zing: Social Mind for LLMs [PDF]
  2. [arXiv'25] CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization [PDF]
  3. [arXiv'25] Sotopia-RL: Reward Design for Social Intelligence [PDF]
  4. [arXiv '25] Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewards [PDF]

Science, Math, and Education

  1. [arXiv'26] Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards [PDF]
  2. [arXiv'26] Reinforcement Learning for Special Education: Aligning LLM Tutors to Diverse Learners through Disability-Adaptive Training [PDF]
  3. [arXiv'26] Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals [PDF]
  4. [arXiv '26] Think-with-Rubrics: From External Evaluator to Internal Reasoning Guidance [PDF]
  5. [arXiv'26] Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards [PDF]
  6. [arXiv'26] Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning [PDF]
  7. [arXiv '26] Optimizing In-Context Demonstrations for LLM-based Automated Grading [PDF]
  8. [arXiv '26] Confusion-Aware Rubric Optimization for LLM-based Automated Grading [PDF]
  9. [arXiv '26] Improving Data and Reward Design for Scientific Reasoning in Large Language Models [PDF]
  10. [arXiv'26] Skill-Conditioned Gated Self-Distillation for LLM Reasoning
  11. [arXiv '25] Training AI Co-Scientists Using Rubric Rewards [PDF]
  12. [arXiv'25] TableGPT-R1: Advancing Tabular Reasoning Through Reinforcement Learning [PDF]
  13. [arXiv '26] Rubric-Conditioned LLM Grading: Alignment, Uncertainty, and Robustness [PDF]
  14. [arXiv '25] Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics [PDF]
  15. [arXiv '25] DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning [PDF]
  16. [arXiv '25] Reward and Guidance through Rubrics: Promoting Exploration to Improve Multi-Domain Reasoning [PDF]
  17. [arXiv '25] mR3: Multilingual Rubric-Agnostic Reward Reasoning Models [PDF]
  18. [arXiv '25] Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning [PDF]
  19. [arXiv '25] Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning [PDF]
  20. [arXiv'25] DeepSeek-R1 Incentivizes Reasoning in LLMs Through Reinforcement Learning
  21. [arXiv'24] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models [PDF]
  22. [arXiv'23] ReAct: Synergizing Reasoning and Acting in Language Models
  23. [arXiv] The cot encyclopedia: Analyzing, predicting, and controlling how a reasoning model will think [PDF]

Instruction Following and Safety Alignment

  1. [arXiv'26] Rewarding Better Thinking for LLM Preference Alignment [PDF]
  2. [arXiv'26] RubriQ: Rubric-Guided Group Relative Policy Optimization for Constraint-Aware Quantum Circuit Synthesis [PDF]
  3. [arXiv '25] ARCANE: A Multi-Agent Framework for Interpretable and Configurable Alignment [PDF]
  4. [arXiv'25] Beyond Monolithic Rewards: A Hybrid and Multi-Aspect Reward Optimization for MLLM Alignment [PDF]
  5. [arXiv'25] Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factuality [PDF]
  6. [arXiv '25] ACE-RL: Adaptive Constraint-Enhanced Reward for Long-form Generation Reinforcement Learning [PDF]
  7. [arXiv] VERIF: Verification Engineering for Reinforcement Learning in Instruction Following [PDF]

Other Domains

  1. [arXiv'26] Unified Context Evolution for LLM Agents
  2. [arXiv'25] Enhancing Tool Learning in Large Language Models with Hierarchical Error Checklists [PDF]
  3. [arXiv'25] AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging
  4. [arXiv'25] ARIA: Training Language Agents with Intention-Driven Reward Aggregation
  5. [arXiv'25] Recursive Reward Aggregation
  6. [arXiv'24] AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learning
  7. [arXiv'24] Tulu 3: Pushing Frontiers in Open Language Model Post-Training [PDF]
  8. [arXiv'23] Toolformer: Language Models Can Teach Themselves to Use Tools
  9. [arXiv'22] Training Language Models to Follow Instructions with Human Feedback
  10. [arXiv'22] Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback [PDF]
  11. [arXiv'20] Learning to Summarize with Human Feedback
  12. [arXiv'19] Fine-Tuning Language Models from Human Preferences [PDF]
  13. [arXiv'18] An Analysis of Categorical Distributional Reinforcement Learning
  14. [arXiv'17] Successor Features for Transfer in Reinforcement Learning
  15. [arXiv'17] Deep Reinforcement Learning from Human Preferences
  16. [arXiv'17] Proximal Policy Optimization Algorithms [PDF]
  17. [arXiv] PROMETHEUS 2: An Open Source Language Model Specialized in Evaluating Other Language Models [PDF]
  18. [arXiv] Microverse: A preliminary exploration toward a micro-world simulation [PDF]
  19. [arXiv] Deepseek-v4: Towards highly efficient million-token context intelligence [PDF]

Come and Join Us! 👊

Contribution

Feel free to open a pull request or contact us if you find related papers that are not included here.

  1. Fork the project into your own repository.
  2. Add the paper under the matching section using the following format:
N. `[Venue'YY]` Paper Title [[PDF](paper link)] [[Code](code link)]
  1. Submit the pull request to this branch.

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors