Skip to content
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? 4 min

Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, Gao Huang · arXiv, NeurIPS Runner-Up 2025

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

A NeurIPS 2025 runner-up asks if RL teaches reasoning models anything new. Slide the number of tries and see why the answer was surprising.

Read the original paper

AI & ML · Yellow Belt · Pro · 4 min read

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

After DeepSeek-R1, a popular story took hold. Reinforcement learning with verifiable rewards, or RLVR, lets a model practice on math and code, get a reward only when the answer checks out, and keep improving. Many people expected that this would teach models genuinely new ways of reasoning, as reinforcement learning taught game-playing agents new strategies.

A Tsinghua and Shanghai Jiao Tong team asked whether that is actually true. Their answer, from a paper that received a Runner-Up award at NeurIPS 2025, is a careful no, at least for the training methods and models they tested.

RLVR-trained models beat their base models when each gets one try. But when both get many tries, the base model solves as many problems, or more. The correct reasoning paths of the RL model were already inside the base model. RL mostly made those paths likelier to be chosen. Distillation from a stronger model did add new reasoning.

A better question: what could the model ever solve?

The usual score is accuracy on one attempt. That tells you how often a model is right at first try. It does not tell you what it is capable of with more attempts. The paper uses a metric called pass@k: give the model k tries at a problem, and count the problem as solved if any one of them is correct.

When k is 1, pass@k is ordinary accuracy. When k is large, such as 256 or 1,024, it measures the boundary of what the model can do at all. Nobody would ship a product that needs 256 tries. The point is to ask whether RL pushed that boundary outward. To avoid crediting lucky guesses, the authors checked the reasoning chains by hand for samples of math problems, and used coding tasks with unit tests.

The paper's picture of what reinforcement learning does. It makes a path the model could already take far more likely, so the score on one try goes up. It does not, by itself, add paths the base model could never reach.

Keep reading with Pro

This paper is part of Pro. Pro and Architect open the full breakdown, with animated diagrams, interactive elements you can play with and a quiz. A few papers stay free to read in full.

Still to read

3 more sections, 1 more diagram and 2 interactive elements to play with

  • What they found
  • Why the paths were already there
  • Distillation is different

Found this breakdown useful?

Share it with someone else wrestling with this paper.

Discussion0

Join the Discussion

Sign in to leave comments, reply to others, or like insights.

Sign In to ScaleDojo

No comments yet. Be the first to start the thread!

Enjoyed this? Get more like it.

New paper breakdowns, levels, and one concept worth knowing, straight to your inbox.

No spam, ever. Unsubscribe in one click.

Enjoyed this content?

Your support keeps us creating free resources

We put a lot of hours into researching and writing these guides. If it helped you, consider buying us a coffee. Every bit goes toward keeping ScaleDojo's content free and growing.

$

One-time payment via Stripe. ScaleDojo account required.