A new kind of chatbot stops to think. It writes a long chain of reasoning before it answers, and it does better on hard math and coding tests. These are called large reasoning models. But are they really reasoning, or just getting better at test questions they may have seen before?
A team at Apple tried a cleaner test. Instead of exam questions, they used puzzles such as the Tower of Hanoi, where the difficulty can be turned up one notch at a time, and the rules never change. Then they watched what the models did.
On easy puzzles, ordinary models did as well as thinking models, and used fewer tokens. On medium puzzles, thinking helped. On hard ones, every model collapsed to near zero accuracy. Oddly, the thinking models also began to think less as puzzles got harder, and even being handed the solution steps did not rescue them. The paper has been debated.
Puzzles with a difficulty dial
Popular reasoning tests have a problem. The questions may already be in the training data, and a final answer says nothing about how it was reached. The authors used four puzzles instead: Tower of Hanoi, Checker Jumping, River Crossing and Blocks World. Each has a size N that sets its difficulty: the number of disks, checkers, people to ferry across a river, or blocks.
A simulator checks every move the model proposes. That lets the researchers measure accuracy, and also look inside the model's reasoning to see which partial solutions it tried and where it went wrong.
Try it: the Tower of Hanoi
Move every disk from the first peg to the last. You may move one disk at a time, and never place a bigger disk on a smaller one. The fewest possible moves for N disks is 2 to the power N, minus 1. That number grows fast, and a model that writes the solution must get every one of those moves right.
Three regimes
The authors compared pairs of models with the same backbone, one that thinks and one that does not: Claude 3.7 Sonnet with and without extended thinking, and DeepSeek-R1 against DeepSeek-V3. They gave each pair the same amount of computing.
- Easy puzzles. The non-thinking model matched or beat the thinking one, with fewer tokens.
- Medium puzzles. Thinking models pulled ahead, but spent many more tokens to do it.
- Hard puzzles. Both kinds fell to near zero. Thinking only postponed the collapse.
This held across the reasoning models they tested, including o3-mini, DeepSeek-R1 and Claude 3.7 Sonnet with thinking. Accuracy slid down as the puzzles grew, until it reached a model-specific point after which it was close to zero.
Thinking less when it is hardest
The most surprising result is about effort. As puzzles grew harder the models first wrote longer reasoning, which is what you would hope. But close to the collapse point, the reasoning got shorter, even though the models were well below their length limits. It was strongest in o3-mini and milder in Claude 3.7 Sonnet with thinking.
Looking inside the traces showed more. On easy problems, a model often found the right answer early and then kept going, a pattern the authors call overthinking. On harder ones, it often fixated on an early wrong answer. On the hardest, it found no correct solution at all.
Even with the recipe, they stumble
In the next test the authors put the full solution algorithm for Tower of Hanoi into the prompt. The model then only had to carry out the steps. Performance did not improve, and the collapse came at about the same puzzle size. Carrying out a long, exact procedure was itself a weak spot.
They also found inconsistency across puzzles. A model could do the 31-move Tower of Hanoi with five disks nearly perfectly, yet fail a River Crossing puzzle with three pairs, which needs only 11 moves.
The paper is a case study, and other researchers have pushed back. A comment posted by other researchers argued that some of the apparent failures came from output-length limits and from River Crossing instances that cannot be solved, and that asking a model for a short program instead of a full list of moves gave high accuracy on Tower of Hanoi. The debate is a good reminder to check how an AI test was built before trusting a headline.
Key takeaways: Puzzles with a difficulty dial let researchers see where reasoning models break. Standard models win on easy tasks, thinking models win on medium ones, and all fail on hard ones. Models can think less just when problems get hardest. Check how a benchmark was built: the paper's claims are debated.
Go deeper
- Read the original: The Illusion of Thinking (Shojaee et al., Apple)
- PPO, RLHF and reasoning models: how thinking models are trained
- DPO and test-time compute: what thinking longer buys you, and what it costs
- Planning and task decomposition: how agents break down long tasks
