Most tests of language models give the model one complete question and score the answer. Real conversations are not like that. People start with half an idea, add details as they think of them, and correct themselves along the way. A chat is a conversation, and it is often a messy one.
This paper asks a simple question: does a model do as well when it receives the same information a little at a time? The answer, across fifteen well-known models, is no. The paper won an Outstanding Paper award at ICLR 2026.
The researchers took tasks that models do well in one message and fed them piece by piece, one piece per turn. Average performance fell by about a quarter. The models were not much less capable. They were far less reliable, because they answered too early and then stuck to their first guess.
How the test works
The team started from existing benchmarks across six kinds of work: writing code, database queries, API actions, math, describing data in text, and summaries. They turned each instruction into a list of small pieces that they call shards. Each shard holds one fact from the original instruction.
A simulated user, itself an AI, then revealed one shard per turn and replied naturally to the model. Nothing was hidden or changed in meaning. In all, the study ran more than 200,000 simulated conversations. It also ran the single-message version of each task as a baseline, so any difference comes from how the information arrived.
