Today's chatbots can take in enormous prompts: hundreds of pages in one go. It is tempting to treat that as a hard drive. Paste in everything, ask a question, and trust the model to find the right line.
A Stanford, Berkeley and Samaya AI team tested that trust in a simple way. They hid the answer to a question inside a pile of other documents, and moved it around. The model could read every document equally. Did it use them equally?
No. Models answered best when the useful passage was at the very start or the very end of the input, and worst when it sat in the middle, a U-shaped curve. With the answer in the middle, one popular model did worse than if it had been given no documents at all. Longer-context versions of models were not better at using the extra room.
How the test works
The main experiment is multi-document question answering. Each question comes from NaturalQuestions, a set of real search queries. The prompt holds a number of Wikipedia passages: 10, 20 or 30. Exactly one passage contains the answer. The rest are distractors that look related but do not. The authors then reorder the passages so the answer appears first, last, or anywhere in between, and measure how often the model answers correctly.
They tested open models, MPT-30B-Instruct and LongChat-13B, and closed ones, OpenAI's GPT-3.5-Turbo and Anthropic's Claude-1.3, plus versions with longer context windows. Two reference points help. In the closed-book setting the model sees no documents. In the oracle setting it sees only the one document with the answer. For GPT-3.5-Turbo these scores were 56.1 percent and 88.3 percent.
