Ask a large language model a question about something it learned during training, and it often answers well. Ask about last week's news, your company's policies, or an obscure fact, and it may answer confidently and wrongly. The knowledge a model has is frozen into its weights. It is hard to check where an answer came from, and to update a fact you would have to retrain.
This 2020 paper from Facebook AI Research, University College London and NYU proposed a simple remedy that has become the standard pattern for AI products that must stay accurate: retrieval-augmented generation, or RAG. Before answering, look up relevant documents, and let the model write its answer with those documents in front of it.
RAG combines a language model's built-in knowledge (parametric memory) with a searchable document store (non-parametric memory). For each question it retrieves the most relevant passages, then generates an answer conditioned on them. The knowledge can be updated by changing the index, not the model.
Two kinds of memory
The paper opens with a limitation. Pre-trained language models store factual knowledge in their parameters, but their ability to access and precisely use that knowledge is limited. They also cannot easily show where an answer came from or be updated when the world changes. Earlier systems that did look things up were mostly built for narrow tasks, such as extracting an answer span from a passage.
The authors build a general-purpose version. It works for any task where the output is text, from answering open questions to writing a short fact-checked statement.
