Retrieval-Augmented Generation (RAG)
GenAI SystemsGive an LLM an open-book exam: before it answers, look up the relevant passages in your own documents and put them in the prompt, so the answer comes from real sources instead of memory.
A student answering from memory may confidently get a detail wrong, and cannot know anything that happened after they stopped studying. In an open-book exam the student first finds the right pages in the textbook, then writes the answer with those pages in front of them. RAG does that for an LLM: look up first, then answer.
The problem it solves. A language model only knows what was in its training data, which stops at a cutoff date and never included your private documents. When it does not know something, it often produces a fluent, plausible answer anyway (a hallucination). Retraining the model every time your documents change is far too slow and expensive.
Step 1, index your documents (done ahead of time). Split each document into chunks of a few paragraphs. Turn each chunk into an embedding, a list of numbers that captures its meaning. Store the embeddings, with the original text, in a vector database.
Step 2, answer a question (done on every request). Turn the question into an embedding in the same way. Ask the vector database for the chunks whose embeddings are closest. Paste those chunks into the prompt together with the question, and tell the model to answer using only that material. The model writes its answer from the pages in front of it.
What you gain. Answers can be grounded in, and cite, specific sources. Knowledge is updated by re-indexing a changed document, with no retraining. Private data stays in your own store instead of being baked into model weights.
What it does not fix. RAG is only as good as its retrieval. If the wrong chunks come back, the model answers confidently from the wrong pages. Every request also costs more: a longer prompt (more tokens) and an extra lookup before the model even starts. Real systems add careful chunking, hybrid keyword-and-vector search and a reranking step to improve what gets retrieved.
- RAG changes what the model sees at question time, not what it was trained on.
- Two phases: index the documents ahead of time, then retrieve and generate on each question.
- Most RAG failures are retrieval failures: when answers are wrong, check which chunks came back first.
- Citing the retrieved chunk lets a user verify the answer, but a citation is not proof the answer is faithful to it.
- Fresh knowledge means re-indexing, not retraining.
Curated Curation & Deep Insights
ScaleDojo CertifiedOur system architects are vetting high-quality, authorized video guides for Retrieval-Augmented Generation (RAG) with zero third-party platform links.
We are preparing premium, zero-competitor deep-dives for Retrieval-Augmented Generation (RAG). Authorized reading references will appear automatically.