Everyone Was Building the Same Plumbing
By late 2022, every team building AI applications was writing the same boilerplate code. Chunk documents. Embed them. Store in a vector database. Retrieve relevant passages. Construct a prompt. Call an LLM. Parse the output. Each team built their own version from scratch. Each version had slightly different bugs. Everyone was solving the same problems in slightly different ways, and none of it was the interesting part of their application.
LangChain (Harrison Chase, October 2022) and LlamaIndex (Jerry Liu, November 2022) emerged to solve this problem. They are framework libraries that provide pre-built, tested components for the entire RAG and agent pipeline, so you can assemble a working system in dozens of lines instead of thousands. They did for GenAI what Express.js did for Node.js web servers: eliminated the boilerplate so you could focus on your actual product.
LangChain: The Swiss Army Knife
LangChain's core abstraction is the chain, which is a sequence of components that process data in order. A basic RAG chain might look like: DocumentLoader into TextSplitter into Embeddings into VectorStore into Retriever into PromptTemplate into LLM into OutputParser. Each component is pluggable. Swap Pinecone for Chroma. Swap GPT-4 for Claude. Swap recursive splitting for semantic splitting. The chain handles all the wiring between components.
Beyond chains, LangChain provides several key abstractions:
Agents: LLMs that use tools through the ReAct pattern. LangChain handles the reasoning loop, tool execution, and observation parsing automatically.
Memory: conversation history management. Short-term memory stores recent messages. Long-term memory summarizes and stores older conversations. Window memory keeps the last N exchanges.
Callbacks: hooks for logging, monitoring, and tracing. Every chain step can be observed, timed, and recorded. This is essential for debugging and evaluation.
Integrations: connectors for dozens of LLM providers, vector databases, document loaders, and tools. LangChain has more integrations than any other framework.
LangChain became the default framework for prototyping GenAI applications. When a developer wanted to build a chatbot, a RAG system, or an agent, LangChain was usually the first tool they reached for. The community grew rapidly, with thousands of examples, tutorials, and extensions.
The Criticism
LangChain is sometimes called 'over-abstracted,' and the criticism has merit. Simple tasks that would take 10 lines of raw API calls can require understanding multiple LangChain abstraction layers, class hierarchies, and configuration patterns. The framework makes easy things easy but sometimes makes simple things complicated.
For production systems, many teams found themselves fighting the framework's assumptions. When LangChain's default behavior did not match their requirements, working around the abstractions was sometimes harder than writing raw code. The API also changed frequently in early versions, breaking tutorials and examples.
The latest versions (LangChain v0.2+ and the new LangGraph for agent workflows) have significantly simplified the API and added better escape hatches in response to this feedback. The framework is maturing, and the abstractions are becoming more useful and less constraining.
LlamaIndex: The Data Framework
While LangChain tried to be everything for everyone, LlamaIndex focused specifically on the data side of GenAI: connecting LLMs to your data sources. Its core abstraction is the index, a structured representation of your documents optimized for LLM queries.
LlamaIndex excels at complex document structures. Real-world data is not just plain text paragraphs. It includes tables, hierarchical documents with nested sections, code with comments, multi-modal content mixing text and images, and structured data in databases. LlamaIndex has built-in support for all of these.
Key differentiators that set LlamaIndex apart:
Multi-modal indexing: handle text, images, and tables within the same index. A financial report with charts and tables can be indexed and queried holistically.
Knowledge graphs: represent relationships between entities, not just text similarity. 'Who reports to the VP of Engineering?' requires understanding org structure, not just text matching.
SQL query generation: let the LLM write SQL queries against your structured data. Ask 'What were our top 5 products last quarter?' and LlamaIndex generates and executes the SQL.
Recursive retrieval: for hierarchical documents, retrieve at the section level, then drill into subsections for detail. This handles long documents much better than flat chunking.
Advanced index types: tree indices for summarization, keyword indices for exact matching, and composable indices that combine multiple strategies.
When to Use What
This is the question every GenAI developer asks. Here is a practical guide:
LangChain: when you need agents with tool use, complex multi-step chains, or integrations with many different services. Best for applications where the orchestration logic is the hard part.
LlamaIndex: when your focus is data ingestion, indexing, and retrieval. Best for applications where the data pipeline is complex (multi-modal, hierarchical, structured data).
Both together: use LlamaIndex for data handling and retrieval, LangChain for orchestration and agent behavior. They integrate well and many production systems use both.
Neither: for simple RAG applications, direct API calls to OpenAI or Anthropic plus a vector database might be simpler and give you more control. Not every project needs a framework.
A useful heuristic: if you can describe your system as 'retrieve documents, generate answer,' LlamaIndex is probably sufficient. If you need 'retrieve documents, check a database, call an API, decide whether to escalate, then generate answer,' LangChain's orchestration capabilities become valuable.
The Bigger Picture: Framework vs. Raw Code
The framework debate in GenAI mirrors the same debate in web development. Some teams swear by Rails or Django. Others prefer minimal frameworks or raw code. The trade-off is always the same: frameworks accelerate prototyping and enforce patterns, but they add abstraction overhead and constrain your design space.
The recommendation: use frameworks to learn the patterns. Understand what a chain is, what an agent loop looks like, how memory works, and what retrieval patterns exist. Then decide whether the framework's abstractions are helping or hindering your specific use case. Many teams prototype with LangChain, then replace it with custom code for production once they understand what they need.
Frameworks accelerate prototyping but can slow down production. Use them to learn the patterns, then decide whether the abstraction overhead is worth it for your specific use case. The patterns matter more than the framework.
Try It in the Lab
The GenAI Pipeline Lab's visual builder teaches the same component patterns that LangChain and LlamaIndex implement in code. When you drag a Chunker into EmbeddingModel into VectorDB chain, you are building what LlamaIndex calls an index. When you add a Router with tool-calling, you are building what LangChain calls an agent. The lab gives you the mental model; the frameworks give you the code.
Further Reading
LangChain documentation (python.langchain.com) for comprehensive guides and tutorials
LlamaIndex documentation (docs.llamaindex.ai) for the data framework approach
LangGraph documentation for stateful agent workflows
Harrison Chase's talk at AI Engineer Summit on LangChain design decisions
Jerry Liu's blog posts on RAG patterns and LlamaIndex architecture