A legal-tech RAG chatbot answers "what's our standard termination clause" perfectly, pulling the exact chunk from a well-indexed contract template. Ask it "which of our Q3 vendor contracts have auto-renewal clauses that conflict with our new data-residency policy," and it confidently returns one contract that doesn't actually conflict, missing two that do. The first question was single-hop: one query, one relevant chunk, one correct answer. The second needed the system to find the auto-renewal contracts, then cross-reference each against a separate policy document, then reason about conflict, three dependent steps a pipeline that retrieves once and generates once structurally cannot do. That gap between single-hop and multi-hop is exactly what agentic RAG was built to close, and there isn't one fix, there are four distinct patterns, each solving a different failure mode.
Why does naive RAG fail on anything but single-hop questions?
Because it retrieves once, generates once, and never checks itself. Fixed top-k vector search embeds the query, pulls the k nearest chunks by cosine similarity (the full mechanism is in why your RAG chatbot gives wrong answers), and hands them to the LLM regardless of whether they're actually relevant, sufficient, or need a second lookup to resolve something the first chunk only referenced. A multi-hop question needs retrieval to happen more than once, each round informed by what the last one returned, which a single fixed pass cannot structurally provide. We've written separately about why static RAG breaks multi-step agents; agentic RAG is the architectural answer to that breakage.
What makes a RAG pipeline "agentic" in the first place?
An autonomous agent, not a fixed pipeline, controls retrieval: it plans sub-queries, routes across data sources, iterates on weak results, and checks its own context before the LLM generates a final answer. In practice, retrieval stops being a hardcoded function call that runs exactly once per turn and becomes a decision the model can make, repeat, or skip entirely.
Pattern 1: Router — when is the problem which source to use, not retrieval quality?
Use a router when your failures come from querying the wrong index, not from bad matches inside the right one. A router agent inspects the incoming query before any retrieval happens and decides which source or tool should handle it, a pricing question goes to the product database, a policy question goes to the compliance vector store, a general question skips retrieval entirely. It's the cheapest agentic pattern to add because it doesn't touch how retrieval works inside any single source, it just stops you from asking the wrong one.
Pattern 2: Self-RAG — when should the model grade its own retrieval?
Use Self-RAG when you need the model to decide, as it writes, whether retrieval was even necessary and whether what it got back actually supports the claim it's making. Self-RAG trains the model to emit four reflection tokens inline with its output: Retrieve decides whether to call the retriever at all, IsRel scores whether a retrieved passage is actually relevant to the query, IsSup checks whether the generated claim is fully, partially, or not supported by that passage, and IsUse scores overall response quality. Because the critique happens in the same forward pass as generation, unsupported claims get flagged before they ship, not after a human catches them downstream.
The tradeoff is that this needs a model actually fine-tuned to produce those tokens, the original research trains a 7B critic model to over 90% agreement with GPT-4 judgments, so it isn't a drop-in wrapper you bolt onto an existing production pipeline the way the next two patterns are.
Pattern 3: Corrective RAG — how do you recover when retrieval pulled the wrong chunk?
Use Corrective RAG (CRAG) when your corpus is incomplete or goes stale faster than you can re-index it. CRAG adds a lightweight retrieval evaluator that grades every retrieved document as Correct, Incorrect, or Ambiguous, and that grade decides what happens next: Correct lets generation proceed normally, Incorrect discards the chunk and triggers a web search or alternate source instead of letting the LLM generate from garbage, and Ambiguous blends both the retrieved and externally sourced content. The evaluator itself can be a small classifier rather than another LLM call, which keeps the added latency low.
This is the pattern for the freshness problem specifically, see why RAG chatbots answer with outdated information for the drift CRAG is built to catch before it reaches the user.
Pattern 4: Adaptive RAG — how do you avoid paying multi-hop cost on a simple question?
Use Adaptive RAG when most of your traffic is simple and only a minority of queries genuinely need multi-hop reasoning. Adaptive-RAG trains a classifier to route each incoming query down one of three paths: no retrieval at all for questions the model can answer from its own parametric knowledge, single-step retrieval for moderate questions, and multi-step, multi-hop retrieval only for genuinely complex ones. Running every query through a full multi-hop pipeline just in case is the single biggest unnecessary cost driver in a production RAG system, and this pattern caps that cost exactly where complexity actually requires it.
Which pattern fits which failure mode?
Match the pattern to the specific symptom, not to whichever one is trending:
- **Wrong source entirely** → Router.
- **The model can't tell when it hallucinated versus when it retrieved well, and you can fine-tune** → Self-RAG.
- **Retrieval returns stale or irrelevant chunks and you can't re-index fast enough** → Corrective RAG.
- **Most queries are simple but a minority need real multi-hop reasoning** → Adaptive RAG.
- **The answer depends on relationships between entities across many documents, not one chunk** → GraphRAG, covered separately in GraphRAG vs. vector RAG for multi-hop reasoning.
How AIBOOTSTRAPPER helps
We built ComplyNexus (see case studies) around exactly this kind of self-checking retrieval loop. The RAG engine doesn't just pull the nearest regulatory clause and generate a summary, it continuously monitors regulatory sources, interprets new rules against the client's existing control library, and flags gaps rather than confidently answering from a stale or irrelevant match, the Corrective-RAG instinct applied to a compliance corpus that changes weekly. That approach cut manual review time by 92%, turned a 3-week regulatory turnaround into about 2 hours, and gave the client's own auditors 100% audit-ready traceability.
If your RAG system is confidently wrong on anything but the simplest questions, book a call and we'll diagnose which pattern actually fixes it, or see our AI product development services.
Want this done for you?
Book a free strategy call and we'll show you how to build and market your business with AI.