Author(s): Mehul Ligade

Originally published on Towards AI.

Beyond Basic RAG (Part 3): Agentic RAG, CRAG, Self-RAG and GraphRAG Explained | M012 | Mehul Ligade

Part 3 of a 3-Part RAG Series

If you have reached this article, congratulations. You now understand more about Retrieval-Augmented Generation than most people who casually throw the term “RAG” around on social media.

In Part 1, we learned how documents become searchable knowledge through chunking, embeddings, vector databases, and similarity search.
In Part 2, we explored how modern retrieval systems actually find information using HNSW, IVF, BM25, Hybrid Search, Re-ranking, Metadata Filtering, and Query Expansion.

At this point, it feels like we have solved retrieval. But there is a problem. Even the best retrieval system can still retrieve the wrong information. And once that happens, the language model starts building answers on top of weak evidence.

This is where the next generation of RAG begins. This is where systems stop behaving like search pipelines and start behaving more like reasoning systems.

In this article, we will explore Agentic RAG, CRAG, Self-RAG, GraphRAG, evaluation metrics, and the future direction of retrieval systems.

The Hidden Weakness of Traditional RAG

Imagine a student asks:

Explain the latest improvements in Transformer architectures.

The system retrieves five chunks, sends them to the language model, and generates an answer. Everything seems fine. But what if those chunks are outdated? What if they only partially answer the question? What if the most useful document was never retrieved at all?

Traditional RAG makes one very important assumption: retrieval worked correctly.

That assumption is often wrong.

The language model does not know whether the retrieved information is complete, relevant, or trustworthy. It simply uses whatever evidence arrives. If retrieval succeeds, the answer looks brilliant. If retrieval fails, the answer may still sound convincing while being incomplete or misleading.

This limitation inspired many of the innovations we are about to discuss.

Agentic RAG: When Retrieval Becomes a Decision

Traditional RAG follows a fixed workflow. A question arrives, retrieval happens, context is attached, and the model generates an answer.

Agentic RAG breaks this rigid pipeline.

Instead of forcing the system to follow a predefined path, we allow it to make decisions during execution. The model can determine whether retrieval is needed, whether another search should be performed, whether an external tool should be called, or whether enough information already exists to answer confidently.

This might sound simple, but it fundamentally changes how the system behaves.

Think about how humans solve problems. If you cannot find an answer in one chapter of a textbook, you do not immediately stop searching. You try different keywords, look at another book, search online, or ask a follow-up question.

Agentic RAG introduces similar behavior into AI systems.

The model becomes an active participant in the retrieval process rather than a passive consumer of retrieved documents.

Why Agentic RAG Is Becoming So Popular

Consider a user asking:

Compare Graph Neural Networks and Transformers for recommendation systems.

A traditional RAG system might perform a single retrieval operation and hope that the retrieved chunks contain enough information.

An Agentic RAG system can behave differently.

It may first retrieve information about Graph Neural Networks. Then it may retrieve information about Transformers. Next, it may search specifically for recommendation-system research. Finally, it combines everything and generates a comparison.

Notice what happened -> The system did not simply retrieve information. It planned how to gather information.

That shift from retrieval to information gathering is one of the most important trends currently shaping modern AI applications.

CRAG: Corrective Retrieval-Augmented Generation

One of the biggest weaknesses in traditional RAG is the assumption that retrieval succeeded.

CRAG, which stands for Corrective Retrieval-Augmented Generation, directly challenges this assumption.

Instead of blindly trusting retrieved documents, CRAG introduces an evaluation stage. Before generation begins, the system attempts to determine whether the retrieved information is actually useful.

The workflow becomes:
Question → Retrieval → Retrieval Evaluation → Generation

If the retrieved documents appear relevant and complete, the system continues normally.

If retrieval quality appears poor, the system can perform another search, reformulate the query, retrieve additional information, or even search external sources.

Think of a student preparing for an exam. If the textbook page clearly answers the question, they continue reading. If the page seems unrelated, they search somewhere else. CRAG applies this same idea to AI systems.

Self-RAG: Teaching the Model to Critique Itself

CRAG evaluates retrieval quality.

Self-RAG goes one step further. It evaluates the generated answer itself.

This is one of the most fascinating ideas in modern retrieval research because it introduces a feedback loop between generation and retrieval.

In a traditional RAG system, retrieval happens once and the answer is generated once.

Self-RAG changes that process.
After generating an answer, the model asks itself important questions:

• Did I answer the question completely?
• Is my answer supported by evidence?
• Do I need more information?
• Should I retrieve again before responding?

If the answer appears weak or unsupported, the system can perform additional retrieval and generate a stronger response.

In other words, the model develops a form of self-reflection.

The goal is not simply generating answers quickly. The goal is generating answers that are supported by evidence and capable of surviving internal scrutiny.

Why Multiple Retrieval Attempts Matter

When humans learn something new, the first search is rarely enough.

Imagine you are learning Reinforcement Learning.

You might start by searching:

What is Reinforcement Learning?
Then you discover Markov Decision Processes.
Then Q-Learning.
Then Policy Gradients.
Then PPO.

Each search builds on the previous one. Learning is iterative.

Modern retrieval systems are increasingly adopting the same principle. Instead of relying on a single retrieval step, they perform multiple retrieval cycles, gradually building a richer understanding of the problem.

This is one of the reasons Agentic RAG often outperforms traditional retrieval pipelines on complex questions.

The Lost-in-the-Middle Problem

Many beginners assume that more retrieved information automatically leads to better answers.

Unfortunately, that is not always true.

Large language models often struggle when important information appears in the middle of a long context window. Information placed near the beginning or end of the context tends to receive more attention, while important details buried in the middle may be overlooked.

This issue is commonly known as the Lost-in-the-Middle problem.

Imagine retrieving ten excellent chunks and placing them into a prompt. If the most important chunk ends up buried in the middle, the model may not use it effectively.

This is one reason retrieval quality alone is not enough. Context organization, Chunk ordering, Re-ranking matters.

The challenge is not only finding relevant information but also presenting it in a way that the model can use effectively.

GraphRAG: Beyond Similarity Search

Until now, most retrieval systems we have discussed relied heavily on similarity.

GraphRAG introduces a completely different perspective.

Instead of storing information as independent chunks connected only through embeddings, GraphRAG stores relationships between entities.

Imagine a knowledge graph like this:

Andrew Ng → teaches → Machine Learning
Machine Learning → contains → Neural Networks
Neural Networks → use → Backpropagation
Backpropagation → depends on → Gradient Descent

Traditional vector search retrieves information because two pieces of text are semantically similar.

GraphRAG retrieves information because concepts are connected through relationships.

This becomes especially powerful when answering questions that require reasoning across multiple concepts.

For example:

How does Gradient Descent influence Backpropagation?

A knowledge graph already understands the relationship between those concepts.

Instead of searching purely through similarity, the system can traverse connections and build a richer explanation.

That ability makes GraphRAG particularly attractive for enterprise knowledge systems, research assistants, and scientific applications where relationships matter as much as content.

Traditional RAG vs GraphRAG

A useful way to think about the difference is through a library analogy.

Traditional RAG finds books that look similar to your query.

GraphRAG finds books and understands how the topics inside those books relate to one another.

One focuses on similarity.
The other focuses on relationships.
The future will likely combine both approaches.

Vector search is excellent for finding relevant information quickly. Graph structures are excellent for understanding how information connects. Together, they can create systems that are both knowledgeable and contextually aware.

How Do We Know a RAG System Is Actually Good?

One of the biggest mistakes beginners make is assuming that a working RAG system is automatically a good RAG system.

Building a pipeline is relatively straightforward. Evaluating it is much harder.

A production-grade RAG system needs answers to important questions:

• Are we retrieving relevant information?
• Are we missing important information?
• Is the answer grounded in evidence?
• Is the model hallucinating?
• Does the answer actually help the user?

Without evaluation, improving a system becomes guesswork. Good engineering requires measurement.

Important Evaluation Metrics

Several metrics are commonly used when evaluating retrieval systems.

Context Precision measures how much of the retrieved information is actually relevant.

Context Recall measures whether retrieval successfully found all the information needed to answer the question.

Faithfulness measures whether the generated answer is supported by retrieved evidence.

Answer Relevance measures whether the response actually addresses the user’s question.

Together, these metrics provide a more complete picture of system performance. Evaluation helps identify where those failures occur.

What I Learned the Hard Way

When I first started exploring RAG, I believed retrieval was the entire challenge. Later, I realized retrieval is only one piece of a much larger puzzle. A system can retrieve perfectly and still generate weak answers. A system can retrieve partially and still succeed. What matters is whether the right information reaches the model at the right moment and in the right format.

The strongest RAG systems are not necessarily the ones with the most components. They are the ones where retrieval, ranking, reasoning, evaluation, and generation work together as a single system.

The Future of RAG

The future is clearly moving beyond simple retrieve-and-generate pipelines.

We are seeing systems that plan before retrieving, evaluate retrieval quality, critique their own responses, search multiple times, use tools dynamically, build knowledge graphs, and decide when retrieval is necessary.

In short, retrieval systems are gradually becoming agents. And that evolution is only beginning.

The next generation of AI assistants will not simply retrieve information. They will reason about information.

Final Thoughts

Three articles ago, we started with a simple question:

How do we give AI better memory without retraining it?

The answer turned out to be much deeper than vector databases and similarity search.

We learned how knowledge is transformed into chunks and embeddings. We explored how retrieval systems search through millions of vectors using HNSW, IVF, Hybrid Search, and Re-ranking. Finally, we discovered how modern systems are evolving into Agentic RAG, CRAG, Self-RAG, and GraphRAG.

The journey from basic retrieval to intelligent retrieval mirrors the broader journey of AI itself.

We are moving from systems that merely find information to systems that reason about information.

And that is where things start becoming truly interesting.

Complete RAG Roadmap

Part 1: RAG Explained From Scratch: How PDFs Become Searchable Knowledge for AI

Part 2: How RAG Actually Finds Answers: HNSW, IVF, BM25, Hybrid Search and Re-Ranking

Part 3: Beyond Basic RAG: Agentic RAG, CRAG, Self-RAG and GraphRAG Explained

Mehul Ligade

X: @MehulLigade
LinkedIn: linkedin.com/in/mehulcode12

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI