Author(s): Surya Maddula

Originally published on Towards AI.

How to move past one-shot retrieval and build agents that grade their own evidence, rewrite their own questions, and catch themselves before they lie. With a small experiment you can run in your terminal tonight.

My first RAG chatbot lied to me on day two.

The article argues that naïve (one-shot) RAG breaks because it retrieves rigidly once, can’t recover when the retrieval is wrong, and provides no internal steps to verify relevance or grounding—leading to fluent fabrication. It explains how “controllable” or “agentic” RAG changes the pipeline by adding structured control flow (state machines/graphs with decision points, bounded loops, and optional human-in-the-loop) so the system can grade retrieved documents and generated answers, rewrite queries, and fall back to web search when needed. It then contrasts chain-based DAG approaches with LangGraph-style cyclic graphs, reviews influential prior work (ReAct, Self-RAG, CRAG, Adaptive-RAG, FLARE, and retrieval-improvement techniques), and lays out an end-to-end LangGraph architecture with routers, graders, generators, and guardrails. Finally, it provides a practical terminal experiment comparing naïve vs controllable systems, discusses key tuning knobs (chunking, top-k, temperatures, and cost/latency), highlights common failure modes (loops, misfiring graders, lost-in-the-middle, and prompt injection via retrieved documents), and recommends measurement using the “RAG triad” (context relevance, groundedness, answer relevance) plus tooling like RAGAS before deciding whether building an agent is worth the added complexity.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI