Author(s): Katakamvivek

Originally published on Towards AI.

How to use RAG with structured data — a hands-on POC that converts plain English into safe, verified SQL, with real examples of what breaks and how to fix it.

Almost every RAG example you see online uses unstructured data. PDFs, wikis, support articles — chunk them, embed them, retrieve them, done. But here’s the thing: most of the data a business actually cares about doesn’t live in documents. It lives in relational databases — customers, orders, invoices, products.

The article walks through a hands-on proof of concept for applying RAG to structured data via a text-to-SQL pipeline that is designed to be safe and correct. After introducing the approach, it explains how to build a knowledge base by creating a table ontology, listing and verifying what users will ask (as a trusted FAQ layer with prewritten SQL templates), and defining policies/guardrails that restrict tables, row limits, and retries. It then details the end-to-end flow—routing to templates, prescreening for scope and clarity, retrieving relevant ontology slices with embeddings, generating SQL with grounded context, validating deterministically (single clean SELECT, ontology checks, restricted-category blocks, and bounded repair), executing on a read-only database, and finally using a second independent LLM call to verify that the SQL truly answers the question before making one of several decisions (answer, answer with caveat, clarify, refuse). The author demonstrates real examples where the system succeeds and where it fails without guardrails, including cases the syntax checker can’t catch, and provides evaluation numbers showing large accuracy gains. The post closes with practical guidance for builders—prioritizing retrieval/semantic quality, using independent verification, implementing feasible guardrails in code, improving question phrasing to reduce ambiguity, and keeping transparency by showing generated SQL—plus thoughts on when to build versus buy.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI