Author(s): Utk Umang

Originally published on Towards AI.

Your Dictionary to Everything AI Agents

You don’t need a technical background to read this. Every section is self-contained, so skim freely and go deep only where it interests you. A new AI tool or framework launches every other day, but most of the fundamentals boil down to the concepts covered here. This is the reference I wish existed when I started building agentic systems.

1. Prompting: The Foundation

Every production AI system starts with one thing: a well-written prompt. If you’ve used ChatGPT or Claude, you already know what a prompt is. You type something, the model responds. Simple enough.

But there’s a massive gap between “hey, summarise this for me” and a prompt that works reliably thousands of times without a human babysitting it. Production prompts are engineered, not typed.

The Prompt Structure Framework

A well-structured prompt has five components, and the more precisely you define each one, the more predictable your output becomes.

Types of Prompting

Now, even with a perfectly structured prompt, how you use it matters. There are three main prompting strategies, and each trades off between simplicity and accuracy.

Zero-shot is the simplest. You give the AI a task with no examples and expect it to figure it out. “Translate this sentence to French: The meeting is at 3pm.” Works well when the task is well-defined and the model already knows the pattern.

Few-shot is the next step up. You provide a few examples of input-output pairs so the model understands the exact pattern or format you want. Instead of describing your requirements in words, you show it: “Here are three examples of how I want emails summarised. Now summarise this one.” This is surprisingly effective for getting consistent formatting and tone.

Chain-of-thought is the heavy hitter. Instead of asking for a direct answer, you ask the AI to reason through the problem step by step before concluding. This is what powers “reasoning models” like OpenAI’s o1 or Claude’s extended thinking mode. It trades speed for accuracy, and it’s the go-to for complex analytical tasks where a snap answer would miss nuance.

2. From Casual Use to Production Systems

If you’ve spent any time with ChatGPT or Claude, you’ve probably developed a rhythm: send a message, get an answer you don’t love, tweak your ask, try again, repeat three or four times until the output clicks. That works when you’re the human in the loop, manually steering the conversation.

But in a production system, software needs to do this reliably, automatically, thousands of times a day. There’s no human sitting there hitting “regenerate” or “sending new instructions in a new message”. You can’t afford retries.

The solution is deceptively simple: stop asking one model to do everything in one shot. Instead, figure out the manual workflow first , what steps would a human take to solve this task? Then break those steps into smaller, discrete pieces, and assign each piece to a focused AI agent.

This is the core idea behind agentic AI systems. Instead of one model doing everything and hoping for the best, you decompose the problem into focused steps, each handled by a smaller, cheaper, more reliable agent. The compound effect is a system that’s faster, more predictable, and significantly easier to debug when something goes wrong.

3. Sub-Agents and Model Parameters

Now that you know the “why” behind breaking things down, let’s look at the building blocks.

A sub-agent is an AI model assigned to one specific, narrow task within a larger workflow. One agent extracts data from a PDF invoice. Another validates that data against a database. A third formats and sends a confirmation email. Because each sub-agent does a focused job, you can use smaller, faster, and cheaper models instead of one heavy model doing everything.

But assigning the right model isn’t enough. You also need to tune how the model behaves. The most important dial here is temperature.

Temperature controls how “creative” or “random” a model’s output is. Think of it as a spectrum. At the low end (close to 0), the model plays it safe, it picks the most predictable response every time. Ask the same question twice and you get the same answer. This is what you want for deterministic tasks like extracting data from a document or classifying a support ticket.

At the high end (close to 1), the model takes more risks and explores a wider range of possibilities. The output will vary each time you run it. This is useful for creative tasks like brainstorming, writing, or generating ideas.

The rule of thumb is simple: if the task needs consistency, go low. If the task needs creativity, go higher.

4. Agentic Workflow Paradigms

You have sub-agents. Now you need a way to connect them. There are two main paradigms, and understanding the difference between them is one of the most important architectural decisions you’ll make.

The first is chain-based workflows. This is the simplest pattern: the output of Agent 1 feeds into Agent 2, which feeds into Agent 3, and so on. Linear, predictable, easy to debug. LangChain is the most popular framework for building these. Its key benefit is abstraction → it doesn’t care whether you’re using Claude, GPT-4, or any other model under the hood. Switching providers requires minimal code changes. It also ships with ready-made components for common tasks like connecting to databases, handling memory, and formatting outputs, so you write far less boilerplate.

The second is orchestration-based workflows. This is where things get powerful. Instead of a fixed linear chain, a single Orchestrator Agent sits at the top of the system. You tell it what sub-agents are available and what each one does. When a task comes in, the orchestrator reads it, figures out a plan, and decides which sub-agents to call, in what order, and what to do with their outputs.

The critical difference: orchestration can be cyclic. The orchestrator can call Agent A, send its output to Agent B, get a result back, decide it needs to call Agent A again with new information, and loop until a condition is met. LangGraph is the framework for this. It extends LangChain, and the distinction is exactly this: LangChain is for linear chains, LangGraph is for graph-based workflows that can branch, loop, and route dynamically.

5. Agentic Patterns

Beyond how agents are wired together, there are established patterns for how an individual agent reasons and acts when given a task. Two of the most important ones are ReAct and Plan and Execute.

ReAct (Reasoning and Acting) is a loop. When given a task, the agent doesn’t immediately produce an answer. Instead, it cycles through three steps: Reason (what do I know, what do I still need?), Act (call a tool, fetch data), and Observe (is this enough to answer?). If the answer is no, it loops back to Reason and tries again.

This pattern is powerful because the agent is adaptive. It doesn’t commit to a fixed plan upfront. It responds to what it actually finds at each step, which makes it well suited for tasks where the path to the answer isn’t known in advance.

Plan and Execute takes the opposite approach. Instead of reasoning one step at a time, the agent first builds a complete plan before doing anything. A Planner Agent generates the full step-by-step breakdown, and then an Executor works through that plan sequentially. The advantage is predictability and efficiency, you know the full plan upfront, which makes it easier to parallelise steps, estimate cost, and debug failures. The trade-off is rigidity: if something unexpected comes up mid-execution, the plan may need to be revised.

The decision between the two comes down to the nature of the task. Use ReAct when the task is exploratory or unpredictable and the agent needs to adapt based on what it finds. Use Plan and Execute when the task is well-defined and you want efficiency, parallelism, and a clear audit trail of what was supposed to happen.

6. Context Engineering

Your AI agent can only make good decisions if it has the right information. Context engineering is the discipline of figuring out what information to inject into each prompt, and doing it efficiently.

The naive approach is to dump all user data into every prompt. The problem: prompts get huge, slow, and expensive. AI models charge by the token (roughly by word count), so sending a 50-page document when you only need two paragraphs from it is burning money for no reason.

The smart approach is to dynamically fetch only what’s relevant, right before sending the prompt. There are two main techniques depending on where your data lives.

If the relevant data lives in a structured database (rows and columns), you use tool calling to run a SQL query and pull only the relevant rows. A user asks “what’s my order status?” → the system queries the orders database for that specific user’s recent orders, injects just those rows into the prompt, and the agent answers accurately.

If the relevant data lives in unstructured form (documents, PDFs, notes, emails), you can’t just run a SQL query. This is where RAG (Retrieval-Augmented Generation) comes in. You build a pipeline that breaks all your documents into small chunks, converts those chunks into numerical vectors (a way of representing meaning mathematically), and when a query comes in, finds the chunks that are closest in meaning to the query. The AI sees only the most relevant pieces of your knowledge base, not everything. Check my others post for a detailed guide into RAG.

The key insight here is that context engineering is about precision, not volume. The less irrelevant noise you put in the prompt, the better the agent performs.

7. Harness Engineering

If context engineering is about what information the agent receives, harness engineering is about what capabilities and behaviours the agent is equipped with. Think of it like fishing, the right bait gets you the right output.

The most common tool in harness engineering is Skills. A skill is a markdown file (a simple text file) that describes how the agent should behave in a specific situation. It’s not a prompt for a user task, it’s a behaviour guide embedded in the agent’s system.

For example, an email reply agent might have a file called email-reply-skill.md that specifies: always start with the customer's name, never promise refunds without checking the policy tool, keep replies under 150 words, and match the tone of the incoming email.

The agent reads this skill file as part of its setup and follows these rules every time it writes an email. Skills make agents more predictable and significantly easier to update → you change a behavior by editing a markdown file, not by rewriting the entire prompt.

Together, these two layers cover the full picture. Context engineering ensures your agent has the right information. Harness engineering ensures it does the right thing with that information.

8. RAG vs Fine-Tuning

These two come up in almost every conversation about making an AI model better at a specific use case, and they’re often confused. They solve fundamentally different problems.

RAG changes the information the model sees. You’re giving a smart person the right reference book before an exam. The model itself doesn’t change, you’re just making sure it has access to the right data at the right time. It’s relatively cheap, you can update the data anytime, and it’s the first thing you should try.

Fine-tuning changes the model’s internal weights, how it thinks. You’re sending that person to school for a year. The model learns to behave differently: a specific tone, a specific format, a specific reasoning style. It’s expensive, requires training compute and labeled data, and the changes are baked into the model.

The rule of thumb: try RAG first. It’s faster, cheaper, and easier to update. Only fine-tune when you’ve confirmed that the model’s behavior or thinking pattern is the bottleneck, not the information it has. If your agent keeps getting the facts wrong, that’s a context problem → use RAG. If your agent gets the facts right but writes in the wrong tone or follows the wrong reasoning pattern, that’s a behavior problem → consider fine-tuning.

9. Tool Calling and MCP

Your AI agent knows a lot from its training data, and you can give it relevant context via the techniques we just covered. But sometimes it needs real-time, external data that neither its training nor your database has.

Say you want an agent to check a creator’s Instagram follower count before making a recommendation. That number changes daily and lives outside your system. This is where tools come in.

You define a tool by giving the agent a simple spec: a name, a description of what it does, what inputs it expects, and what it returns. The agent doesn’t need to know the implementation details, it just needs to know the tool exists and what it’s for. When the agent decides it needs follower data mid-task, it generates the right API call, gets the data back, and uses it in its reasoning.

Tools are essentially a catalog of external capabilities you hand to the agent. The agent decides when and which ones to use.

Now, defining tools manually for every external service gets tedious fast. MCP (Model Context Protocol) is the solution. MCP is an open standard that lets external services publish their tools in a standardised format. Instead of you writing every tool definition, the service hosts an MCP server that your agent connects to. Once connected, the agent automatically discovers all available tools from that server and knows how to use them.

Think of it as a plugin store for AI agents. Companies like Apify, Slack, and Google have published MCP servers. Your agent connects to one endpoint and instantly gains access to dozens of tools without manual integration work.

10. Reliability in Production: The Judge Agent

Getting an agentic system to work once in a demo is easy. Getting it to work correctly thousands of times in production is the hard part. There are two sides to reliability: making sure the output is good, and making sure the system stays up.

The Judge Agent

A judge agent is a separate AI model whose only job is to review the output of your system and decide: does this actually answer the user’s question correctly? It reads both the original input and the final output, then produces a verdict.

There are two ways to deploy a judge. In a sequential setup, every output goes through the judge before reaching the user. If the judge rejects it, a Refiner Agent tries to correct the output, and the loop repeats until it passes. This adds latency but guarantees quality, use it when accuracy is critical.

In a parallel setup, the user gets their answer immediately. The judge reviews it in the background, and if something looks wrong, it fires an alert (typically a Slack message or a review queue entry). Zero added latency for users, while still catching errors for the team. This is the more common production pattern.

Two critical details: first, use a different model for your judge than the one powering your system. A model tends to rate its own outputs favourably. If your system runs on GPT-4, use Claude as the judge, and vice versa. Second, enable extended thinking mode on your judge, instead of a simple pass/fail, the judge reasons through why the output is good or bad, which makes verdicts more accurate and gives your team much more useful debugging information.

The LLM Gateway

Evaluating output quality is one side of reliability. The other side is simpler: is the system even running?

Every AI provider has rate limits. Hit the ceiling and your requests start failing. Providers also go down sometimes. If your entire system is hardwired to one model from one provider, their bad day becomes your bad day.

An LLM Gateway sits in front of all your model calls and acts as a traffic controller. It handles two things: rate limit management (if you’re hitting OpenAI’s limit, it routes overflow to a second API key or a different provider entirely) and failover (if one provider starts failing consistently, it reroutes to another). Your system keeps running. Your users see nothing.

Think of it as a load balancer, but specifically for AI model APIs. Popular open-source options like LiteLLM also give you a unified API, so your code doesn’t need to know which model it’s talking to at any given moment.

11. Memory in Agentic Systems

When a conversation is short and simple, memory isn’t a problem. But what happens when a conversation runs fifty messages deep? Or when an agentic workflow crashes halfway through and needs to resume? Memory management becomes a critical layer in production systems.

The brute force approach is what you’ve already seen in ChatGPT. Every time you send message five, the system quietly packages up your previous four messages and four AI responses and sends all of it to the model again. That’s how the AI “remembers” what was said earlier. It works, but it doesn’t scale. As conversations get longer, you’re sending more tokens on every request. Costs go up, speed goes down, and eventually you hit the model’s context limit.

The smarter approach is to add a Summariser Agent to the system. After every batch of messages (say every ten), the summariser reads the conversation so far and produces a compact memory object: the key facts, decisions, and context, stored as a structured summary. Going forward, instead of sending all fifty past messages, you send the compact summary plus the last few messages in full. The prompt stays lean while preserving the important context.

But memory isn’t just for conversations. In a multi-step agentic workflow, if your system has five sub-agents and crashes at step three, you don’t want to start over from step one. Checkpointing solves this. After each sub-agent completes its task, its output is saved to a persistent store. If the workflow fails mid-run, the retry picks up from the last successful checkpoint. This is both a reliability pattern and a cost-saving one, you don’t waste compute redoing work that already succeeded.

12. Observability: Knowing What Actually Happened

Building an agentic system is one thing. Understanding what it did on any given run is another. Observability is the practice of logging and inspecting every step of an agentic execution, and it’s the difference between “something went wrong” and “I know exactly what went wrong and why.”

The tool most commonly used for this is Langfuse. Every time your agentic system runs, Langfuse records a trace: a detailed log of every step, showing which agent was called, what prompt it received, what the model returned, how long each step took, and how many tokens were used.

This matters for two reasons. First, debugging. When your system produces a wrong output, you don’t have to guess. You open the trace for that run and walk through it step by step. Was the wrong context injected? Did a sub-agent get malformed input? Did the judge agent fire correctly? The trace tells you.

Second, evaluation. When you’re trying to understand whether your system is improving or regressing over time, traces give you the raw evidence. You can compare runs, measure latency across sub-agents, and identify which step is the bottleneck.

Think of observability as your system’s black box recorder. You hope you don’t need it, but when something goes wrong, it’s the first place you look.

13. Human in the Loop (HITL)

Not every decision should be made entirely by an AI agent. HITL is the design pattern of intentionally building moments into your workflow where a human reviews, approves, or redirects before the system proceeds.

The simplest form is a chat interface. After every AI response, the human can react: ask for a revision, correct something, or confirm before the next step begins. Simple, effective, but not always the best UX.

The smarter approach is purpose-built interfaces. Consider an AI system that generates a webpage. A naive HITL design would be: show the page, let the user describe what they don’t like in a chat box, regenerate the whole thing. That’s slow and frustrating. A better approach is a direct editing interface where the user can click on any element, change an image, tweak the copy, adjust the layout → without going back to the AI for minor fixes. The AI handles the heavy lifting, the human handles the fine-tuning through purpose-built controls.

HITL also serves as a safety valve. Remember the judge agent from earlier? If the judge is uncertain about an output, rather than sending it to the user or retrying automatically, it can route the output to a human reviewer with a note: “I’m not confident this email should be sent. Can you review before it goes out?” This is especially important for irreversible actions, sending emails, making payments, deleting records, posting publicly. Any action you can’t undo is a good candidate for a human checkpoint.

Good HITL design is really a UX problem: what’s the minimum friction way to let a human correct or guide the system at the right moments?

14. Structured Outputs

LLMs produce free-form text by default. That’s fine when the output is meant for a human to read. But in a production agentic system, the output of one agent is usually the input of the next. If Agent 2 expects a specific JSON format and Agent 1 returns something slightly different, the whole pipeline breaks.

Structured outputs solve this. Instead of letting the agent return whatever text it wants, you define an exact schema: a data structure that specifies which fields must be present and what type each field should be. For example, an invoice extraction agent might be required to return a JSON object with vendor_name (string), invoice_date (date), total_amount (number), and line_items (array). Every time the agent runs, its output is validated against this schema the moment it arrives, before it touches anything downstream.

This matters because without schema validation, a missing field might not cause an error until three steps later in the pipeline, at which point tracing it back to the original agent is painful. Catching it early makes errors obvious, fast to fix, and easy to log. Most LLM providers now support structured output or JSON mode natively, and frameworks like LangChain have built-in support for defining and enforcing output schemas.

15. Putting It All Together

Here’s how a production agentic system actually looks when all of these pieces combine. Every layer solves a specific problem: prompting ensures each agent gets clear instructions, sub-agents break the work into focused pieces, context engineering ensures agents have the right information, harness engineering ensures they behave consistently, tool calling and MCP connect them to the outside world, structured outputs keep the pipeline stable, judge agents guard quality, memory keeps context alive, observability lets you debug, and HITL keeps humans in control where it matters.

That’s the full picture. None of these layers exist in isolation, they’re designed to work together. Context engineering feeds the orchestrator the right information. The orchestrator, equipped with skills, routes work to focused sub-agents. Those agents use tools and MCP to reach the outside world. Structured outputs keep the data flowing cleanly between them. Judge agents catch errors before they reach the user. Memory keeps context alive across long conversations and multi-step workflows. Observability logs everything so you can debug and improve. And HITL keeps humans in the loop where the stakes are highest.

A new AI tool or framework launches every other day, but the fundamentals don’t change that fast. If you understand these concepts, you have the mental model to evaluate anything new that comes along.

If you found this useful, I write about my experiments with AI engineering on Substack. You can subscribe here: https://utkarshumang.substack.com/

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI