Author(s): Sandeep Chaudhary

Originally published on Towards AI.

Agentic AI is not a Feature. It is a New System Design Paradigm.

Introduction: From Reliability to Reasoning

Distributed systems taught us how to build software that scales, recovers, and performs. Agentic systems demand something deeper — software that reasons, learns, and earns trust.

The shift isn’t just technical; it’s philosophical. Where traditional architecture optimizes for uptime, agentic architecture optimizes for judgment. Each principle below reinterprets a familiar pattern through the lens of autonomy and trust.

System design used to be about drawing neat boxes and arrows. Today, those boxes are expected to think, adapt, and earn trust. That shift — from static systems to agentic AI — is rewriting the rules of architecture.

Finance teams don’t just want automation. They want systems they can rely on, especially when the stakes involve reconciliation and compliance. Trust is no longer a nice‑to‑have; it’s the foundation.

Five years ago, I designed a fraud detection system for a financial services client. It was textbook distributed systems design: microservices architecture, event‑driven flows, stateless services behind an API gateway. Circuit breakers, retry logic, blue‑green deployments, structured logging on every request and response. It worked beautifully.

Last year, I was asked to extend it with an agentic AI layer — an agent that would investigate flagged transactions, gather additional context, assess risk, and recommend a disposition. I started with the same design principles I had always trusted. Within six weeks, I hit four architectural walls those principles had never prepared me for.

The Walls I Hit

• The agent needed to remember context across multiple investigation steps — stateless design broke down.
• Its decisions needed to log reasoning chains, not just inputs and outputs — structured logging was insufficient.
• Deploying a new agent version was not the same as deploying a new service version — blue‑green had no equivalent.
• Defining what the agent was allowed to do autonomously was a different problem from defining an API contract.

Agentic AI does not invalidate distributed systems design principles. It transforms most of them. The architects who adapt fastest are the ones who can map precisely what changes, what stays the same, and what is genuinely new — rather than assuming their existing mental models transfer unchanged.

The Core Shift: From Scripted to Goal-Directed

The Agentic Architecture Manifesto — Designing Systems That Deserve Autonomy

Agentic AI doesn’t invalidate distributed systems principles. It transforms them. The architects who adapt fastest are the ones who can map precisely what changes, what stays the same, and what is genuinely new — rather than assuming their mental models transfer unchanged.

Every distributed systems principle rests on an unspoken assumption: the system does what it is programmed to do. A microservice receives a request, executes a defined function, and returns a response. The behaviour is deterministic. The execution path is authored by a human and encoded in service logic. The system has no goals — only instructions.

An agentic AI system breaks this assumption. The agent receives a goal and decides how to achieve it — which tools to invoke, in what order, with what inputs — at runtime. The execution path is not authored. It is reasoned. And that shift — from scripted execution to goal‑directed reasoning — cascades through every design decision you make.

What follows are listed system design principles that change when you make that shift, and two that do not. For each one:

• What the principle looks like in traditional distributed systems design
• How it breaks or transforms in an agentic context
• What the new version requires you to build

Principle 1: API Contract → Authority Limit

Traditional View In distributed systems, the API contract is the boundary agreement between services. It defines what the service does: which operations are available, what inputs are accepted, and what outputs are produced. The caller decides what to invoke, and the service executes accordingly. Governance is enforced by schema validation — if your input doesn’t match the contract, the call is rejected before execution begins.

Agentic Shift With agentic AI, the boundary changes. The agent decides what to call at runtime, choosing tools and sequencing them dynamically. The schema question — what inputs a tool accepts — still matters, but it’s no longer the primary governance concern. The real question becomes: what is this agent permitted to do autonomously, and what requires human authorization?

Example Take a fraud investigation agent. It may have access to a customer profile tool, a transaction history tool, a case creation tool, and a block‑account tool. Clearly, not all of these should carry equal authority. Reading a customer profile is low‑risk. Blocking an account is consequential and irreversible. The authority limit defines which tool calls can execute autonomously and which must be intercepted by a constraint enforcement layer that requires human approval before execution.

Authority limits must be enforced in the architecture — in a constraint service that intercepts tool calls before execution — not in the agent’s instructions. An instruction that says ‘do not block accounts without approval’ is a guideline. A constraint service that intercepts every block-account tool call and routes it to an approval workflow is a control. Regulated systems need controls, not guidelines.

Principle 2: Request/Response Logging → Decision Audit Trail

Traditional View In distributed systems, structured logging captures what the service received and what it returned. It’s operational by design — recording the request payload, response, latency, status code, and correlation ID that links the call to its upstream context. These logs answer one question: what happened to this request?

Agentic Shift In an agentic AI system, that question isn’t enough. When an agent recommends a fraud disposition, completes a KYC verification, or adjusts a price, regulators and customers don’t just ask what happened — they ask why. What information did the agent use? What did it consider? What did it decide not to do?

The audit trail must therefore capture the full reasoning chain:

• The goal the agent was given
• Each reasoning step and tool invoked
• The outputs and intermediate conclusions
• The confidence score at decision time
• The final recommendation and the factors behind it

This isn’t an enhanced operational log; it’s a governance record. It must be stored immutably — for example, in Azure Cosmos DB with append‑only access policies — separate from operational logs that can be rotated or deleted.

In BFSI, this isn’t optional. India’s DPDP Act 2023 requires organisations to explain how automated decisions affecting individuals were made. The RBI FREE‑AI framework demands auditable AI systems. The audit trail is what turns compliance from aspiration into accountability.

Principle 3: Blue‑Green Deployment → Shadow Mode

Traditional View Blue‑green deployment is the classic pattern for zero‑downtime updates. Two environments run in parallel — blue serving live traffic, green running the new version. Traffic shifts gradually while both are monitored. If the new version underperforms, traffic shifts back. The evaluation criteria are clear: error rates, latency, schema correctness — all observable in minutes.

Agentic Shift Deploying a new agent version isn’t about uptime; it’s about trust. The question isn’t does it run without errors? — it’s can it be trusted to make good decisions autonomously? And trust can’t be measured in five‑minute latency graphs.

Shadow Mode is the agentic equivalent of blue‑green, but it operates on a different timescale. The new agent runs in parallel with the existing process, generating recommendations while humans make the actual decisions. Every divergence between agent and human judgment is logged and analysed — not for performance, but for alignment.

Over days or weeks, the data reveals where the agent agrees with experienced reviewers, where it diverges, and why. Sometimes divergence means the agent has learned something new; sometimes it means it missed something critical. That analysis is what earns the agent the right to act autonomously.

Example: KYC Context Before a new agent version is trusted to auto‑approve low‑risk identity verifications, it runs in shadow mode alongside manual review. After two weeks, the divergence analysis shows:

• 96 % agreement with human reviewers in standard cases
• 3 % divergence where humans later agreed the agent was right
• 1 % divergence on edge‑case document formats with low confidence

That evidence base justifies expanding the agent’s autonomous authority to standard low‑risk cases. The 1 % edge cases remain in human review.

Principle 4: Circuit Breaker → Confidence Threshold Fallback

Traditional View The circuit‑breaker pattern protects distributed systems from cascading failures. When a downstream service fails repeatedly, the circuit opens — subsequent calls return a cached or default response, giving the failing service time to recover. The logic is binary: the service either responds or it doesn’t. The fallback is defined at design time.

Agentic Shift Agents don’t fail in binary terms. They always return something — the question is how much to trust it. A fraud‑detection agent that flags a transaction with 0.94 confidence isn’t the same as one that flags it with 0.61 confidence, even though both respond without error.

Confidence‑Threshold Fallback is the agentic equivalent of the circuit breaker. When the agent’s confidence falls below a defined threshold, the system doesn’t throw an error — it escalates to a human reviewer. The agent’s analysis, confidence score, and uncertainty factors are presented together, turning fallback into a designed human‑review path rather than a cached response.

Domain‑Specific Governance Thresholds are not hard‑coded constants; they’re configuration — version‑controlled and auditable.

• A credit‑decisioning agent might auto‑approve above 0.90, route to second‑line review between 0.75 and 0.90, and escalate below 0.75.
• A pricing agent might act autonomously above 0.85 for standard SKUs but always require human approval for regulated categories, regardless of confidence.

Confidence thresholds transform failure handling from reactive protection to proactive governance. They make uncertainty visible, measurable, and manageable — the foundation of trust in agentic systems.

Principle 5: Stateless Design → Stateful Reasoning

Traditional View Stateless design is the backbone of scalable distributed systems. Each request is independent; no instance holds session‑specific state. Any service instance can handle any request because the request carries all the context it needs. The service processes it, returns a response, and forgets what came before. This simplicity makes systems easy to scale and resilient to failure.

Agentic Shift Agents don’t just process requests — they reason across steps. A fraud investigation agent that gathers transaction history, checks related accounts, queries sanctions lists, reviews prior cases, and synthesises a risk assessment isn’t executing isolated calls. Each step depends on what the previous ones discovered. The agent’s working memory — what it has tried, learned, and decided to explore next — is the mechanism of its intelligence. Lose that context, and you lose the reasoning.

Stateful Reasoning Agentic systems require explicit state‑management design. The agent’s conversation history — goals, reasoning steps, tool calls, and outputs — must persist across the full task.

• For short tasks, this lives in the in‑memory context window of the Azure AI Agent Service thread.
• For long‑running tasks spanning hours or days — such as complex credit assessments or multi‑stage KYC investigations — the state must be persisted externally in Azure Cosmos DB or Azure Cache for Redis.

This design must handle agent restarts, context‑window limits, and concurrent state access. The scalability principle isn’t lost — it’s reinterpreted. Multiple agent instances can still run in parallel, as long as each retrieves its task’s state at the start of every step. The system remains stateless across instances but stateful within each task — a subtle but profound shift.

Principle 6: Single Responsibility → Agent Specialisation

Traditional View The single‑responsibility principle keeps microservices cohesive and independently deployable. Each service does one thing and does it well: a payment service processes payments, an account service manages accounts, a notification service sends messages. Boundaries are defined by the data each service owns and the operations it exposes. Services communicate through well‑defined interfaces, so changing one doesn’t require changing others.

Agentic Shift Specialisation still matters — but the boundary moves. In multi‑agent systems, the boundary isn’t about data ownership; it’s about capability. Each agent specialises in a reasoning domain rather than a dataset.

In a KYC multi‑agent system:

• A Document Intelligence Agent extracts and validates identity document fields.
• An Identity Verification Agent cross‑references extracted data against reference databases.
• A Risk Assessment Agent synthesises signals into a structured risk score.
• A Compliance Agent checks outputs against regulatory requirements and generates the audit record.

Each agent has a defined capability domain and a clear output contract.

Dynamic Orchestration Here’s the key difference: in microservices, Service A calls Service B through a fixed API. In multi‑agent systems, an orchestrator agent — or a planner implemented in Semantic Kernel — decides at runtime which specialist agents to invoke, in what order, and with what context.

The specialisation is static. The orchestration is dynamic. That combination — defined capabilities + adaptive orchestration — is what makes multi‑agent systems capable of handling tasks that vary widely in structure and complexity.

Principle 7: Monitoring → Behavioural Observability

Traditional View Infrastructure monitoring tells you whether your system is healthy. Metrics like CPU, memory, latency, error rate, and throughput are objective, measurable, and directly actionable. A spike in error rate triggers an alert; the on‑call engineer investigates logs, finds the failing service, and rolls back or fixes forward. These signals answer one question: is the system running within SLA?

Agentic Shift In agentic systems, infrastructure metrics still matter — latency, error rates, token costs are observable and actionable. But they don’t tell you whether the agent is behaving as intended. An agent can be fast, cheap, and error‑free while systematically making poor decisions.

What you need is behavioural observability:

• Tool selection patterns — Which tools is the agent invoking most often, and in what sequence? Unexpected patterns are early signals of reasoning drift.
• Confidence score distribution — Is the average confidence shifting over time? A drop from 0.88 to 0.71 signals model drift, even if outputs look correct.
• Divergence from expected paths — For tasks with known sequences (e.g., standard KYC verification), deviations reveal inputs outside training distribution.
• Human override clustering — Where reviewers most often override agent recommendations, exposing systematic reasoning gaps.
• Reasoning chain anomalies — Steps appearing in agent outputs that weren’t expected, often precursors to incidents.

On Azure Behavioural observability requires custom instrumentation:

• Structured events published to Azure Event Hub from agent tool invocations
• Real‑time pattern detection via Azure Stream Analytics
• Long‑term storage in Azure Data Lake for offline analysis
• Infrastructure metrics still tracked in Azure Monitor, but complemented by behavioural signals you design and build.

Principle 8: Idempotency → Reversible Actions

Traditional View Idempotency ensures that repeating an operation produces the same result. It’s what makes retry logic safe. If a payment service call fails due to a network timeout, the caller retries — and because the service is idempotent, the duplicate call doesn’t double‑charge the customer. Designing for idempotency means identifying which operations are naturally idempotent and which require idempotency keys to deduplicate.

Agentic Shift Agents don’t just compute; they act. They send communications, update records, trigger workflows, and make decisions that affect real customers. These actions exist in the world — and many are neither idempotent nor easily retried.

A fraud‑investigation agent that blocks a customer’s account can’t be made idempotent in the same way a payment check can. The question isn’t what happens if this runs twice? — it’s what happens if this turns out to be wrong?

Design for Reversibility Reversibility means defining, for every agent action category, a clear reversal path:

• A blocked account can be unblocked.
• A communication can be followed by a correction.
• A record update can be rolled back through an audit‑trail entry.

Reversibility also changes how you think about sequencing. In a fraud case, the agent should gather all evidence before taking any consequential action — not act first and analyse later. The sequence becomes: Read → Analyse → Conclude → Confirm (human approval if above threshold) → Act.

Consequential actions come last, not first, so that if the conclusion changes mid‑analysis, no irreversible step has already been taken.

Principle 9: Compliance Layer → Governance Anchoring

Traditional View Compliance is layered on top of systems — audit logs, access controls, periodic reviews. It’s external to the architecture.

Agentic Shift Governance must be native. Every agent action produces a governance artifact: decision trail, reversibility path, confidence threshold record. Compliance isn’t bolted on; it’s baked in.

Example In BFSI, DPDP Act and RBI FREE‑AI demand explainability. Governance anchoring ensures every agent decision is automatically auditable, not retrofitted.

Principle 10: Human Fallback → Human‑in‑the‑Loop Boundaries

Traditional View Humans intervene only when systems fail — the fallback engineer who rolls back or fixes forward.

Agentic Shift Humans intervene when confidence drops, when reversibility is required, or when ethical judgment is needed. They aren’t fallback engineers; they’re trust validators.

Example In credit scoring, agents auto‑approve high‑confidence cases, but borderline cases escalate to human reviewers. The boundary is confidence, not failure.

Principle 11: Throughput Optimization → Cost‑Aware Intelligence

Traditional View Infrastructure is optimized for throughput and latency. Efficiency means faster responses at lower compute cost.

Agentic Shift Optimization shifts to value per token. Tiered model selection, escalation paths, and caching of reasoning steps become part of the design.

Example A triage agent uses a small model for routine checks, escalating only complex cases to a larger model. This balances accuracy with inference cost — a CFO‑friendly design principle.

Principle 12: Redundancy → Resilience Through Diversity

Traditional View Resilience comes from redundancy — replicating identical services across nodes.

Agentic Shift Resilience comes from agent diversity. Multiple agents with different reasoning strategies cross‑validate outputs. Diversity reduces drift risk and strengthens defenses against adversarial inputs.

Example In fraud detection, one agent focuses on transaction patterns, another on identity anomalies, and a third on behavioural signals. Their combined perspectives create resilience that replication alone cannot.

Principles That Do Not Change

Not everything transforms. Two foundational distributed systems principles apply to agentic systems with minimal modification:

Loose coupling

Design agents so that each one depends on the interfaces of other agents and tools, not their internal implementations. A KYC orchestrator agent should not know how the Document Intelligence Agent extracts fields — it should know what it asks for and what it expects back. When the document extraction implementation changes, the orchestrator is unaffected. The principle is identical. The implementation uses agent tool definitions and output contracts rather than REST API schemas.

Observability-first design

The principle that you build observability in from the start — rather than adding it after something goes wrong — applies with even more force to agentic systems than to traditional distributed systems. The difference is that the observability surface is wider and more complex. Infrastructure metrics, tool invocation traces, confidence score distributions, reasoning chain logs, human override events — each of these needs to be instrumented from day one. Adding observability to an agentic system in production is significantly harder than adding it to a microservice, because the reasoning that needs to be captured is ephemeral unless you design its capture into the system from the beginning.

Security

Authentication, authorization, encryption, and secure communication channels.

In Agentic this is exactly the same. Agents may reason differently, but they still operate within secure boundaries. Zero‑trust, least privilege, and strong cryptography remain non‑negotiable.

What I Got Wrong

Four mistakes from extending the fraud investigation system with an agentic layer — each one a lesson in why distributed systems principles alone weren’t enough:

1. Instructions ≠ Constraints

• Mistake: I treated the agent’s system prompt as an authority limit.
• Failure: An edge case led the agent to interpret instructions narrowly, blocking an account without human approval.
• Lesson: Constraints must be enforced in code, not just in prompts. Authority boundaries are architectural, not textual.

1. Logs ≠ Audit Trails

• Mistake: I designed operational logging and called it an audit trail.
• Failure: Logs captured tool calls but not the agent’s reasoning chain. Compliance couldn’t answer “why” a data source was skipped.
• Lesson: Governance‑grade audit trails must capture reasoning steps, not just actions.

1. Aggregate Accuracy ≠ Trust Readiness

• Mistake: I moved from shadow mode to autonomy after two weeks based on high overall agreement with human reviewers.
• Failure: Rare but critical failure cases (cross‑border transfers with unusual currency pairs) were hidden in aggregate metrics.
• Lesson: Trust requires segmented divergence analysis, not just aggregate accuracy.

1. Scaling ≠ Context Management

• Mistake: I underestimated how stateful reasoning affects horizontal scaling.
• Failure: Context windows overflowed mid‑investigation, with no design for summarization or retrieval.
• Lesson: Context management must be designed up front — deciding what to retain, summarise, or fetch on demand.

The New Design Vocabulary

Distributed systems gave us a vocabulary that has served for decades: contracts, statelessness, idempotency, circuit breakers, blue‑green deployments. Agentic systems don’t discard it — they extend it:

• API contracts still matter → authority limits sit alongside them.
• Operational logging still matters → governance audit trails sit alongside it.
• Blue‑green deployments still matter → shadow mode precedes them.
• Circuit breakers still matter → confidence threshold fallbacks extend them.
• Stateless design still matters → stateful reasoning sits above it.
• Single responsibility still matters → agent specialisation applies it at the capability level.
• Monitoring still matters → behavioural observability sits alongside it.
• Idempotency still matters for reads → reversibility governs consequential actions.

🔑 Closing Thought

The architects who will design the best agentic systems aren’t those who discard their distributed systems background. They’re the ones who know which principles transfer unchanged, which transform, and which are genuinely new.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI