Author(s): Dave R | Microsoft Azure & AI MVP ☁️

Originally published on Towards AI.

How the Azure OpenAI Decisions API in Microsoft Foundry turns routing, triage, and guardrail checks into typed probabilities your code can act on

If your agent asks a generative model “is this ticket urgent?” and then parses the reply, you pay for output tokens nobody reads and maintain parsing code that breaks on the first unexpected sentence. The Azure OpenAI Decisions API fixes this. You send evidence and typed questions, and you get back a probability, a choice, or a score, billed on input only at $0.10 per million tokens on GPT-6 Luna. The verdict: put every routing, triage, and guardrail decision in your agent on this endpoint, and keep generative models for the steps that produce content. This article covers the request contract, the three question types, a reference architecture, threshold and refusal handling, caching math, and a checklist you can apply today.

After the introduction, the article explains why generated text is a poor tool for decisions—agents make many small gating calls that don’t need prose, yet generation forces extra parsing, repairs, and retries. It then details the Decisions API request contract (endpoint, model/deployment mapping, auth, SDKs, regions), the “decision plane” architecture that separates control-flow decisions from content generation, and the response structure. The core of the guide breaks down the three question types (predicate, choice, score) and shows how to interpret probabilistic outputs safely—especially highlighting the “score trap” where different distributions can share the same averaged score. It covers using images as evidence, batching many independent questions in one request, handling dependent question sequences across multiple requests, and converting probabilities into auditable actions via calibrated thresholds and explicit refusal routing. The article also provides pricing and prompt-caching math (including cache-token telemetry and cache-write behavior), guidance for choosing the right tool versus the Responses API and function calling, parity notes for Azure versus OpenAI’s platform, and “early numbers” indicating large latency improvements when placing fast decisions in front of generation. It concludes with a production checklist and final thoughts emphasizing that most agent work is choosing what happens next—and that Decisions API gives those choices a typed, testable path your code can reason about.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI