Author(s): Shrashti Singhal

Originally published on Towards AI.

Your agent doesn’t have a performance problem. It has a unit-economics problem — and the fix is engineering, not a bigger budget.

There’s a moment every team building agents eventually hits. I’ve started calling it the invoice moment.

The article argues that agent costs are an engineering problem driven by token accounting, not a vague “model performance” issue. It explains why agent conversations are input-dominated (earlier turns get resent and re-billed), introduces a ledger-based approach to track five token types (including cache reads/writes and thinking tokens) so teams can attribute spend to tasks, and shows how prompt caching can provide a large discount when prompt prefixes remain byte-for-byte stable. It then covers cost optimization levers such as model routing (static/dynamic routing and cascades, with attention to verification and reasoning effort), the sharp multiplier of multi-agent systems (often around 15x due to repeated prompt/tool costs and limited cache sharing), and how the real metric is cost per completed task with attention to long-tail “doom loop” failures rather than averages. The piece concludes with practical harness controls—budgets and circuit breakers, trace-based cost attribution, percentile dashboards, deduping and result caching, batching async work, output dieting, and careful planning/ROI checks for advanced tactics—illustrated through annotated “before/after” invoices and a first-week rollout plan.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI