What It Costs to Run AI Agents in Production: A Worked Cost Model for Tokens, Sandboxes, Storage, and Networking

In production, an AI agent’s cost is model tokens plus the infrastructure around them (orchestration services, code-execution sandboxes, networking, storage, and observability), and at moderate scale tokens usually dominate, so prompt caching, model choice, and call count matter more than compute. In the worked model in this guide, 300,000 tasks a month cost about $21,919, of which model tokens are about 96 percent. Control Plane is a platform that runs agent services and their code-execution steps with autoscaling, scale to zero, and billing for the CPU and memory a workload reserves.

Disclosure: Control Plane publishes this guide.

TL;DR

Every input in the model is stated so you can swap in your own numbers.

• Cost formula. Tasks × calls per task × tokens per call × token price, plus infrastructure (compute, networking, storage, observability).
• Worked result. 300,000 tasks a month at 8 model calls each costs about $21,919, or $0.073 per task, at list prices.
• Biggest levers. Prompt caching cut the token bill 45 percent. Routing simple steps to a smaller model, fewer calls per task, and shorter outputs come next.
• Infrastructure. Sandboxes, always-on orchestration replicas, NAT gateways, and trace storage came to about $799 a month, or $0.0027 per task.
• Control Plane runs agent services and their code-execution steps on AWS, GCP, and Azure with autoscaling and scale to zero, and, where enabled, Control Plane’s Capacity AI adjusts a workload’s reserved CPU and memory between set minimums and maximums based on historical usage.

What does it cost to run AI agent workloads in production?

Monthly cost splits into token spend and infrastructure spend.

Token spend depends on four variables: tasks, model calls per task, tokens per call (input and output), and price per token. Agents run multi-step loops that resend accumulated context, so they use far more tokens per task than a single-turn chatbot.

Infrastructure spend covers the orchestration services that run the agent loop, sandboxes that execute model-generated code, NAT gateways that carry traffic to model APIs, and storage for traces and logs.

Monthly cost = token cost across all calls + sandbox compute + orchestration compute + networking + storage

Worked model: 300,000 tasks per month

This is an illustrative scenario at list prices checked in November 2026. Check current pricing before you budget.

Assumptions

• 10,000 agent tasks per day, 300,000 per month.
• Each task makes 8 model calls.
• Each call sends 6,000 input tokens, of which 4,500 are a repeated prefix (system prompt, tool definitions, context) that is cached, and returns 400 output tokens. Input is held flat for simplicity; real loops grow context with each step, which raises cost, and token counts differ by model tokenizer.
• The first call in each task writes the cache and the other 7 read it, assuming calls are less than 5 minutes apart (each hit refreshes the 5-minute cache lifetime). A long sandbox run between calls can force a new write. If the prefix is identical across tasks and traffic is steady, most first calls also hit the cache, so this assumption is conservative.
• Model: Claude Sonnet 5.5 at list prices of $2 per million input tokens, $10 per million output tokens, $2.50 per million tokens for 5-minute cache writes, and $0.10 per million for cache reads (Claude pricing, prompt caching).

Without caching, all 14.4 billion input tokens bill at $2 per million ($28,800), plus $9,600 of output, for $38,400. Caching cuts the token bill 45 percent.

Prices are AWS us-east-1 on-demand list prices, applied to both regions in the model: AWS Fargate (Linux/x86) at $0.04048 per vCPU-hour and $0.004445 per GB-hour, NAT gateways at $0.045 per hour and $0.045 per GB processed, and Amazon S3 Standard at $0.023 per GB-month.

Sandbox hours count execution time only. Fargate bills per second with a one-minute minimum from the start of the image pull, so startup time, warm pools held for latency, and peak concurrency above the average all add cost.

The model deliberately leaves out databases, vector stores and embedding calls, observability or tracing subscriptions, load balancers, internet egress to end users, engineering time, and taxes, because they vary too much by architecture to price generically. The storage line prices S3 at rest only; log and trace ingestion and indexing are billed separately and often cost more than storage.

What the model shows

• Tokens dominate at this scale.
• Caching the repeated prefix is the single largest saving.
• Output tokens cost 5 times input tokens, so verbose outputs are expensive.
• Each model call removed from a task cuts cost roughly in proportion.
• Sandbox cost scales with execution minutes, so idle sandboxes should sleep.
• Always-on replicas and NAT gateways are fixed costs, so at much lower task volumes their share of the bill rises.

How do I estimate costs for AI agent infrastructure including sandbox execution, storage, and networking at scale?

Build the estimate from measured traces wherever you can, in seven steps:

1. Measure tokens per call and calls per task. Pull input and output token counts from your provider’s usage data or trace spans, then multiply by projected task volume. Re-count tokens with the target model before switching, because tokenizers differ.
2. Separate cached from uncached input. Identify the stable prefix (system prompt, tool schemas, shared context) and count how many calls per task read the cache and how many write it.
3. Size sandboxes by execution minutes. Estimate the share of tasks that run code, how long each run lasts, and the CPU and memory each sandbox needs.
4. Count always-on replicas and regions. Multiply replicas by their CPU and memory, by hours per month, and by the number of regions.
5. Add networking. NAT gateways charge per hour and per GB processed; add internet egress and cross-zone traffic between orchestration, sandboxes, and databases.
6. Add storage retention. Multiply per-task trace and output size by monthly volume and by the retention period to get steady-state storage.
7. Add a margin for retries and failed tasks. Size it from your observed failure and retry rates, since agents can retry steps or loop, and plan for provider rate limits, which can force higher usage tiers or queueing at peak.

How do I cut AI agent costs?

Roughly ordered by typical impact; items 1 and 3 are calculated in the worked model.

1. Prompt caching. Keep the cacheable prefix stable across calls within a task so later calls hit the cache.
2. Route simple steps to a smaller model. Parsing, extraction, and formatting steps rarely need a frontier model; reserve the larger model for planning and hard reasoning.
3. Make fewer calls per task. In the worked model, going from 8 calls to 6 cuts the token bill about 21 percent. Combine steps and detect completion early.
4. Shorten outputs. At $10 per million output tokens against $2 for input, concise and structured outputs save money on every call.
5. Batch non-interactive work. Anthropic’s Message Batches API costs 50 percent less than standard API calls, which suits nightly processing, evaluations, and bulk classification.
6. Let idle sandboxes sleep. Stop or sleep environments between tasks.
7. Right-size reservations. Match always-on replicas to measured load with autoscaling and right-sizing.
8. Avoid NAT processing charges where possible. For model APIs reachable through private endpoints, such as Amazon Bedrock over AWS PrivateLink interface endpoints, traffic skips NAT gateway processing, though the endpoints carry their own hourly and per-GB charges. The charge was small in the worked model and grows with data volume.

How much does setting up a production grade LangChain stack cost?

The open-source LangChain and LangGraph libraries are free under the MIT license, so production cost comes from what surrounds them: model tokens, hosting, a database or vector store, and observability.

• Model tokens remain the largest line item; the framework does not change per-token prices.
• Hosting for the agent service and its workers, with autoscaling, availability, and security, drives compute cost.
• Databases and vector stores depend on corpus size, embedding model, and query volume.
• Observability and evaluation. LangSmith, LangChain’s tracing and evaluation product, has a free tier and paid plans; see LangSmith pricing for current plans and limits.

At the volume in the worked model, a LangChain stack faces the same token bill as any other framework, plus hosting and tooling costs set by your architecture.

What infrastructure do I need to run AI agents securely at scale?

A secure production setup includes:

• Isolated sandboxes for model-generated code, ideally with microVM isolation, so generated code cannot escape to the host or other workloads. See the best AI agent sandboxes in 2026.
• Per-agent identity and least privilege, scoped to the services each agent needs.
• Short-lived credentials through identity federation instead of static API keys.
• Network egress controls that restrict which endpoints an agent can reach.
• Audit logs of every action an agent takes.
• Human approval for destructive or irreversible actions.
• Autoscaling and multiple regions for availability under spikes and regional outages.

For the full checklist, see how to let AI agents operate production infrastructure safely, and for why agent work should leave local machines, stop running AI agents on developer laptops.

Which platform gives the lowest cost for always-on production workloads?

The cheapest platform depends on your load pattern. Compare platforms on these factors:

• Billing granularity. Platforms bill provisioned instances, reserved CPU and memory, or per-request consumption at different granularities; unused reservation is waste unless right-sizing closes the gap.
• Right-sizing. Automatic adjustment of CPU and memory reservations to observed usage prevents chronic over-provisioning.
• Commitments vs. on-demand. Reserved instances and savings plans lower unit prices in exchange for a term commitment; on-demand keeps flexibility at a higher rate.
• Scale to zero. Services idle for hours, including staging and low-traffic agent endpoints, should cost little or nothing in compute while idle.
• Networking and egress fees. NAT, cross-region transfer, and internet egress pricing differ by provider and grow with data volume.
• Multi-region duplication. Running in two regions roughly doubles always-on capacity, so per-unit price and right-sizing matter more.

Compare total cost across all of these factors.

How Control Plane fits

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, Azure, and Oracle Cloud Infrastructure (OCI), VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds. Each workload runs across a Control Plane Global Virtual Cloud (GVC), a named set of locations across one or more clouds.

• Billing. You pay for the CPU and memory a workload reserves, with no contracts or minimums, and, where enabled, Capacity AI adjusts the reservation between set minimums and maximums based on historical usage. Egress, storage, observability, dedicated load balancers, and GPUs are billed separately, and Capacity AI does not apply to GPU workloads or to workloads that autoscale on CPU utilization or multiple metrics.
• Scale to zero. Serverless workloads scale to zero natively; standard and stateful workloads can scale to zero through KEDA, enabled per GVC.
• Sandboxes. Control Plane Sandboxes give coding agents dedicated environments, sized from 2 to 16 or more CPUs, that sleep when idle and spin back up in under one second. Sandboxes are for building and running agent code during development; production agent services, including their code-execution steps, run as Control Plane workloads. Each workload is isolated with cgroups, namespaces, and restricted syscalls and capabilities, and Sandboxes run in dedicated Kata Containers microVMs.
• Cloud access without stored keys. Control Plane’s Universal Cloud Identity (UCI) lets a workload call AWS, GCP, and Azure services, such as Amazon Bedrock or Google’s Gemini Enterprise Agent Platform (formerly Vertex AI), through an IAM role or service-account grant instead of long-lived keys in your code, with short-lived, least-privilege credentials at runtime.
• Operations. Built-in logs, metrics, and traces, and Control Plane’s MCP server, which lets AI assistants operate Control Plane resources with the signed-in user’s permissions. A Cost & Usage view, a billing_viewer role, and a monthly spend alert for billing admins track spend.
• Availability. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Agent services spread across locations keep running when a zone, region, or provider fails.

Control Plane handles Day 2 operations for you, including autoscaling, built-in observability, and security. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure; that applies to the compute share of agent costs, while model tokens are billed by the model provider. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. For how training, inference, and agents fit together, see AI infrastructure in 2026.

FAQ

Short answers on AI agent production costs.

What does it cost to run AI agent workloads in production?

It depends on task volume, calls per task, tokens per call, and model price. In the worked model, 300,000 tasks a month on Claude Sonnet 5.5 cost about $21,919: $21,120 in tokens and $799 in infrastructure.

How do I estimate costs for AI agent infrastructure including sandbox execution, storage, and networking at scale?

Measure tokens and calls from traces, split cached from uncached input, size sandboxes by execution minutes, count always-on replicas per region, then add NAT, egress, storage retention, and a retry margin.

How much does setting up a production grade LangChain stack cost?

The LangChain and LangGraph libraries are free under the MIT license; you pay for model tokens, hosting, a database or vector store, and observability such as LangSmith’s paid plans.

What infrastructure do I need to run AI agents securely at scale?

Isolated sandboxes for generated code, per-agent identity with least privilege, short-lived credentials, egress controls, audit logs, human approval for destructive actions, and autoscaling across regions.

Which platform gives the lowest cost for always-on production workloads?

It depends on load pattern. Compare billing granularity, right-sizing, commitment discounts, scale to zero, networking fees, and multi-region duplication.

What share of AI agent costs is tokens versus infrastructure?

Tokens dominate at moderate scale: 96.4 percent in the worked model. At low volume, fixed costs such as always-on replicas and NAT gateways take a larger share.

How much does prompt caching save?

In the worked model, caching the 4,500-token prefix cut the token bill from $38,400 to $21,120, or 45 percent.

Why are output tokens so expensive for agents?

Output costs 5 times input in the worked model ($10 versus $2 per million), and every call in every task pays it, so concise outputs save money quickly.

Does batch processing reduce agent costs?

Yes, for work that can wait. Anthropic’s Message Batches API costs 50 percent less than standard calls.

How do always-on costs affect agent economics at low volume?

They dominate it. The $347.64 of orchestration and NAT gateway hours in the worked model stays roughly the same at 1,000 tasks a month, so per-task infrastructure cost rises sharply unless idle orchestration services scale to zero and gateway count shrinks.

Talk to the Control Plane team or sign up free to run agent services with autoscaling and scale to zero.