How Much Does It Cost to Host an AI Model on the Cloud?
At 50M input and 12M output tokens per month, an open model like Llama 3.3 70B costs about $45/mo on AWS Bedrock and $44/mo on Azure. A flagship like GPT-4o runs ~$245/mo and Gemini 2.5 Pro ~$183/mo, while budget models such as Amazon Nova Lite ($6/mo) or Gemini 2.0 Flash and GPT-4o mini (~$15/mo) are dramatically cheaper. Output tokens dominate the bill because they cost 2-5x more than input.
Meta (open weights)
Open-weight model offered as managed inference on AWS Bedrock, Azure AI Foundry, and Cloudflare Workers AI — a true like-for-like price comparison. Note Cloudflare charges much more for output than input.
Llama 3.3 70B Instruct — 50M tokens in / 12M tokens out per month
Same model, priced on 3 clouds. Cheapest is highlighted.
Workers AI
$41.69
per month
- Input
- $0.29/1M
- Output
- $2.25/1M
- You save
- $2.95/mo
Azure OpenAI / AI Foundry
$44.02
per month
- Input
- $0.71/1M
- Output
- $0.71/1M
Amazon Bedrock
$44.64
per month
- Input
- $0.72/1M
- Output
- $0.72/1M
Which model should you use?
The best-value model on each cloud for the job, at 50M tokens in / 12M tokens out. — means the cloud has no comparable managed model in that tier.
| Use case | AWS | Azure | Google Cloud | Cloudflare |
|---|---|---|---|---|
Budget / high-volume Classification, extraction, cheap chat — cost over reasoning depth | Amazon Nova Lite $5.88/mo | GPT-4o mini $14.70/mo | Gemini 2.0 Flash-Lite $7.35/mo | Llama 3.1 8B Instruct $24.02/mo |
General-purpose agents Balanced quality and cost for everyday assistant and RAG work | Llama 3.3 70B Instruct $44.64/mo | Llama 3.3 70B Instruct $44.02/mo | Gemini 2.5 Flash $45.00/mo | Llama 4 Scout 17B $23.70/mo |
Coding & agents Strong reasoning and tool use for dev workflows and autonomous agents | Claude Sonnet 5 $330.00/mo | GPT-4.1 $196.00/mo | Gemini 2.5 Pro $182.50/mo | — |
Premium / frontier The hardest, long-horizon and agentic work where quality wins | Claude Opus 4.8 $550.00/mo | — | Claude Opus 4.8 $550.00/mo | — |
Tiers are our editorial read of each model's sweet spot at its price, not a benchmark score. Frontier tiers (Claude Opus/Fable, Gemini 2.5 Pro) live on AWS Bedrock and GCP Vertex; Azure and Cloudflare top out at general/coding models today.
How this is calculated: monthly cost = (input millions × input $/1M) + (output millions × output $/1M). Prices are on-demand / pay-as-you-go list rates per 1M tokens for US regions, last verified 2026-08-04. They exclude batch (~50% off on Bedrock), provisioned throughput, and committed-use discounts. Google offers Llama via Vertex Model Garden but not at a flat published token rate, so Google is represented by its native Gemini models. Model a full application stack with the cloud cost calculator.
Running a large language model in production means paying per token on a managed inference API — AWS Bedrock, Azure OpenAI / AI Foundry, Google Cloud Vertex AI, or Cloudflare Workers AI. The same model can cost very different amounts depending on the cloud and how much you send it. Pick a model and your monthly input/output token volume below to compare the real per-1M-token cost across providers.
How Managed Model Inference Is Priced
Unlike VMs or databases, managed foundation-model APIs bill per 1,000,000 tokens, split into an input (prompt) rate and a usually-higher output (completion) rate. Your monthly bill is simply (input millions × input $/1M) + (output millions × output $/1M). Because output tokens are typically 2-5x the price of input tokens, workloads that generate long responses cost far more than their prompt volume suggests. On-demand pricing needs no commitment; batch and provisioned-throughput tiers trade flexibility for lower rates.
Bedrock vs Azure OpenAI vs Vertex AI
AWS Bedrock hosts open-weight models (Meta Llama, Amazon Nova) and Anthropic Claude behind one API. Azure serves OpenAI's GPT-4o family plus open models like Llama through Azure AI Foundry. Google Cloud Vertex AI offers its own Gemini family. For an open model available on more than one cloud — Llama 3.3 70B, for example — you get a true like-for-like comparison: it runs about the same on AWS Bedrock and Azure (~$0.71-0.72 per 1M tokens each way). For proprietary flagships, each cloud is the only source: GPT-4o only on Azure, Gemini only on Vertex, Nova and Claude only on Bedrock.
How to Cut Inference Costs
The biggest lever is model choice: a budget model like Amazon Nova Lite, GPT-4o mini, or Gemini 2.0 Flash can be 20-50x cheaper than a flagship for tasks that don't need frontier reasoning. After that, shorten prompts and cap output length, cache repeated context, and move steady high-volume workloads to batch or provisioned-throughput pricing. Always compare the cheapest capable model on each cloud rather than defaulting to the best-known name.
Key Cost Factors
- Model tier: flagship vs budget models differ 20-50x per token
- Input tokens: your monthly prompt volume in millions
- Output tokens: completion volume, usually priced 2-5x higher than input
- Cloud: the same open model can vary in price between providers
- Pricing mode: on-demand vs batch (~50% off) vs provisioned throughput
- Context length: some models (e.g. Gemini Pro) charge more for long contexts
Frequently Asked Questions
How much does it cost to host an AI model on AWS, Azure, or Google Cloud?
For managed inference you pay per 1M tokens, not per server. At a moderate 50M input / 12M output tokens a month, an open model like Llama 3.3 70B costs roughly $45 on AWS Bedrock and $44 on Azure. Flagship models are far more: GPT-4o on Azure is about $245/mo and Gemini 2.5 Pro on Vertex about $183/mo at the same volume. Budget models (Amazon Nova Lite, GPT-4o mini, Gemini 2.0 Flash) drop the same workload to $6-$15/mo.
Which cloud is cheapest for LLM inference?
It depends on the model. For an open model offered on multiple clouds, such as Llama 3.3 70B, AWS Bedrock and Azure AI Foundry are within a rounding error of each other (~$0.71-0.72 per 1M tokens each way). For the absolute lowest cost, the budget tier matters more than the cloud: Amazon Nova Lite on Bedrock, GPT-4o mini on Azure, and Gemini 2.0 Flash on Vertex are all in the same low range. Use the calculator to compare at your own token volume.
Why are output tokens more expensive than input tokens?
Generating tokens requires a forward pass through the model for each one, so output (completion) tokens are more compute-intensive than reading the prompt. Providers price this in: output rates are typically 2-5x the input rate. That means a chatbot that returns long answers costs more than its prompt volume implies, and trimming max output length is one of the fastest ways to cut an inference bill.
How accurate are these AI inference cost estimates?
Prices are on-demand, pay-as-you-go list rates per 1M tokens for US regions, last verified 2026-08-04, drawn from AWS Bedrock's pricing page, the Azure Retail Prices API, and Google's Vertex AI pricing page. They exclude batch (~50% off on Bedrock), provisioned throughput, and committed-use discounts. Gemini Pro output is tiered by context length, and Bedrock lists a temporary Claude Sonnet promo — see the notes in the tool. Treat the numbers as a provider comparison, not a guaranteed invoice.