How AI API pricing actually works
Last updated 2026-07-18
AI API pricing looks simple until you try to forecast a real bill. Most text models charge separately for input tokens and output tokens. Some providers add prompt caching, batch discounts, tool charges, image or audio units, and provisioned throughput. The published rate card is the beginning of the calculation, not the end.
A token is a chunk of text. It may be a word, part of a word, punctuation, code, or markup. Input tokens are what you send to the model: instructions, user messages, retrieved documents, tool results, and chat history. Output tokens are what the model generates back. Output is often priced higher because generation is computationally expensive and happens step by step.
Why input/output mix changes the winner
Two workloads with the same total tokens can have very different costs. A document summarizer may send 80 million input tokens and produce 5 million output tokens. A long-form writing tool may send 10 million input tokens and produce 40 million output tokens. If output is much more expensive, the second workload may cost more even though the total token count is smaller.
Why invoices can differ from naive estimates
Prompt caching can reduce the cost of repeated context. Batch APIs can reduce the cost of work that can wait. Tool use can add separate charges. Long-context prompts may have different rates. Some enterprise deployments use committed capacity or provisioned throughput. And some coding tools wrap raw API usage inside subscriptions or credit pools. That is why AICostCompass uses monthly workload assumptions instead of only listing unit prices.
How to use this in practice
For a first estimate, write down request volume, average input tokens, average output tokens, and whether the workflow can use caching or batch mode. Then compare providers in the comparison tool. Do not blend input and output into one number unless the provider actually prices them as one unit. Blending is the fastest way to mis-rank providers.
References and fact checks
- Anthropic Claude API pricing - documents separate input, output, cache write, and cache hit pricing
- Google Gemini API pricing - documents input/output prices, context caching, batch pricing, tools, and media notes
- AWS Bedrock cost and usage report guide - explains why production billing needs distinct token usage types
- OpenAI prompt caching announcement - explains automatic caching for repeated prompt prefixes and discounted cached input
How to do this in AICC
Turn the article into an answer you can use
Use AICC to translate token pricing mechanics into a real monthly estimate, including input/output mix and provider-specific unit differences.
- 1
Use Pricing to inspect units
Open the Pricing map and look at the unit column for each provider row. Confirm whether the price is per input token, output token, image, video second, seat, or another unit.
- 2
Shortlist comparable models
Choose models that can plausibly do the same job. Comparing a budget classifier to a frontier coding model is only useful if both are realistic options for your workflow.
- 3
Use Compare for workload math
Open the Comparison tool and enter monthly input and output tokens separately. AICC will convert the unit rates into estimated monthly and annual cost.
- 4
Use ROI after the cost is clear
Once you understand the monthly estimate, open ROI and test whether the workflow saves enough human time to justify the spend.
What you should have at the end
You should understand not only which model has a lower unit price, but which one is cheaper for your actual monthly workload and business value.
Frequently asked questions
Why is output priced higher than input on most AI APIs?
Generating tokens is computationally sequential and more expensive than processing input, so providers typically charge more per output token than per input token.
Can I estimate cost by just adding input and output tokens together?
No. Blending input and output into one number hides real cost differences, since providers price them separately and at different rates — keep them separate when comparing.
Do prompt caching and batch discounts change the published rate card?
They don't change the published rate, but they reduce the effective price for stable repeated context (caching) or work that can wait (batch), so your real bill can be lower than the raw unit price suggests.