To cut cloud and LLM costs at a SaaS company, start by making spend visible for each team, product and customer. Then remove idle and oversized resources. Next, lower what each LLM request costs with model routing, prompt caching and token budgets. Finally, build cost guardrails into the pipeline and make each team own its bill.
Why do cloud and AI bills grow faster than revenue at SaaS companies?#
Cloud and AI bills grow faster than revenue because you pay per unit of usage, while most customers pay a flat subscription, and nobody owns the gap between the two.
In practice, a few things push the bill up:
- Heavy tenants cost more than they pay. Without cost tracked per customer, your ten largest accounts may be costing far more than their contracts bring in, and you won't see it.
- AI features have flat prices but variable costs. A user who pastes long documents into a copilot can cost many times more per session than one who asks short questions.
- Environments pile up. Preview, staging and load-test stacks keep running after the work that needed them is done.
- Nobody owns data and query costs. Dashboards and AI agents each run their own expensive queries against the warehouse.
AI is making the waste worse. Flexera's 2026 State of the Cloud report found that fast-growing AI workloads pushed estimated wasted cloud spend up to 29%, the first rise in five years. In the same survey, 81% of respondents said they use generative AI, up from 72% the year before and 47% in 2024.
How do you see spend per team, product and customer?#
Tag every resource and every LLM call with an owner, a product and a tenant. Then pull all billing data into one consistent dataset you can query.
- Agree on a tagging scheme. At minimum:
team,product,environmentandcost-center. Enforce it in infrastructure-as-code so untagged resources fail the deploy instead of showing up later as "unallocated." - Write rules for shared costs. Shared Kubernetes clusters, databases and networking need a written way to split them, such as by namespace resource requests or by request volume. Pick one rule and keep it stable so month-to-month trends mean something.
- Send all LLM traffic through one gateway. For each call, record the model, input and output tokens, the feature and the tenant. Provider invoices can't tell you which customer or feature used the tokens. Your gateway can.
- Use one billing format. The FinOps Open Cost and Usage Specification (FOCUS) is an open standard for billing data. It gives cloud providers, AI services and SaaS vendors a common format, so you can compare a hyperscaler bill and an LLM bill side by side.
- Track unit metrics, not just totals. Useful ones: cost per active customer, cost per 1,000 API requests, and AI cost per user as a share of the plan price.
Where does cloud waste usually hide?#
Most waste sits in resources nobody is watching: non-production environments, oversized compute, forgotten storage and data workloads with no owner.
| Where it hides | How you spot it | Typical fix |
|---|---|---|
| Dev, staging and preview environments | Running nights and weekends at production size | Shut them down on a schedule; delete them when the pull request merges |
| Oversized instances and containers | Requested CPU and memory far above actual use | Rightsize from real usage data; turn on autoscaling |
| Forgotten storage | Unattached volumes, old snapshots, log buckets with no expiry | Lifecycle and retention policies |
| Data transfer | Traffic between regions or zones from chatty services | Put services that talk a lot in the same place |
| Warehouse and query spend | A few dashboards or jobs account for most compute | Assign each query and pipeline an owner |
| Commitments | Reserved capacity left unused, or steady workloads on on-demand prices | Review commitment coverage every quarter |
| AI infrastructure | Idle inference endpoints, full evaluation runs on every small change | Scale to zero; run evaluations sized to the change |
Of these, data platforms are the most often missed. Our Data Platforms work traces every pipeline and query to an owner and a cost, because warehouse spend tends to grow faster than usage when nobody owns query costs.
What drives LLM costs per request?#
What one request costs depends on how many tokens you send, how many come back, which model handles them, and how many model calls one user action triggers.
A simple way to calculate it:
Cost per request = (input tokens × input price) + (output tokens × output price) + retrieval and infrastructure costs, minus any caching discounts. Multiply by the number of model calls per user action.
The main drivers:
- Prompt size. The system prompt, tool definitions, retrieved documents and chat history all count as input on every call. A long chat resends its whole history each turn.
- Output length. Answers with no length limit, and wordy reasoning steps, add billed tokens.
- Model tier. Using a frontier model to classify a support ticket is often the biggest single overspend.
- Calls per action. An agent that plans, calls tools, checks itself and retries can make many model calls for one click.
- Retries and failures. Timeouts that retry the same large prompt get billed twice.
A made-up example: a copilot request carries about 6,000 tokens of system prompt and tool definitions, 3,000 tokens of retrieved context, 500 tokens from the user and 400 tokens of output. Most of the input is the same on every call. That repeated part is your first target.
Our Generative AI Apps practice treats cost as a design input. Each feature gets latency and cost targets per request. Caching, streaming and token limits are tracked per feature, so you know the price of a request before usage grows.
How do model routing, caching and fallbacks lower token spend?#
Send each request to the cheapest model that passes your evaluation set. Reuse context you've already paid for. When a provider fails, switch to another one instead of retrying the expensive call blindly.
- Model routing. Send simple tasks, like extraction, classification and short rewrites, to a small model. Save the large model for hard reasoning. If the small model's answer fails a format or grounding check, send it up to the large model. The router is only as good as your evaluation set, so build that set from real user questions and score every routing change against it.
- Prompt caching. Put static content (system prompt, tool definitions, reference documents) at the start of the prompt so it can be cached. Anthropic bills cache reads at 0.1× the base input price, but writing to the cache costs 1.25× for a five-minute lifetime. Caching only pays off if the same prefix is reused before it expires. OpenAI applies caching automatically to prompts of 1,024 tokens or more.
- Response caching. For questions that come up again and again, store the answer at the application level. Keep a separate cache for each tenant and permission level, so one customer never gets an answer built from another customer's data.
- Batching. Work nobody is waiting on, such as nightly summaries, data enrichment and evaluation runs, doesn't need real-time prices. OpenAI's Batch API launched with a 50% discount and returns results within 24 hours.
- Fallbacks and retry caps. When a provider slows down or fails, switch to another provider or model. Cap the number of retries, so one outage doesn't double your bill.
- Trimming. Limit output tokens, summarize long chat histories, and retrieve fewer, better-ranked chunks.
What guardrails keep costs down once they're cut?#
Savings last when cost checks sit where engineers already work, in pull requests, the deploy pipeline and the LLM gateway, instead of in a monthly finance report.
- Cost estimates in pull requests. Show the estimated monthly impact of infrastructure changes before they merge. The FinOps Foundation's State of FinOps 2026 names this "shift left" as a trend that is already happening.
- Untagged resources fail the deploy. If a resource has no owner, it doesn't ship.
- Evaluations that check cost. Make token count and latency part of the evaluation suite. A prompt change that doubles tokens should fail review just like a quality regression.
- Token budgets per feature and per tenant. Send a soft alert as usage nears the budget. At the hard limit, switch to a cheaper model, rate-limit, or ask the owner to approve more.
- Anomaly alerts sent to the team that owns the spend. A daily alert to the engineers responsible works better than a monthly email to finance.
- Pricing that matches cost. Fair-use limits on AI-heavy plan tiers keep heavy users from eating the margin.
Who should own cloud and AI spend in an engineering organization?#
The engineering teams that create the spend should own it. A small central function owns the data, tools and rules. Finance owns forecasts, and product owns the gap between what each customer pays and what each customer costs.
The FinOps Foundation's State of FinOps 2026 reports that 78% of FinOps teams now report to the CTO or CIO, with a dotted line to finance. Flexera's 2026 survey found that 63% of organizations have set up FinOps teams.
A growing SaaS company doesn't need a large FinOps team on day one. A workable split:
- Engineering teams: own their tagged spend and unit costs, and act on anomaly alerts.
- Platform or central FinOps lead: owns the tagging scheme, the FOCUS-format data, the LLM gateway and the shared-cost rules.
- Product: owns cost per customer compared with plan price, and decides on fair-use limits.
- Finance: owns forecasts and commitment purchases, using engineering's usage data.
- CTO: runs a monthly review of unit cost trends, not just the total bill.
Our Cloud Cost Optimization consulting is built for teams whose cloud and AI bills grow faster than revenue. We show where the money goes, cut the waste, and set guardrails so it stays cut. The consultants are engineers who also build, so recommendations come with a path to production. We start with an assessment, test the plan in a paid pilot, and then scale.
Conclusion#
You can only cut what you can see. Tag every resource and every LLM call, track cost per customer and per feature, and remove the obvious waste first. Then lower what each request costs: route to the cheapest model that passes your evaluations, cache repeated context, batch work nobody is waiting on, and cap retries. Finally, build budgets and cost checks into the pipeline and the gateway, and make the teams that create the spend responsible for it.
Want to know what your cost per request should be? Talk to an engineering lead →
Frequently asked questions
What is a good target for AI cost as a share of SaaS revenue?
There's no universal benchmark, because it depends on your pricing model and how central AI is to the product. Set a target for AI cost per user as a share of each plan's price, track it monthly, and measure it per tenant, not only as one total.
Is a self-hosted open-weight model cheaper than a commercial API?
Sometimes, but only at steady, high volume. Self-hosting turns per-token costs into GPU, operations and on-call costs, and idle capacity is expensive. Compare total cost per request at your real traffic, including engineering time, and run both options against the same evaluation set.
How quickly can a SaaS company see savings from FinOps work?
Removing idle environments, unattached storage and oversized instances often pays off within the first billing cycle. LLM savings from caching and routing show up as soon as they're deployed and validated. Lasting savings depend on guardrails, which take longer to build.
Does prompt caching ever increase costs?
It can. Some providers charge more to write to the cache than for normal input, so if a cached prefix expires before reuse you've paid extra. Caching pays off when a large, unchanging prefix is reused often within the cache's lifetime.
Do we need a dedicated FinOps tool to get started?
Not at first. Consistent tags, billing exports in a common format such as FOCUS, and an LLM gateway that logs tokens per tenant and feature answer most early questions. A dedicated tool makes sense with several clouds, many teams, or a need for automated anomaly detection.
- FinOps
- Cloud Cost Optimization
- LLM Costs
- SaaS
- Generative AI
- Prompt Caching
Newsletter
Get new articles by email
A short monthly round-up of what we've written on engineering pods, quality and AI. No spam.
