A cost structure finance hasn't budgeted for before
Traditional infrastructure cost was predictable enough to forecast a year out: you provisioned capacity, and the bill followed roughly the plan. AI inference cost doesn't behave that way. It moves with how many people use a feature, which model a request gets routed to, how long a conversation runs, and how aggressively an agent decides to reason before responding, none of which shows up on a capacity plan.
The result is a cost line that can move meaningfully month to month without anyone changing a budget, a headcount, or a contract. Finance teams used to managing IT spend as a fixed cost are discovering it now behaves more like a variable cost tied to product usage, and needs to be modeled that way.
Where the money actually goes
In every AI cost review we've run, the single biggest lever isn't the sticker price of the model being used. It's how many requests get routed to a frontier-tier model by default when a smaller, cheaper model would produce an equivalent result for the task at hand. Most organizations default to the most capable model everywhere because nobody owns the decision to route intelligently, and it's the single highest-leverage fix available.
The second-largest lever is usually unbounded agent reasoning: a system that's allowed to think, retry, and call tools with no ceiling on how much of that it does before returning an answer. Put a budget on it the way you'd put a budget on anything else, and the cost curve changes immediately without touching quality in any way a user notices.
The biggest cost lever is rarely the model's price. It's how many requests get routed to an expensive model when a cheaper one would do the job.
Fahmid Khan, Chief Technology Officer
Why this needs joint ownership
AI cost sits in an awkward spot organizationally: engineering understands the technical levers, model routing, caching, prompt length, but rarely owns the P&L impact. Finance owns the P&L impact but usually lacks the technical fluency to evaluate whether a cost spike reflects growth, waste, or a model choice that could be swapped without anyone noticing.
The functions handling this well have built a joint review, engineering and finance in the same room monthly, looking at cost per feature and cost per user alongside the usual infrastructure dashboards. Neither function can do this well alone, and treating it as purely an engineering problem or purely a finance problem is how the cost gets managed too late to matter.
What a working AI FinOps practice actually does
It tracks cost at the level of a feature and a customer segment, not just a model or a vendor invoice. It sets explicit ceilings on how much reasoning or tool use an agent can do per task, reviewed and adjusted like any other budget. And it routes requests to the cheapest model that meets a defined quality bar by default, reserving the most expensive models for the tasks that actually require them.
None of this is exotic. It's the same operating discipline finance already applies to every other significant variable cost in the business. The only real change is that engineering now has to be at the table when that discipline gets applied, because they're the ones who can see the lever finance can't.
