This is the first article in a six-part series exploring the economics of AI from a Finance and FP&A perspective. The series will examine what really drives AI cost, how Finance can forecast and govern it, and how organisations can connect technical activity to business value.

Understanding the Economics of AI

There is no shortage of writing about AI, much of it with an understandable level of concern about the enormous investments being made in models, infrastructure and data centres. At the same time, people and companies are getting to grips with the technology itself, whether experimenting with ideas, building pilots or trying to scale solutions that already work. One thing they increasingly have in common is a desire to understand and manage what all of this actually costs.

For someone working in Finance, this is familiar territory. What does it really cost, and how do we know whether it is worth it - in other words, is it creating value?

AI initially looks unusually accommodating on the first question. Tokens can be counted, providers publish prices and usage can be measured in impressive detail. Yet after spending much of the past year studying AI, I have become increasingly interested in what sits behind those apparently simple numbers.

Reading around the subject, there is already plenty being written about the cost of AI. Two pieces in particular caught my attention: Shrashti Singhal’s explainer of agent economics and an FT Alphaville analysis of open-model economics. Both go beyond headline token prices and raise useful questions about routing, model choice, infrastructure and how changing technology could alter the economics of using AI.

What interests me from a Finance and FP&A perspective is what happens when we follow those costs through the wider system. A token is easy to count, but an AI application may also retrieve information, call tools, route work between models, retry failed steps, use cached context, consume dedicated infrastructure or eventually hand something back to a human. Reducing one visible cost does not necessarily reduce the cost of the whole process. This is where my interest in systems thinking becomes useful: a change intended to improve one part of the system can shift cost, effort or risk somewhere else, so the economics need to be understood at the level of the whole process rather than through one metric alone.

This article explores how Finance might think about AI economics as these systems move from experimentation towards wider use. I want to understand why simply counting tokens is unlikely to be enough, what actually drives the cost of AI-enabled work, and what Finance, FP&A and CFO teams can start putting in place now so that they are better positioned to understand, forecast and eventually optimise those costs as they grow.

I do not expect there to be one finished answer. Models, architectures, prices and use cases are changing far too quickly for that. My aim is more practical: to pull together what I have learned so far, connect it with some familiar finance concepts, and explore what a useful management framework might look like before the bills become large enough to make the question rather less theoretical.

Looking Beneath the Token Price

Part of what has changed my thinking is simply learning more about what sits underneath the price we eventually see. My studies at ETH Zürich over the past year have moved progressively from models and neural networks into RAG, agents, infrastructure and, more recently, the economics of computation. That does not make me an infrastructure engineer, but it does make it rather harder to look at a provider bill and regard the number as self-explanatory.

I learned that the cost of training a model is driven by its size, measured in parameters, and the amount of data used to train it. Together, they determine the amount of computation required; that computation becomes accelerator hours, and those hours ultimately cost money and require energy, which has its own cost dynamics.

The end cost, therefore, reflects decisions about model size, hardware, workload design and how efficiently the available infrastructure is being used.

Utilisation is a good example. A GPU, which is needed to run frontier LLMs, for example, may have an impressive theoretical maximum, but large-model training runs can operate well below peak performance because the machine is not simply performing useful calculations at maximum speed every second. Data has to move, workloads have to be coordinated, batches vary and hardware can sit waiting. There is a distinction between capacity purchased and productive work performed, which should sound familiar to anyone noticing they’re the only one left in the office at 4pm on a sunny Friday!

The same principle becomes even more relevant when we move from training models to using them. A business application is unlikely to consist of one neat stream of identical requests. Some tasks are simple, some difficult; some require current information, others do not; some contain sensitive data; some need a very high level of reliability. The resources required to deal with them can therefore be very different, even if they all appear under the same heading of “AI usage”.

Tokens are not a universal unit of AI consumption either. Image-generation models based on diffusion, for example, have different underlying cost drivers: model size, image resolution, the number of iterative denoising steps and, in practice, how many attempts are needed before a usable image is produced. Speech, video and other AI modalities have their own resource profiles. The Finance question is still the same: what activity consumed the resources, what drove that consumption, and what useful outcome did it produce?

My first assignment on my latest course in building AI/ML applications makes that problem rather concrete. We were asked to build an LLM router: a system that receives a question and chooses which model should answer it. The stated objective is to select the lowest-cost model that is still expected to produce a good answer. The exercise does this by predicting an error probability for each available model and combining that prediction with the known cost of calling the model. The balance between quality and cost can then be adjusted depending on how much weight the user wishes to place on either side.

For an assignment, this is a pleasingly well-behaved optimisation problem. We know the price of each model, we have a way of estimating whether it will answer correctly, and we select accordingly. For a real business application, however, the optimisation is rather less clear-cut.

The price of the model call may be only part of the cost. The application may first retrieve documents, create embeddings, build a larger context, use cached information, call an external tool, make several model calls, retry when one fails and eventually ask a person to intervene. Much of the model activity will still show up as token consumption when using a commercial API, but other parts of the workflow may be charged separately through retrieval, storage, external APIs or other infrastructure. If the models are running on dedicated or self-hosted infrastructure, the same underlying computation does not disappear: the cost simply shows up differently, through accelerator capacity, utilisation, energy and the infrastructure required to run it. If the task contains sensitive information, the cheapest technical route may not be an acceptable one at all.

The definition of a “good answer” also starts to expand. Correctness matters, but so might latency, security, resilience, traceability, compliance and the amount of human checking subsequently required. A more expensive model could therefore produce a cheaper overall process if it succeeds more often or reduces downstream work. Equally, paying frontier-model prices for a task that a much smaller model could handle reliably would be difficult to defend once the volumes become meaningful.

I think the simple router problem is a useful way of looking at the larger economics: the real question is no longer which model is cheapest, but what it costs to achieve an acceptable outcome.

Sources:

1. Shrashti Singhal. The Economics of Agents: Token Accounting, Caching, and Routing

2. Financial Times / Alphaville. How big is the open-model threat to AI hyperscalers?