Given how all the recent headlines have companies laying off tens of thousands of human workers in favor of “cheaper” AI automation, it seems impossible that the reverse is now true.
But go back a few years ago, and the AI productivity story sounded simple.
Give employees access to generative AI. Let them summarize documents faster. Then developers can write code faster. Let analysts create reports faster. Enable call center teams to draft responses faster. Then watch labor costs go down.
That was the pitch to profit-motivated, cost-conscious executives.

And to be fair, it was not completely wrong. A well-used AI assistant can absolutely reduce the time required to draft an email, summarize a meeting, review a document, generate test cases, explain code, or prepare a first-pass analysis. For many knowledge workers, AI is still one of the most powerful productivity tools introduced in the last twenty years.
But a new problem is starting to show up inside enterprises: The AI bill is no longer invisible.
That is the uncomfortable truth behind the recent Economy Media video titled “How AI Became More Expensive Than The Workers It Replaced,” which discusses the idea of “token maxing.” The exact phrase may sound like internet shorthand, but the underlying issue is very real. AI does not work like traditional software licensing. It doesn’t even behave like a fixed desktop productivity tool. AI may feel like software, but economically it behaves more like metered infrastructure. Each prompt has a cost. Each agent retry adds more. Each repository scan, long conversation thread, and oversized document stuffed into a frontier model burns compute before the business ever sees the final answer.
That distinction is the whole point.
The enterprise AI question now shifts from just, “Can AI do the work?” The better question is, “Can the organization control the economics of AI doing the work?”
The Hidden Shift from Software Cost to Compute Cost
Most enterprise leaders are used to software costs that are relatively predictable. You buy a license, and then assign seats and estimate usage. Maybe you negotiate an enterprise agreement. The cost may still be painful, but at least the commercial model is familiar.
Generative AI changes that.
The unit of consumption is often the token. Tokens represent chunks of text processed by the model. Think of tokens like miles on a toll road. The AI model is the vehicle, and the text you send through it is the trip. The more text the model has to read and the more text it has to generate, the farther the trip. Tokens are the unit used to measure that distance, and the bill increases as the trip gets longer.
In the corporate context, tokens are like the billable miles of an AI conversation. Every word, phrase, document excerpt, code block, and generated response adds distance to the trip. A short prompt is a quick drive across town. A long document review is a cross-state haul. The model may still get you there quickly, but the meter is running the entire time.
Input tokens are what the model reads. Output tokens are what the model generates. Some platforms also charge for cached input, tool calls, file search, code execution, web search, storage, embeddings, vector databases, and higher-priority processing.
OpenAI’s current API pricing, for example, lists GPT-5.5 at $5 per 1 million input tokens, $0.50 per 1 million cached input tokens, and $30 per 1 million output tokens. GPT-5.5 Pro is much higher at $30 per 1 million input tokens and $180 per 1 million output tokens. OpenAI also separately prices platform tools such as web search, file search, code interpreter containers, storage, and other services.
Anthropic’s Claude pricing follows the same basic consumption logic. Its platform documentation shows token-based API pricing, and its Claude Code documentation specifically notes that coding costs vary by model selection, codebase size, and usage patterns, including multiple instances and automation. Anthropic reports that enterprise Claude Code deployments average around $13 per developer per active day and $150 to $250 per developer per month, with 90% of users staying below $30 per active day.
That may sound affordable until you scale it across thousands of employees, dozens of applications, multiple departments, heavy coding usage, large document workflows, and agentic automation.
A single AI answer can be cheap, but an unmanaged AI operating model can be quite expensive as companies like Microsoft recently learned.
What “Token Maxing” Really Means
“Token maxing” is not just employees using AI a lot. That would be too simplistic. Token maxing is what happens when AI usage becomes disconnected from business value.
Token maxing usually does not look reckless at first. In fact, it looks like an employee asking for ten versions of the same short email, a developer letting a coding agent keep chasing a failing build, or a business analyst repeatedly uploading the same 200-page policy because no one built a better retrieval path. Over time, the organization starts mistaking AI activity for AI value. Usage goes up, token consumption climbs, and teams treat the rising compute burn as evidence of productivity, even when no one has tied it back to a measurable business outcome.
Of course not.
That is like measuring cloud success by CPU consumption instead of business throughput. Nobody should celebrate a higher AWS or Azure bill unless there is a corresponding business result. The same logic applies to AI.
The danger is that token usage can feel productive. People see activity. They see drafts. They see summaries. They see code. They see responses. But underneath that activity, the enterprise may be paying for repeated reasoning, bloated prompts, long context windows, unnecessary retries, and premium models being used for low-value work.
Keep that in mind: AI cost is not just a finance issue. It is an architecture issue.
Side Note: Compute burn is the consumption of processing resources (such as GPU time, CPU time, memory, storage, and network activity) required to run a digital workload. In AI, it usually refers to the infrastructure cost created each time a model reads input, generates output, calls tools, retries tasks, or runs agentic workflows.
Is AI Actually More Expensive Than Human Labor?
The honest answer is: sometimes.
For small, bounded tasks, AI is usually much cheaper than human labor. If a person spends fifteen minutes drafting a status update and an AI model can generate a usable first draft in seconds for a fraction of a cent or a few cents, the economics are pretty obvious. The same is true for many routine summarization, classification, extraction, and translation tasks.
But enterprise AI does not stay inside neat little one-prompt examples.
Once AI becomes embedded into workflows, the cost profile changes at scale. A simple chatbot may answer with a single model call, but agentic AI changes the economics. A coding agent might move through the repository, write a fix, run the tests, fail, read the logs, adjust the code, and try again. A research agent may search, summarize, challenge its own findings, compare sources, and synthesize a final answer. In a financial services setting, a compliance assistant may need to pull policy language, transaction history, member correspondence, and regulatory guidance before it can draft a recommendation. What looks like one user request on the surface may actually be a chain of expensive AI work underneath. That’s a compute workflow.
Research on agentic coding tasks has found that these workflows can consume dramatically more tokens than simple code chat or code reasoning. One 2026 paper found that agentic coding tasks can consume 1,000 times more tokens than simpler code interactions, that token usage can vary by up to 30 times across runs of the same task, and that higher token usage does not necessarily produce higher accuracy.
That last point matters. More tokens do not automatically mean better work. Sometimes they just mean the model wandered longer.
This is where the “AI is cheaper than labor” argument starts to break down. A human developer may take longer, but the human may also stop, reason from experience, ask one clarifying question, or recognize that a path is wrong. An agent may just keep spending tokens unless the architecture tells it to stop.
The Data Center Bill Behind the Token Bill
There is also a larger infrastructure story behind this.
Tokens come across as digital and abstract, but they are backed by very physical infrastructure: GPUs, power, cooling, networking, storage, data centers, specialized chips, cloud platforms, and capital spending. McKinsey has estimated that global data center capital expenditures could reach $6.7 trillion by 2030, with $5.2 trillion tied to AI-capable data centers. That doesn’t mean every AI prompt is expensive. It means the industry is building enormous infrastructure capacity to satisfy demand that is still economically unstable.
Someone has to pay for that infrastructure.
The infrastructure cost does not disappear just because it is hard to see. In some cases, the AI vendor carries it. In others, the cloud provider absorbs part of it or investors subsidize the growth curve. Eventually, though, the economics surface somewhere: in API usage fees, higher subscription prices, throttling, premium tiers, or new usage-based billing models.
The important point for enterprise architects is that AI is not mystical, mysterious labor floating in the cloud. It is a compute-intensive service running on expensive infrastructure. If the enterprise doesn’t govern it, the cost will show up somewhere.
Now bring this into financial services.
The Financial Services Problem
A bank or credit union cannot treat enterprise AI like a casual productivity experiment once it touches lending, fraud, servicing, payments, collections, disputes, regulatory compliance, cybersecurity, financial advice, or member communications. The risk profile notably changes.
The risk depends on where AI sits in the business process. An assistant cleaning up an internal meeting note is one thing. An assistant shaping how a member understands a loan, fee, dispute, fraud claim, account restriction, or payment exception is something else entirely. Once an AI agent can reach into systems, retrieve sensitive data, call APIs, or recommend operational decisions, the institution is no longer dealing with a simple productivity tool. It has placed cost, data, model, operational, compliance, and audit risk into the same workflow.
Financial regulators already expect institutions to manage technology architecture, infrastructure, and operations through sound governance. The FFIEC Architecture, Infrastructure, and Operations booklet describes examiner expectations around risk management processes for the design, implementation, and operation of technology architecture and infrastructure in financial institutions. That applies directly to AI.
NIST’s Generative AI Profile also frames generative AI risk management as a lifecycle concern, not a one-time approval event. It’s intended to help organizations identify generative AI risks and align risk management actions with their goals and priorities.
Model risk guidance is also evolving. In April 2026, the OCC announced updated model risk management guidance relevant to larger banking organizations and institutions with significant model risk exposure. Importantly, the OCC noted that generative AI and agentic AI models are novel and rapidly evolving and are not within the scope of that specific guidance. As a result, this creates an important gap.
Traditional model risk governance still matters, but it does not fully cover the way generative AI and agentic AI are entering regulated workflows. These tools may not make the final decision, yet they can still shape the work around that decision. They may not approve the loan, but they may draft the explanation. They may not file the suspicious activity report, but they may summarize the case. They may not release the payment, but they may recommend the next operational step.
That’s the gap financial institutions need to close with AI cost architecture governance, which alludes to AI cost architecture governance beyond the typical AI governance (or model governance, cloud cost management).
AI Cost Must Become a Non-Functional Requirement
Enterprise architects are already used to non-functional requirements. We ask about availability, scalability, performance, security, recoverability, observability, compliance, latency, data retention, and supportability.
AI cost now belongs in that same conversation. For every meaningful AI use case, the architecture review should ask:
What is the expected AI cost per business outcome?
Monthly spend and user counts only tell part of the story. The more useful measure is cost per outcome: the cost to resolve a servicing case, triage a fraud alert, summarize a loan document, complete an after-call summary, review a regulatory policy, accept a code change, or analyze a payment exception. That is the number that shows whether AI is creating measurable value or simply increasing activity.
An AI solution that costs $50,000 per month may be a bargain if it eliminates $500,000 of manual effort and improves quality. Another solution that costs $5,000 per month may be wasteful if it generates low-value drafts nobody uses.
The architecture discipline is to connect token consumption to business value.
The New Governance Pattern: AI Use Case Intake
Financial institutions need formal AI use case intake before production usage scales.
This does not mean every employee prompt needs an architecture review. That would be absurd. But any AI capability embedded into a business process, enterprise application, operational workflow, member-facing experience, engineering pipeline, or regulated decision support process should go through intake.
The intake should capture enough context to make the AI use case governable. The business owner explains why the capability is needed and who benefits from it. The technology owner is accountable for how it is built, integrated, supported, and changed over time. The data owner confirms what information the AI can access and whether that access is appropriate. The user population matters because a tool used by five analysts has a different exposure profile than one used by 5,000 branch or contact-center employees.
The intake should also describe the business process being affected, because AI risk depends heavily on where the tool sits in the workflow. Risk tier and data classification help determine the level of review, control, and monitoring required. Model selection matters because different models carry different cost, performance, privacy, and reliability tradeoffs. Expected volume, token consumption, monthly cost, and cost per transaction help the institution understand whether the economics make sense before usage scales.
Finally, the intake should identify whether human review is required, how prompts and outputs will be logged, what fallback process exists if the AI fails, and which vendors or third-party platforms are involved. Without that information, the organization may approve an AI use case that works in a demo but is difficult to govern, audit, control, or afford in production.
That intake should also classify the use case.
An internal summarization assistant is different from a fraud investigation assistant. A coding assistant is different from a loan servicing copilot. A policy research tool is different from a member-facing financial guidance bot. A chatbot that answers HR questions is different from an agent that can call payment APIs.
The governance mistake is treating all AI usage as the same category. It is not.
Model Routing Is Architecture, Not Optimization
One of the fastest ways to waste AI spend is to send everything to the most expensive model.
That may be fine during experimentation, but it is not a production architecture.
A mature financial institution should route work by task complexity and risk. Routine classification can go to a smaller model. Simple summarization can use a mid-tier model. Sensitive member-facing explanations may require a stronger model plus approved retrieval and human review. Complex architecture analysis may justify a frontier reasoning model. Autonomous agent workflows may require a restricted model set, iteration limits, tool limits, and stronger observability.
The principle is simple: use the cheapest model that can safely and reliably perform the task.
That is not anti-AI. That is basic engineering economics.
No architect would recommend running every workload on the largest compute instance available. No cloud architect would approve premium storage for every data set by default. No network architect would route every workload through the most expensive path without a reason.
AI should follow the same discipline.
Token Budgets Need to Be Designed Into the Platform
Token budgets should not be managed through after-the-fact spreadsheet reviews.
They need to be designed into the AI platform.
A financial institution should define AI limits at the levels where cost and risk actually show up. Enterprise-level limits protect the overall technology budget. Line-of-business limits make each business area accountable for its own AI consumption. Application and use-case limits keep a single solution from exceeding its approved purpose or budget. User-group limits recognize that a small expert team and a large contact-center population will create very different usage patterns. Agent-workflow limits are especially important because one instruction can trigger many model calls, tool calls, retries, and intermediate steps. Individual request limits prevent one oversized prompt or runaway task from consuming resources out of proportion to its value.
The platform also needs technical ceilings that can be enforced at runtime. Maximum input tokens control how much text the model can read. Maximum output tokens control how much it can generate. Context-length limits prevent teams from dragging unnecessary history, documents, or logs into every request. Tool-call and agent-iteration limits stop workflows from looping through expensive steps. Runtime limits keep tasks from running longer than intended. Cost-per-workflow limits connect each AI run to an economic boundary, while daily and monthly spend limits prevent small overages from turning into a material budget problem.
This is especially important for agentic AI.
A chatbot may be annoying when it gives a bad answer. An agent can be expensive while giving a bad answer. It can search too broadly, call tools unnecessarily, retry the same failing task, generate excessive intermediate output, or spin through a workflow that should have stopped five steps earlier.
Agentic AI needs a budget, a leash, and a kill switch.
That may sound harsh, but in financial services it is just responsible architecture to keep the lights on within budget limits.
Another major source of AI waste is poor context design.
Retrieval Beats Brute Force Context
One of the easiest ways to waste AI spend is to give the model far more context than it needs. It happens innocently enough. A user pastes an entire policy, contract, procedure, call transcript, log file, or code file into a prompt because that feels like the fastest path to an answer. Developers often do the same thing in early prototypes. The model gets a massive context window, the demo looks impressive, and everyone celebrates that the AI can “read everything.”
But reading everything is not the same as understanding what matters.
A better architecture is more selective. It retrieves only the smallest approved slice of context needed to answer the question, using governed retrieval, source-level access controls, clean chunking, useful metadata, freshness rules, data minimization, sensitive-data filtering, and clear citations back to the source. That approach reduces token cost because the model processes less unnecessary text. It also reduces risk because sensitive information is not sprayed into the AI workflow just because it happened to be nearby.
That matters even more in financial services. Banks and credit unions handle nonpublic personal information, transaction records, account data, credit information, fraud notes, identity data, and internal control documentation. When too much of that context is pushed into an AI model, the institution increases both compute cost and data exposure.
The design pattern should not be “give the model everything,” but instead “give the model the right thing.”
AI FinOps Has to Evolve Beyond Cloud FinOps
Most enterprises already have some muscle memory around Cloud FinOps. They know how to look at cloud spend, reserved capacity, utilization, tagging, showback, chargeback, and optimization. That discipline still matters, but AI introduces a different kind of consumption problem. The cost is no longer tied only to infrastructure running in the background. It is also tied to how often models are called, how much context they process, how much they generate, and how many steps an AI workflow takes before it produces something useful.
That is why AI FinOps has to extend the cloud cost conversation into model consumption.
The organization needs to understand which models are being used, how much input and output they are processing, how often cached context is being reused, and when tools such as file search, web search, code execution, embeddings, vector databases, retries, and agent loops are driving up the bill. But the reporting cannot stop there. The more important question is how that usage maps back to real users, departments, applications, and business outcomes.
A dashboard that only shows total AI spend is useful, but incomplete. It may tell leaders that the bill is going up, but it does not tell them whether the spend is justified. A better dashboard shows where AI is creating measurable value and where the organization is simply burning compute.
A contact-center copilot may look expensive until leadership sees that it reduces after-call work, improves consistency, shortens handle time, and lowers escalation volume. A coding agent may look inexpensive until the data shows it generates plenty of pull requests but very few accepted changes. A compliance assistant may appear productive until audit discovers that employees are running the same prompts over and over instead of using approved reusable workflows.
That is the real purpose of AI FinOps. It should not exist to punish employees for using AI. It should help the enterprise use AI with economic intelligence.
Governance Should Productize Repeated Work
One of the biggest AI cost opportunities is also one of the easiest to miss.
When hundreds or thousands of employees keep asking AI the same kind of question, the organization should stop treating those questions as isolated prompts. At that point, the repeated behavior is telling architecture something important. It is a candidate for a reusable enterprise capability.
That might be a policy interpretation pattern, a servicing script, a compliance summary, a code review prompt, a document analysis workflow, or a knowledge assistant built around a controlled source of truth. The specific use case will vary, but the architectural lesson is the same: repeated AI work should be productized.
Otherwise, every employee ends up asking the AI to interpret the same policy. Every analyst uploads the same procedure. Every developer invents a slightly different version of the same prompt. Every business team builds its own assistant over similar knowledge. The company pays for the same reasoning again and again, while also increasing the chance of inconsistent answers.
A better approach is to turn common patterns into approved prompt templates, reusable workflows, shared retrieval indexes, department-specific copilots, precomputed summaries, approved answer libraries, and governed AI skills. This is where architecture governance can reduce cost and improve quality at the same time. Employees get more consistent results, and the organization stops paying for duplicated computation.
That is a win on both sides of the ledger.
Human-in-the-Loop Is Also a Cost Control
Human review is usually described as a risk control, and that is true. But it is also a cost control.
A person can stop an AI workflow that is clearly going nowhere. A fraud analyst can reject a weak recommendation before more time is wasted. A developer can recognize that an agent is attacking the wrong problem. A compliance reviewer can identify when a generated memo is too speculative. A call-center supervisor can tell when an answer should be shorter, clearer, and more direct instead of longer and more expensive.
The key is to apply human review based on the risk of the work.
Low-risk productivity tasks should not be buried under unnecessary approval. A meeting summary does not need a governance committee. But AI-generated member-facing language, fraud recommendations, payment exception analysis, lending support, collections scripts, and account restriction recommendations require stronger oversight. Those workflows sit closer to regulated decisions, customer impact, financial exposure, and operational risk.
In financial services, the most dangerous pattern is not AI helping humans. The dangerous pattern is AI quietly becoming the human without the institution admitting it.
Runtime Enforcement Matters More Than Policy Documents
Policies are necessary, but they’re not enough. If the enterprise says employees cannot use unapproved models, but people can still paste sensitive data into any AI tool, the policy is mostly theater. If the AI standard says token limits are required, but the application has no enforcement mechanism, the standard is only a suggestion. If the architecture review says human approval is required, but the workflow can still call write APIs directly, the control is weaker than it looks.
AI governance has to move into runtime enforcement.
That is where AI gateways, API policies, model allowlists, model denylists, token ceilings, per-user quotas, per-application quotas, sensitive-data detection, prompt filtering, tool-access controls, logging, budget alerts, automatic throttling, and kill switches become important. The objective of all this is to make the approved behavior the easiest and safest behavior.
A central AI gateway can become one of the most important control points in the enterprise AI architecture. It can enforce model routing, capture telemetry, apply cost limits, integrate with identity, inspect prompts for sensitive data, log model versions, and provide the evidence needed for audit and operational review.
The Architecture Review Board Has a New Job
The Architecture Review Board still needs to ask the familiar questions. Is the solution secure? Is it resilient? Does it align to standards? Does it fit the enterprise architecture? Can it be supported?
But for AI, that is no longer enough.
The ARB also needs to ask whether the solution is economically bounded. What is the expected monthly cost? What is the cost per business transaction? What happens if adoption is ten times higher than expected? What happens if an agent loops? What happens if users keep uploading large documents instead of using retrieval? What model-routing logic prevents premium models from being used for routine work? What telemetry proves that the AI is producing business value instead of just generating activity?
Those questions may feel new, but they are a natural extension of architecture governance. AI cost is now part of solution design.
The Bottom Line
AI has not universally become more expensive than human labor. That claim is too broad.
For many tasks, AI is still dramatically cheaper than manual work. But enterprises should not take comfort in that general statement. AI can become more expensive than expected when organizations confuse AI activity with AI value. Token maxing, long-context abuse, agent loops, premium-model defaulting, repeated prompts, poor retrieval design, and weak FinOps can turn AI into an unmanaged compute bill.
Financial institutions should pay especially close attention because they operate in regulated, risk-sensitive, audit-heavy environments. A bank or credit union cannot afford AI systems that are opaque, economically uncontrolled, poorly logged, loosely governed, or casually connected to sensitive workflows.
The next phase of enterprise AI will not be won by the organizations that simply use AI everywhere. It will be won by the organizations that know where AI behaves like cheap labor, where it behaves like expensive compute, and where it is too risky to run without human control. That’s the real architecture challenge.
AI does not need governance because it is bad. It needs governance because it is powerful, scalable, easy to overuse, and expensive when nobody is watching.
The companies that understand that will build AI into their operating model with discipline. The companies that don’t will eventually discover that their productivity revolution came with a data center invoice attached (and data centers are a whole other topic that deserves its own series).















