The model became cheaper; the system became expensive: the new AI cost chain

Between November 2022 and October 2024, the inference cost of a GPT-3.5-level system fell more than 280-fold, according to Stanford's 2025 AI Index. Later Epoch AI analyses also find rapid declines when price and performance are compared together, although the rate varies substantially by benchmark. That transformation is real. It does not, however, establish that AI products became cheap.

The model price is only the first line in a chain that runs through context, tools, orchestration, evals, CI, observability, security, human intervention, incidents, and learning. As inference gets cheaper, the relative share of those other bills grows. The bottleneck moves.

Executive summary

The useful denominator is not a call, token, or task started. It is the outcome accepted:

modelos + dados/contexto + ferramentas + execução/CI + verificação
+ operação + trabalho humano + falhas + evolução
---------------------------------------------------------------
outcomes aceitos

This formula is not an accounting standard. It is an observability contract that prevents savings in one layer from being celebrated while cost, risk, or rework moves to another.

The complete chain

1. Inference: the visible line item

Tokens, images, audio, cache, batch, reasoning, and repeated calls remain relevant. Price declines are not uniform across quality, modality, latency, or context windows. Compare models on the same task, with the same acceptance criterion and period; do not turn a benchmark into a future invoice.

2. Data and context: what feeds the decision

Documents must be discovered, authorized, cleaned, versioned, indexed, and retired when they expire. Cheap retrieval from the wrong source produces a cheap answer and an expensive decision. Licensing, quality, privacy, and governance costs belong to the same economic unit.

3. Tools and integrations: every action expands the surface

Search, databases, browsers, queues, storage, and APIs charge in money and complexity. An agent may perform many operations to complete one delivery. The AWS Agentic AI Lens warns that plan–act–verify–reflect loops and multi-agent coordination can increase consumption, latency, and tool calls. A cost limit without a stopping condition is merely a late warning.

4. Harness and orchestration: making capability repeatable

Prompts are one component. State, identity, capabilities, routing, policies, memory, sandboxing, timeouts, retries, and handoffs form the system. Anthropic recommends starting with simple patterns and adding agentic behavior only when it materially improves the result, because autonomy trades predictability and latency for performance on suitable tasks.

The harness also accumulates experimental knowledge: which model works for each task class, what context is sufficient, which verifiers detect failures, and when to escalate. Calibrating it is an investment; maintaining it is a recurring cost.

5. Evals and verification: turning output into trust

Agents are harder to evaluate because they choose paths, use tools, and change state. Anthropic's evals guide recommends task sets, graders, and transcript analysis; those measurements also establish baselines for latency, tokens, cost per task, and error types.

Removing evals makes the bill look smaller by ceasing to measure failure. Too many poorly chosen evals create slow, expensive CI. The goal is risk-proportional coverage, with sampling, tiers, and deterministic gates before more expensive judges.

6. CI and infrastructure: the tax on frequent change

Every change to a prompt, model, tool, policy, dataset, or dependency may require tests, builds, ephemeral environments, scanners, runners, storage, and repetition. AI-native systems increase both the rate of change and the number of possible configurations. Without risk-based test selection, safe caching, controlled concurrency, and expiring environments, the cost of proving a change may exceed the cost of running it.

This is a sign of maturity, not an argument for removing CI. The question is: what evidence must be produced for this class of change, and which duplicate execution adds no confidence?

7. Operations and security: the system after deployment

Gateways, observability, egress, secrets, identity, rate limits, isolation, retention, support, and recovery do not appear in the demo. The NIST AI RMF and its generative-AI profile treat measurement, monitoring, and risk management as ongoing work. Savings are not sustainable when logs, controls, or rollback are removed to lower the current bill.

8. Human work: the bill that often disappears

People prepare cases, review exceptions, explain decisions, recover from failures, and support customers. The State of FinOps 2026 reports that 98% of respondents already manage AI spending, but only 28% include labor in the calculation. The survey covers 1,192 professionals representing more than US$83 billion in annual cloud spend; it is a portrait of the FinOps community, not of every company.

Human review is not automatically waste. It may be the right control. It must be visible by task class and consequence so that pricing, margin, and automation are honest decisions.

9. Failure, rework, and debt: costs that arrive later

A rejected output consumes every preceding layer and does not enter the denominator. Incidents add investigation, containment, communication, rollback, and trust. More than a decade ago, *Hidden Technical Debt in Machine Learning Systems* showed that rapid ML gains can carry maintenance costs through data dependencies, feedback loops, configuration, and boundary erosion. LLMs changed the interface; they did not eliminate that dynamic.

10. Energy and capacity: the systemic bill

The IEA estimates that global data-center electricity consumption will rise from about 485 TWh in 2025 to roughly 950 TWh in 2030, with AI accounting for a large share. This scenario should not be allocated linearly to a request or product. It shows how energy, grids, chips, location, and capacity lead times can reappear as price, constraint, and supply-chain risk.

Four engineering decisions that change the curve

Route by task, not prestige

FrugalGPT and RouteLLM show, in specific benchmarks and configurations, that cascades and routing can reduce cost while preserving quality. Their results are not a production guarantee. The transferable pattern is to maintain a case set, an acceptance criterion, and a fallback, then promote the cheapest route that passes the gate.

Put stopping conditions in the budget

Limits on turns, time, tokens, calls, mutation, and cost must produce stop, escalate or inconclusive; never a pass through exhaustion. Budget is a property of the execution contract, not a dashboard observed afterward.

Measure goodput, not activity

Throughput counts attempts. Goodput counts valid results within quality and latency targets. The FinOps Foundation proposes connecting tokens, inference, and workflow to cost per outcome. In FORGE, the equivalent is preserving a receipt that links version, cost, verifier, and acceptance. Observability must reach a unit of value and its cost, not stop at the technical resource. [R3-C2]

Treat CI as an evidence portfolio

Organize suites in layers: fast deterministic contracts; selective integration; sampled evals; red-team and long-running exercises on their own cadence. Record failure rate, flakiness, time, cost, and bugs detected by each suite. Disabling a test requires evidence of redundancy or irrelevance, not merely a high bill.

The minimum dashboard per outcome

Do not use zero when the data does not exist. Use unknown, declare the allocation method, and reconcile estimates against official billing interfaces. A token estimate does not replace an invoice. [R3-C3]

What may come next

Five pressures are likely to become more important:

1. Cost of evidence. Greater autonomy will require more evidence of authorization, provenance, and verification; evidence itself may become an explicit infrastructure class.
2. Margin volatility. Outcome-based pricing transfers the risk of retries, exceptions, and model changes to the provider; gross margin will need to be observed by cohort and task.
3. Localized scarcity. Energy, chips, networks, data residency, and capacity windows can constrain expansion even when the model's nominal price falls.
4. Evaluation debt. Suites age, datasets stop representing actual use, and graders may share the generator's biases. Maintaining measurement will be a continuous internal product.
5. Economic portability. Switching models will look simple in a diagram and be expensive in behavior: prompts, tools, latency, security, and acceptance rates will need recalibration.

These are planning hypotheses, not confirmed forecasts. The purpose is to make them observable before they become a financial surprise.

A 30-day plan

1. choose a flow with volume, an owner, and an acceptance criterion;
2. record a baseline for cost, latency, rework, and intervention;
3. assign version, model, task, customer/cohort, and environment tags;
4. add model, platform, tools, CI/evals, and human time;
5. divide by accepted outcomes, preserving unknown;
6. find the two largest cost shares and the two largest causes of rejection;
7. make one change at a time and compare against the same case set;
8. keep quality, risk, and margin as simultaneous guardrails.

Limitations and conflicts of interest

• Stanford, Epoch AI, METR, FrugalGPT, and RouteLLM study different periods, tasks, and benchmarks; no figure should be projected directly onto a company.
• The METR study found a 19% slowdown across 246 tasks completed by 16 experienced developers in mature open-source repositories. It is important evidence against generalization, not a universal productivity estimate.
• AWS, Anthropic, and Stripe are vendors with commercial interests in the categories they describe.
• Bessemer is an investor; its margin references are a market thesis, not an independent audit.
• The IEA describes global infrastructure; it does not attribute consumption to this article, FORGE, or any product.
• Trustyu and Tech Human commercially provide architecture, governance, and AI product-building services.

Direct sources

1. Stanford HAI — AI Index Report 2025.
2. Epoch AI — LLM inference price trends.
3. FinOps Foundation — State of FinOps 2026.
4. FinOps Foundation — FinOps for AI.
5. AWS — Agentic AI Lens: Cost Optimization.
6. Anthropic — Building effective agents.
7. Anthropic — Demystifying evals for AI agents.
8. Sculley et al. — Hidden Technical Debt in Machine Learning Systems.
9. NIST — Generative AI Profile, NIST AI 600-1.
10. IEA — Key questions on energy and AI.
11. Chen et al. — FrugalGPT.
12. Ong et al. — RouteLLM.
13. METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity.
14. DORA — State of AI-assisted Software Development 2025.

Editorial and responsibility note

Data and conclusions were linked to the cited sources and read within their stated scope. The cost chain, dashboard, and 30-day plan are the author's professional synthesis, not an accounting standard, savings promise, or claim about Trustyu's private costs. This content is informational and does not replace financial, technical, legal, or security assessment. Research and drafting had AI assistance; factual review, authorship, and publication approval belong to Fernando Parreiras. There was no additional independent human review.