What a production AI budget contains that a pilot budget does not, why the second workflow costs less than the first, and which cost figures still need sourcing.
2 min read
What the current generation of agent tooling changes for an operations team, what it leaves exactly as hard as it was, and which figures still need sourcing before publication.
2 min read
Draft — not for publication. Every figure is an unfilled placeholder. The argument is publishable; the numbers are not, until each carries a source and a date.
A pilot is judged on whether the output looks impressive. A production system is judged on what happens on a bad day. Almost every difficulty in moving from one to the other comes from that change of question, and almost no pilot budget accounts for it.
Evaluation. A held-out set of real cases with known-correct outcomes, maintained as the business changes. Without it there is no way to tell whether a prompt change helped.
Human review, permanently. A standing share of outputs read by a person. This is a running cost, not a launch cost, and it is the first line cut when the system appears to be working.
Monitoring and a rollback path. Traces of what was retrieved and decided, alerts on distribution shifts, and a way to revert to the manual process within the hour.
Corpus maintenance. Documentation, rubrics and policies go stale. Whoever owns them needs the time to own them.
Access and audit. Least-privilege credentials, retention rules for run logs, and an answer to "who saw this data" that survives an audit. Typical share of total cost across these lines: [PLACEHOLDER — verify before publishing]
The first production workflow pays for the whole platform: identity, logging, evaluation harness, review tooling, deployment. The second pays for itself only. Teams that budget the second like the first conclude that automation is expensive; teams that budget the first like the second run out of money before launch. Observed ratio between first and subsequent workflow cost: [PLACEHOLDER — verify before publishing]
Not "what will this save" but "what will we do differently when it is wrong". A budget with no answer to the second question is a pilot budget wearing a production label, whatever the figure at the bottom of it.