Why can’t you measure your AI ROI? Because you are not running simulations

The most repeated sentence in consulting reports is this: “Companies invest in AI but do not get the expected return.” There is tech spend, but no result. The board asks: what did this AI investment earn? And most of the time, no one can give a clear answer.

This looks like a measurement problem: “we are not tracking the right metrics.” But there is a deeper problem. ROI cannot be measured because there is no basis to measure against. The AI system is put into production without its behaviour in the real operational context ever being simulated; so its value can neither be estimated beforehand nor compared against a reference afterward.

ROI is a comparison: “what happened with this decision, what would have happened without it?” To make this comparison, you need a reference point — a baseline, a counter-scenario. When no simulation is run, this reference point never forms. And without a reference point, ROI stays a feeling, not a measure.

The right question is not “which metrics should we track?” It is:

To measure the value this AI creates, do we have a reference scenario to compare it against?

ROI requires a comparison

Measuring the return of an AI investment is not looking at a single result on its own. The sentence “sales rose after AI” is not proof of ROI — because sales may have risen for other reasons. Real ROI is the difference between “the result with AI” and “the result without AI.”

This difference requires a counter-scenario: what would have happened without AI? This is where most companies stumble. The counter-scenario does not exist on its own; it is either measured with a control group or constructed with a simulation. If neither is done, all you have is “the result with AI,” and how much of it came from AI cannot be known.

That is why being unable to measure ROI is usually not a lack of metrics but a lack of a comparison basis. The metric exists; but there is nothing to compare it against.

A simulation evaluates the decision before production

Decision simulation fills this missing basis in two directions: estimation beforehand, comparison afterward.

Beforehand: Before the AI system is put into production, it is run on past data and scenarios. “If this system had made these decisions last year, what would it have recommended and what would the result have been?” This allows estimating the expected value of the investment without real money at risk. The board gets, in answer to “what will this earn?”, not a hope but a simulation-based estimate.

Afterward: While the system is in production, the simulation keeps producing a counter-scenario. “This week the AI recommended this decision; without AI, what would the old method have decided and what is the difference?” This makes ROI continuous, concrete and comparable.

Without simulation, AI is a black box: investment goes in, “maybe it worked” comes out. With simulation, AI becomes a measurable decision layer. (↔ 17 success metrics, 10 not a demo but a system)

The hidden cost of “unpredictable” ROI

Being unable to measure ROI has not only a reporting cost but a decision cost.

When ROI cannot be shown, AI investments are left defenceless. At the first budget squeeze, they are cut as the project “whose return we could not prove” — even if it is creating value. Conversely, an AI project that creates no value can continue for months because ROI is not measured; no one has a basis to question it. In both cases, the absence of a measurement basis leads to a wrong decision.

Simulation-based ROI makes these decisions healthy: the value-creating project is protected, the non-value one is stopped early. Because now the decision rests not on a “feeling” but on a comparable reference.

A simulation gives a comparison basis, not certainty

An expectation adjustment is needed: a simulation does not estimate ROI to the penny. A counter-scenario based on past data does not give certainty about the future; human behaviour and external conditions are uncertain.

But the value of a simulation lies not in certainty but in building a comparison basis. Giving an approximate but realistic answer to “what would have happened without AI?” is far more valuable than giving no answer. A simulation turns ROI not into a precise number but into a defensible range and a clear comparison. This is an honest ROI approach, consistent with GDP’s principle of not promising what cannot be proven.

Closing

“We invested in AI but cannot show the ROI” looks like a measurement-tool problem but is not. ROI cannot be measured because there is no comparison basis: because the AI system is put into production without its behaviour being simulated, its value can neither be estimated beforehand nor compared against a counter-scenario afterward.

Decision simulation builds this missing basis: it estimates expected value beforehand by running on past data, and makes ROI continuously comparable afterward with the “what would have happened without AI?” counter-scenario. It gives not certainty but a defensible comparison.

The right question is:

Are we debating which metric to measure AI’s ROI with, or whether we have a reference scenario to compare it against?