Why can’t you measure the ROI of your AI investment? Because you are not simulating it
Companies are investing in artificial intelligence but cannot demonstrate the return. This appears to be a measurement-tool problem, “we are not tracking the right metrics”, but the deeper issue is that there is no basis for comparison against which ROI can be measured. The AI system is put into production without first simulating what it would do in a real context. As a result, its value can neither be estimated beforehand nor compared with a reference point afterwards.
The problem is widespread and documented. According to a 2025 Forbes survey of more than 1,000 senior executives, a significant share identified measuring AI’s ROI and business impact as one of their greatest challenges. The interesting point is that the same companies can track AI outputs, such as improved decisions or operational efficiency, but cannot connect those outcomes to a reference value. The metrics exist; the comparison does not.
In brief
- A significant share of executives see measuring AI ROI and business impact as one of their greatest challenges (Forbes, 2025 executive survey).
- ROI is a comparison: the difference between the “outcome with AI” and “what would have happened without AI”, the counterfactual.
- A counterfactual does not appear automatically. It must be measured through a control group or constructed through simulation. Without either, ROI remains a matter of perception.
- Simulation builds this foundation in two directions: it estimates expected value before deployment and compares actual results with a counterfactual afterwards.
Why is “sales increased after AI” not proof of ROI?
Measuring the return on an AI investment is not a matter of looking at one outcome in isolation. “Sales increased after AI was introduced” is not proof of ROI, because sales may have increased for other reasons. True ROI is the difference between the outcome with AI and the outcome without AI.
That difference requires a counterfactual: What would have happened if AI had not been used? This is where many companies struggle. A counterfactual does not exist by itself; it must either be measured through a control group or constructed using a simulation. If neither is done, the organization sees only the “AI outcome” and cannot determine how much of it was caused by AI. Failing to measure ROI is therefore often not a shortage of metrics, but a shortage of a valid comparison basis.
There are three ways to establish a reference point:
| Approach | How it works | Result |
|---|---|---|
| Nothing, the default | Only the “outcome with AI” is observed | ROI cannot be measured and remains an impression |
| Control group (A/B) | One group operates with AI and another without it | Strong for operations that can be divided into comparable groups |
| Simulation | Historical data is used to construct “what would have happened without AI” | Applicable to individual decisions that cannot be split into test and control groups |
Simulation evaluates the decision before production
Decision simulation fills this missing foundation in two directions: estimation before deployment and comparison afterwards.
Before deployment, the AI system is run against historical data and scenarios. “If this system had made these decisions last year, what would it have recommended and what would the result have been?” This makes it possible to estimate expected value without risking real money. The board can answer “What will this create?” with a simulation-based estimate rather than a promise.
After deployment, simulation continues to produce a counterfactual. “The AI recommended this decision this week; what would the old method have decided without AI, and what is the difference?” This makes ROI continuous, tangible, and comparable. Without simulation, AI remains a black box: investment goes in and “perhaps it worked” comes out.
The hidden cost of “unmeasurable” ROI
The inability to measure ROI is not only a reporting problem; it creates a decision cost. That cost is increasing. According to S&P Global research, the proportion of companies abandoning most of their AI projects rose to 42% in 2025, from 17% the previous year. The most commonly cited reasons were total cost and uncertain value. This points in the same direction as MIT’s finding that the large majority of enterprise GenAI pilots fail to produce measurable business impact: when value cannot be demonstrated, the investment is abandoned.
When ROI cannot be shown, AI investments become vulnerable. At the first budget squeeze, even a valuable project may be cut because “we cannot prove the return.” Conversely, a project that creates no value may continue for months because no one has a basis for challenging it. In both cases, the absence of a measurement foundation leads to the wrong decision.
The problem is not only time; it is the foundation
One objection is valid: AI returns take time. According to Deloitte’s 2025 research, most organizations see ROI from a typical AI use case only after two to four years, much longer than the 7–12 month payback period associated with conventional technology investments. But time is not the only problem.
According to World Economic Forum assessments, AI’s economic impact is increasingly tied not merely to task speed, but to decision quality: faster, better-informed, and more consistent decisions. The difficulty is that decision quality has historically been one of the hardest variables to measure. Traditional ROI frameworks were built for labor productivity and cost reduction, not for capturing decision quality. This is precisely where simulation becomes useful: it compares the AI-assisted and non-AI versions of a decision and makes the difference in decision quality visible.
Simulation provides a basis for comparison, not certainty
Expectations should be set correctly: simulation will not estimate ROI to the last cent. A counterfactual based on historical data cannot provide certainty about the future; human behavior and external conditions remain uncertain.
The value of simulation is not perfect certainty, but a defensible comparison. An approximate yet realistic answer to “What would have happened without AI?” is far more useful than no answer at all. Simulation converts ROI from one supposedly precise number into a defensible range and a clear comparison. It demonstrates comparable value instead of promising value that cannot be proven.
How does GDP approach it?
Within GDP’s Decision Intelligence approach, an AI investment is placed on a measurable ROI foundation. Before production, the system is run on historical data to estimate expected value. After production, a measurement mechanism generates the counterfactual, “What would have happened without AI?”, and links that comparison to a metric management can monitor. AI therefore stops being a black box and becomes a decision layer whose value remains continuously visible.
Frequently asked questions
Why can’t companies measure AI ROI?
The problem is often not missing metrics, but a missing basis for comparison. ROI requires a comparison between the outcome with AI and the outcome without AI. Without a counterfactual, that comparison cannot be made and ROI remains an impression.
What exactly is a counterfactual?
It is the answer to “What would have happened without AI?” It provides the reference point needed to isolate the value created by AI. It can be measured through a control group or constructed as a simulation using historical data.
How does simulation make ROI measurable?
In two ways: before deployment, it runs AI against historical data to estimate expected value; after deployment, it continuously produces the “without AI” counterfactual. AI therefore becomes a measurable decision layer rather than a black box.
Doesn’t AI simply take a long time to generate returns?
It does. Deloitte reports that many organizations see ROI in two to four years. But time is not the only issue: value is increasingly created through decision quality, which traditional ROI frameworks struggle to measure. Simulation closes that gap by making the difference in decision quality visible.
What is the concrete cost of failing to measure ROI?
Wrong decisions. Valuable projects whose returns cannot be demonstrated are cut during budget pressure, while projects that create no value continue because there is no basis for challenging them. According to S&P Global, the AI project abandonment rate reached 42% in 2025, with uncertain value among the main reasons.
Academic and institutional sources: Deloitte 2025, ROI from a typical AI use case may take 2–4 years; S&P Global, the AI project abandonment rate reached 42% in 2025, up from 17% the previous year; MIT, the large majority of enterprise GenAI pilots fail to produce measurable business impact; World Economic Forum, value is shifting from task speed toward decision quality.
Industry and practitioner sources: Forbes 2025 executive survey, measuring ROI and business impact is seen as one of the greatest challenges.
Last reviewed: July 2026.