How to measure the ROI of AI without fooling yourself

Sooner or later, every AI proposal reaches the same management question:

What will the business get back for the money?

The answer is often less certain than the presentation suggests. A supplier may demonstrate an impressive assistant, an employee may save an hour preparing a report, and an industry study may report a large productivity improvement. None of those facts, on its own, proves a positive return for your organisation.

Return on investment—or ROI—is a comparison between the value an investment creates and its full cost. Measuring it requires more than recording how often employees open an AI tool. Managers need a baseline, a defined business outcome, credible costs and a way to check that speed has not been purchased at the expense of quality or risk.

Productivity is evidence, not a promise

There is credible evidence that generative AI can improve performance on some tasks.

In a study of 5,172 customer-support agents, an AI assistant increased issues resolved per hour by 15% on average. Less-experienced workers received much larger gains than highly experienced workers.[1]

In another experiment, 453 college-educated professionals completing selected writing tasks with ChatGPT finished 40% faster, while independent evaluators rated their output 18% higher in quality.[2]

These results matter, but they are not universal benchmarks. The studies involved particular people, tools, tasks and measurements. Your organisation may have different information, customers, quality standards and implementation costs.

Research also shows why caution is necessary. In an experiment with management consultants, AI improved performance on suitable tasks but made participants more likely to reach an incorrect answer on a task outside the technology’s capabilities.[3]

The right conclusion is neither “AI guarantees productivity” nor “AI cannot be trusted.” It is:

AI’s value is specific to the task and must be tested in the environment where it will be used.

Begin with a baseline

You cannot measure improvement unless you understand the current process.

Suppose a team prepares a weekly management report. Before introducing an AI assistant, record for several reporting cycles:

• How many employee hours are required.
• How long managers wait for the completed report.
• How many corrections are made.
• How often information is missing.
• How satisfied the report’s readers are.
• What software and external support currently cost.

Do not rely only on estimates gathered after the pilot. People may remember the old process as slower or more frustrating once they become accustomed to the new one.

Where practical, compare similar work completed with and without the assistant during the same period. That helps separate the effect of AI from seasonal demand, staffing changes, new policies and other improvements.

Define the result in business language

“Increase AI adoption” is rarely the final business outcome. Useful objectives sound like:

• Reduce the median initial response time for routine enquiries from eight hours to one hour.
• Reduce report-preparation time while maintaining the existing correction rate.
• Increase the number of qualified enquiries handled without lowering customer satisfaction.
• Help new employees reach an agreed quality standard sooner.
• Reduce external transcription costs while meeting accuracy and privacy requirements.

The more specific the outcome, the easier it becomes to decide what should be measured and whether the investment is worthwhile.

Use a complete ROI equation

The familiar formula is:

ROI = (Total measured benefit − total cost) ÷ total cost × 100

If a project produces KSh 1,500,000 in defensible annual benefit and costs KSh 1,000,000 over the same period, the estimated ROI is:

(KSh 1,500,000 − KSh 1,000,000) ÷ KSh 1,000,000 × 100 = 50%

The arithmetic is easy. Deciding what genuinely counts as a benefit or cost is the difficult part.

Count the full cost

The licence fee is only one part of an AI investment.

Initial costs

• Process discovery and project planning.
• Supplier evaluation and contracting.
• Data cleaning and document preparation.
• Configuration or custom development.
• Integration with existing systems.
• Privacy, security and legal review.
• Employee and manager training.
• Time spent testing the system.

Continuing costs

• Software subscriptions or usage charges.
• Human review.
• Maintenance and updates.
• Monitoring quality and incidents.
• Correcting errors and handling complaints.
• Updating approved information.
• Retraining employees.
• Internal support and supplier management.

Often-forgotten costs

• Employee time spent learning a poorly designed workflow.
• Duplicate work during the transition.
• Reduced performance while processes change.
• Additional approval layers.
• Security or data-protection controls.
• Dependence on a supplier that may raise prices or change its product.
• The cost of leaving or replacing the system.

If the business counts every minute saved as a benefit but ignores the time spent checking output, the calculation is biased before the pilot begins.

Treat time savings carefully

An employee who saves five hours a week does not automatically reduce payroll by five hours. The business receives financial value only if the released capacity is used.

Time savings can create value when they allow the organisation to:

• Serve more customers with the same team.
• Avoid overtime or temporary labour.
• Reduce dependence on an external supplier.
• Fill a vacancy more slowly or avoid an additional hire.
• Complete revenue-generating work sooner.
• Improve service in a way that increases retention or sales.
• Redirect employees to valuable work that was previously neglected.

For planning, apply a conservative realisation factor: the proportion of saved time that the business reasonably expects to convert into useful capacity.

Annual capacity value = hours saved × loaded hourly cost × realisation factor

The loaded hourly cost should reflect salary plus relevant employment costs. The realisation factor is a management assumption, not an accounting rule. State it clearly and test the result using a lower and higher value.

For example, if verified savings equal 400 hours a year, the loaded cost is KSh 1,500 per hour and management expects to use 60% of the released capacity productively:

400 × KSh 1,500 × 60% = KSh 360,000 in estimated annual capacity value

This is more defensible than claiming the full KSh 600,000 as a cash saving.

Measure quality beside speed

An assistant may reduce drafting time while increasing checking, corrections or reputational risk. Pair every speed or volume measure with a quality measure.

Where consequences are significant, include human review and incident measures even if they make the apparent productivity gain smaller. A smaller honest return is more useful than a larger fictional one.

A worked example: customer-enquiry support

Consider a hypothetical service business receiving 2,000 routine enquiries a month. Employees currently spend an average of six minutes classifying each enquiry, finding approved information and preparing an initial response.

During an eight-week pilot, an AI assistant prepares a classification and draft reply for employee approval. The measured average handling time falls from six minutes to four minutes, including review and correction.

Measured operational change

2 minutes saved × 2,000 enquiries × 12 months = 48,000 minutes
48,000 minutes ÷ 60 = 800 hours saved per year

Assume the loaded employee cost is KSh 1,200 an hour and management expects to convert 70% of the released time into useful customer work.

800 × KSh 1,200 × 70% = KSh 672,000 annual capacity value

Suppose additional defensible benefits include KSh 180,000 in avoided overtime, producing total estimated annual benefit of KSh 852,000.

Full annual cost

Illustrative ROI

(KSh 852,000 − KSh 620,000) ÷ KSh 620,000 × 100 = 37.4%

THE WORKED EXAMPLE AT A GLANCE

Benefit is not return until the full cost is removed.

Estimated annual benefitKSh 852,000

Full annual costKSh 620,000

Estimated net benefitKSh 232,000

ILLUSTRATIVE ROI37.4%Net benefit ÷ full annual cost

This example is a teaching illustration, not a market benchmark. A real decision would also check that accuracy and customer satisfaction remained acceptable. If complaints increased or employees could not use the saved capacity, the estimated return would fall.

Add uncertainty instead of hiding it

Managers should calculate at least three scenarios.

If the project only makes financial sense in the optimistic case, it may not be ready for expansion.

Also separate measured facts from assumptions. For example:

• Measured: Average handling time fell by two minutes during the pilot.
• Assumed: 70% of that time can be redirected to useful work.
• Estimated: Annual capacity value is KSh 672,000.

This small discipline makes an AI business case much easier to challenge and improve.

Include benefits that are real but difficult to price

Not every valuable outcome has an honest monetary figure.

Potential non-financial benefits include:

• More consistent customer communication.
• Faster access to internal knowledge.
• Reduced frustration from repetitive work.
• Better support for new employees.
• Improved service availability.
• Greater capacity during busy periods.
• New services the company could not previously offer.

Record these outcomes, but do not convert them into money without a defensible method. Customer satisfaction may eventually affect retention, for example, but claiming revenue before that relationship is observed would double-count hope as benefit.

Use a balanced pilot scorecard

A useful scorecard covers five areas.

1. Business outcome

• Did waiting time, throughput or cost improve?
• Did the improvement matter commercially?

2. Quality

• Did errors, corrections or complaints change?
• Did the result meet the existing service standard?

3. People

• Did employees save time after checking the output?
• Could they identify errors and escalate uncertain cases?
• Did workload or work intensity improve or worsen?

4. Risk

• Were there privacy, security, bias or compliance incidents?
• Did the system remain within its approved purpose?

5. Economics

• What benefits were measured?
• Which benefits remain assumptions?
• What was the complete cost?
• How sensitive is ROI to different assumptions?

This prevents a project from being declared successful solely because employees used the tool or a demonstration looked impressive.

Decide before the pilot what happens afterwards

Agree on decision rules in advance.

Expand when the benefits are repeatable, quality remains acceptable, employees can operate the process and risks are controlled.

Revise when the use case is promising but the workflow, training, information or controls require improvement.

Restrict when AI is useful for a narrower part of the task but unreliable elsewhere.

Stop when the benefits do not justify the cost, the risk is unacceptable or a simpler solution would perform better.

Stopping is not evidence that the organisation “failed at AI.” A controlled pilot has done its job if it prevents a weak investment from becoming a large one.

Questions a leadership team should ask

Before approving the business case, ask:

• Which result are we buying?
• How does the process perform today?
• Which benefits were measured and which were assumed?
• Have we counted checking, training and maintenance?
• What happens to the time employees save?
• Did quality improve, remain stable or decline?
• Are some employees benefiting more than others?
• What risks could turn the expected gain into a loss?
• Does the case still work under conservative assumptions?
• Is AI better than improving the process or using conventional software?

The purpose of ROI analysis is not to prove that an AI proposal is good. It is to help management discover whether it is good.

AI can create meaningful value, but credible returns are earned through careful task selection, realistic costing, human oversight and measurement. The organisations most likely to benefit will not be those with the boldest productivity claims. They will be those willing to measure what changed—and equally willing to notice what did not.

If the numbers justify a pilot, the people still determine whether it works in practice. Read how to introduce AI into your business without disrupting your team.

Research and helpful links

1. Erik Brynjolfsson, Danielle Li and Lindsey R. Raymond, “Generative AI at Work,” The Quarterly Journal of Economics, 2025. The study involved 5,172 customer-support agents and measured issues resolved per hour; its results are context-specific.
2. Shakked Noy and Whitney Zhang, “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence,” Science, 2023. The experiment involved 453 college-educated professionals completing selected writing assignments.
3. Fabrizio Dell’Acqua et al., “Navigating the Jagged Technological Frontier,” Organization Science, published online in 2026. The study shows both gains and performance risks across different tasks.
4. OECD, “Generative AI and the SME Workforce: New Survey Evidence,” 2025. The OECD notes that reported benefits do not reveal their magnitude and that survey estimates vary by method and country.
5. National Institute of Standards and Technology, “Artificial Intelligence Risk Management Framework,” 2023, and “Generative Artificial Intelligence Profile,” 2024.

Research checked on 12 September 2026. The worked figures are illustrative rather than a market benchmark; use your organisation's measured costs and outcomes for investment decisions.