AI initiatives are often highly visible in organisations.

This is happening due to several reasons:

AI initiatives are high risk, and therefore generally require more reporting and scrutiny.

The Board tends to be more interested in AI initiatives, given the significant attention that AI currently receives in the press.

AI work is cross-functional and likely to involve several business functions and operating divisions.

Despite this level of attention and reporting, many organisations struggle to understand whether their AI-enabled strategies are actually delivering value. Performance measurement becomes more complex, not clearer, when traditional KPIs are applied without adjustment.

 

Why AI breaks conventional performance metrics

Traditional KPIs were designed for stable systems.

They assume:
• Predictable workflows
• Clear cause-and-effect relationships
• Lagging indicators of performance

AI disrupts these assumptions. It introduces adaptive systems that evolve over time and influence decisions in non-linear ways.

Gartner’s 2025 research on AI value realisation highlights that many organisations rely on metrics that track activity rather than outcomes, making it difficult to assess whether AI is improving execution (Gartner, 2025).

Measurement frameworks lag behind capability.

 

Activity metrics create a false sense of progress

Common AI metrics focus on:
• Number of use cases deployed
• Model accuracy
• Tool adoption rates

These indicators are useful, but they do not explain whether strategy execution has improved.
McKinsey’s 2025 research on AI at scale shows that organisations tracking AI activity without linking it to business outcomes often overestimate progress and underestimate execution risk (McKinsey, 2025).

Activity is not impact.

 

Execution-focused measurement requires outcome clarity

Effective performance measurement begins with clarity on business outcomes.

I always encourage my clients to take a step back and look at the strategic problem they are solving.

This means asking questions such as:
• What are the tangible business outcomes we want to change?
• What customer pain points should improve?
• What cycle times should reduce?
• What risks should be mitigated?
• What value should be realised?

MIT Sloan research emphasises that more sophisticated organisations benchmark AI tools on operational outcomes, rather than generic model benchmarks. (MIT Sloan, 2025).

This means that AI-enabled strategies require metrics that reflect how work and decisions have changed, not just whether tools are in use, or the publicly reported performance benchmarks of these models.

 

Leading indicators matter more than lagging ones

As with all strategy execution and digital transformation programs, AI strategies benefit from leading indicators.

Leading indicators signal whether execution is on track before results appear. They include:
• Decision turnaround times
• Adoption confidence
• Quality of human-AI collaboration
• Frequency of learning loops
• Specific behavioural change and work practices measures

For example, when an organisation implements a new AI-augmented Marketing workflow, they may look at specific work practices such as the proportion of published images used that were generated by AI. This is an excellent measure as it indicates not only actual adoption of the tools, but usage for specific purposes.

These measures allow organisations to adjust execution early rather than react late.

 

Governance keeps measurement meaningful

Performance metrics only matter if they influence decisions.

Governance ensures that:
• Metrics are reviewed consistently
• Insights lead to action
• Measures evolve as strategies evolve

Without governance, dashboards become reporting artefacts rather than execution tools.
If dashboards are used to justify continued use of AI tools, more extensive rollout of a particular AI-augmented workflow or further investment in AI, then the metrics matter.

Strong governance links measurement to accountability and learning for further initiatives.

 

Avoiding metric overload

AI makes it easy to measure everything.

This creates noise.

Effective organisations are selective. They prioritise a small number of meaningful measures aligned to strategic intent.

I encourage fewer measures to make it easier to track, and ensure that the measures are used to influence further investment decisions. As a broad guideline, 5 KPIs should be enough for one workflow.

Where my clients are tracking too many KPIs, it can take longer to generate and analyse the data than is justified by the decision they are making. On the other hand, focus ensures the KPIs are more likely to be understood and used in practice.

Focus enables action.

 

From reporting to execution insight

AI changes what can be measured. It does not automatically clarify what should be measured.

Execution improves when performance measurement:
• Reflects real changes in work and decisions
• Focuses on outcomes rather than activity
• Supports learning and adaptation
• Is reinforced through governance

Strategy execution depends on seeing reality clearly. Measurement frameworks must evolve alongside AI capability to make that possible.

PS More Ways to Accelerate your Strategy

Looking to move your strategy forward? Lisa Carlin works with selected clients directly to go from strategy to plan to result. Email Lisa.

A bespoke in-house Bootcamp is the fastest way to learn. Get the skills your teams need to go from Strategy to Plan to Results. AI optional. Bootcamp info here.

Subscribe here to ensure you don’t miss this weekly newsletter.

Connect with Lisa on LinkedIn.