Published on :
October 16, 2026
by
Anisha Bhattacharjee
Views...
If your team has run an AI pilot, or sat through a convincing demo of one, you may recognise what can come next: a promising result, a motivated team, and then the benefits you hoped for still not showing up in daily work. It can be hard to tell what the AI changes for the team and what it takes to keep that change going.
In short, we think a pilot is more useful when it tests a defined operational decision, measures what changes, involves the people who will use the output and establishes what scaling would require.
Evidence suggests it does not. PwC’s 2026 AI Performance Study, Want ROI from AI? Go for growth, surveyed 1,217 senior executives across 25 sectors, primarily at large, publicly listed companies, and found that the top 20% of organisations captured 74% of AI-driven economic returns. The study focuses on how organisations can improve AI returns through growth, rather than on why individual pilots stall. However, PwC also reported that leading organisations were twice as likely to have redesigned workflows to incorporate AI. That is relevant to facilities management, where an alert or recommendation only matters if someone acts on it and the work genuinely improves.
That difference is what we mean by the AI pilot penalty: the gap between running AI and actually getting value from it. It is a term we have heard in industry conversations, and one we use ourselves. In The AI Pilot Penalty in FM, we look at how this gap was addressed in a live deployment. Here, we focus on the earlier question: what can be put in place before and during a pilot so that the decision about what comes next is better informed?
A demo shows what a tool can do, usually with prepared examples, and it is meant to be a clean, controlled look at the capability. A pilot typically goes a step further: the tool is tested against operational data and workflows within a defined part of the operation, over an agreed period. Both do their job when they show that something works.
The next step is a bigger one. Across a whole operation there are more sites and asset types, more systems feeding in information, more exceptions nobody planned for, and more people who need to use the output while getting on with the rest of their day. So a pilot that works well can still leave open what it will deliver once it is part of everyday work.
A pilot cannot reproduce every part of a full rollout, but it can show a lot about what a wider rollout will need. These are suggestions from our experience, not a formula, and not all of them will apply to every pilot.
Start with the outcome you are hoping for, how the work is handled today as a starting point, and the improvement that would justify going ahead. It also helps to settle who owns the outcome and decides whether to go ahead, and how the result will be checked and over what period.
The outcome you want and the evidence that justifies scaling are related, but they are not the same thing. A pilot might show that the AI prioritises work orders accurately without showing that this reduces delays or maintenance effort. Or it might produce an operational benefit that depends on a level of manual support that would not be practical to sustain. Deciding in advance what evidence would count helps separate the two.
Then look at what scaling would take: the timeline, the people and effort involved, the ongoing cost, and any connections to existing systems or changes to how people work. It helps to decide in advance what would justify scaling, extending the pilot or stopping. A pilot that can end in “not yet” or “no” is one whose “yes” can be trusted, and agreeing this before the results arrive keeps the conclusion from being shaped by them. It also gives you something to track: PwC found that its leading organisations were 80% more likely to systematically track the business impact of their AI initiatives.
In a feature-led pilot, success means the tool does what it was built to do. In a decision-led pilot, success means a specific decision improves in a way you can describe in advance. If the decision is which open work orders to handle first, for example, the pilot might look at whether priorities are set more consistently, urgent work is spotted earlier, or less time goes into gathering information, and whether the suggested priority holds up against the decision the team would actually make, not just whether it looks plausible. Which of these matters depends on the use case, and in FM a slower decision backed by stronger evidence can be better than a faster one.
It is also worth asking what existing work the tool could reduce, not only what it adds: time spent sorting alerts, coordinating between systems, or investigating a fault without the full history. These are hypotheses that can be tested. If the expectation is that engineers will spend less time sorting alerts, record that effort today and compare it during the pilot.
The people who run the operation are the ones who will use the output day to day, which makes them well placed to judge whether it is helping. We think they deserve a strong voice from the start, working alongside IT and information security teams, who help make sound technology choices. In practice, that means involving them in what the output should look like and how a recommendation is explained, so they can see the reasoning and accept it, question it or override it with confidence. PwC reported that employees at its leading organisations were 2.1 times as likely to trust AI-generated insights and act on them, and it points to involving the people who use the tools as one of the things that builds that trust. Trust takes time and an early wrong output can cost a lot of it, so it helps to be open from day one about what the system will and will not get right.
A closed work order records that a job was reported as done, which is useful, but it is different from confirming the problem is resolved. Checking, for instance, whether an asset is back to its expected condition needs live work over a period of time, which a well-designed pilot can offer and a demo cannot. It also helps to watch how often recommendations are used, questioned or overridden, and why, since an accepted recommendation is not necessarily a correct one. We explore the distinction in FM KPIs Measure Activity. Outcome Verification Measures Whether It Worked.
Data does not need to be perfect before starting. The question is which gaps affect the decision being tested and how uncertainty will be handled. Your Facilities Data Isn’t Perfect. Start Using It Anyway covers the data side in more depth. A separate point is how much manual effort supported the pilot, such as a small team cleaning data or resolving exceptions in the background. It is worth asking whether that effort could continue at a larger scale, and what would need to change if it could not.
In facilities management, the challenge is often not getting AI to produce an output. It is connecting operational context, the decision, the action and the check on the result across the systems a team already uses. That is the problem Xempla’s System of Decisions is built around, as a layer above existing CMMS, BMS and IoT tools and not a replacement for them. The case study in The AI Pilot Penalty in FM describes how workflows changed alongside the reported results across roughly 2,000 distributed solar sites, including earlier identification of issues, assessing alerts before deciding whether anyone needs to go to site, and verifying the result after work was completed. Those are results from one deployment, not what any organisation should expect. Keeping a change in place after go-live is a separate subject, which we cover in Why Digital Transformation Stalls in Facilities Management.
A demo shows what a tool can do, usually with prepared examples in a controlled setting. A pilot typically tests the tool against operational data and workflows in a defined part of the operation, over an agreed period, so you can see what it delivers in practice.
A proof of concept shows that something can work. A proof of value looks at what it delivers for the operation once it is in use: the result, the work it reduces and whether that holds up day to day.
There is rarely a single cause. A pilot is designed to show what a technology can do, so questions about who acts on the output, what it changes and what scaling requires can remain open when it ends. Agreeing them before the pilot starts can make the next step clearer.
Not necessarily. The important question is whether the available data is reliable enough for the decision being tested. Identify the gaps that could affect the output, establish how uncertainty will be handled, and decide which data issues need to be addressed before wider deployment. See Your Facilities Data Isn’t Perfect. Start Using It Anyway for more on working with imperfect facilities data.
© Xempla. All rights reserved.