The pattern
Every organisation we talk to has an AI pilot somewhere. A team built a proof of concept on a copy of some data, showed it to a few people, and got a warm reception. That was three, six, nine months ago. It still works. It is still not in production. Nobody has cancelled it, because nothing went wrong — but nothing is going right either.
This is the most common state of enterprise AI today, and it is worth understanding why, because the failure is almost never the one people expect. The model was fine. The demo was fine. The pilot stalled on three things that were decided, by default, before the first line of code: where the data came from, where the output was supposed to go, and what “working” meant.
Data access, not data quality
Ask why a pilot hasn’t shipped and the first answer is usually “data quality”. Dig in and it is rarely quality — it is access. The pilot ran on an extract someone pulled into a notebook. Production needs a governed connection to the warehouse or the document store, with the access controls, lineage, and audit that the real system enforces. The extract had none of that, which is exactly why it was fast to build with.
The gap between “a CSV a colleague sent me” and “a service account with row-level permissions reviewed by security” is where most of the calendar time goes. It is also entirely predictable. If nobody names the production data source in week one, the pilot is being built against something that cannot ship.
Integration is the product
The second stall point is where the output lands. A pilot produces an answer in a chat window or a dashboard nobody has to look at. Production means the answer arrives inside the system where the decision is actually made: a field in the CRM, a recommendation in the case-management queue, a draft in the document workflow, a flag in the claims system.
That integration is not a finishing step. It is most of the work, and it determines the architecture. A model that has to write back into ServiceNow, respect SAP’s approval chain, or attach itself to a Salesforce record has constraints that a chat window never surfaces. When the integration is designed last, it forces a redesign of everything built before it — and the redesign is what quietly kills momentum.
The organisations whose pilots reach production tend to invert the order. They decide the landing system first, build the thinnest possible integration to it, and only then improve the model behind it. The first version is often embarrassingly simple. It is also in production, being measured, three months before the sophisticated version would have been.
Nobody agreed what “done” meant
The third stall is the least technical and the most fatal. The pilot was judged on whether it was impressive. Production has to be judged on whether it is better — and better than what, by how much, measured how, has usually never been written down.
Without an agreed evaluation, every stakeholder applies their own. The risk team finds one bad output and stops the rollout. The sponsor points to the demo and pushes on. The engineers, caught between them, keep tuning. The pilot is neither approved nor rejected; it simply ages.
An evaluation set — a few hundred real cases with agreed correct outcomes, and a pass mark decided before the results are in — ends that argument before it starts. It is unglamorous work. It is also the single strongest predictor we see of whether a pilot ships.
Three questions for week one
None of this requires a bigger budget or a better model. It requires three questions to be answered before the pilot begins, and the honest answers to be written down:
- Who owns the production data source, and have they agreed to the access? If the answer is “we’ll use an extract for now,” the pilot is a demo.
- Which system will the output land in, and who owns that system? If the answer is “a dashboard,” ask who will change a decision because of it.
- What is the pass mark, and who signs off that it was met? If the answer is “we’ll know it when we see it,” you will not.
A pilot that can answer all three is not guaranteed to succeed. But it is the only kind that can finish — and finishing, either way, is the point. A clear no in six weeks is worth far more than a maybe that lasts a year.
