AI implementation: getting from pilot to production

The pilot worked. It answered questions well in a demo, the steering group was impressed, and a budget line appeared for the next phase. Months later the system is still not part of daily work, and nobody can say exactly why. This pattern is common, and the cause is rarely the model.

AI implementation is the work of taking an AI capability from a promising pilot to a system that people use every day, inside the tools they already work in. It covers the data the system depends on, how its output is evaluated, how it is integrated, governed and monitored, and who owns it after go-live.

This guide is for the person who has a pilot that works and a mandate to make it real. It sets out what implementation covers, why pilots stall, the seven stages from pilot to production, who should own each part, what drives cost and time, and what the first ninety days after go-live should look like.

It picks up where strategy ends. If you are still choosing which use case to build first, start with our guide to AI implementation strategy, and check what should already be in place with the AI readiness checklist.

Shipshape Data builds data foundations and the AI that runs on them. We see the same pattern in many of the pilots we are asked to review: the model was fine, and the work around it was never planned.

What AI implementation covers

Implementation starts once a use case has been chosen and a pilot has shown it can work. It ends when the system runs inside daily work, with a named owner, a measured outcome and a plan for keeping it accurate. Everything between those two points is implementation. The choice of use case and the business case sit earlier, in the strategy work.

In practice the work covers six areas: the data the system reads, the way its output is tested, the integration into existing tools, the governance around access and decisions, the monitoring once it is live, and ownership after go-live. A pilot usually touches one or two of these. Production needs all six.

Implementation is not limited to one kind of system. It can mean a retrieval assistant over company documents, a classifier that routes tickets, a forecasting model, or an agent that completes a workflow. Some implementations sit alongside rule-based automation such as robotic process automation (RPA): the AI reads, classifies or drafts, and a fixed rule or workflow carries out the next step. The stages below apply to all of them.

Why AI pilots stall before production

A pilot is built to answer one question: can this work? It is usually run on a clean extract of data, by a small team, in front of a forgiving audience. Production asks different questions, and the gap between the two is where most programmes stop. We cover the wider causes in why AI projects fail. At the pilot stage, four causes come up again and again.

The data was never made ready

The pilot ran on a spreadsheet export or a hand-cleaned sample. Production needs a live feed, refreshed on a schedule, with the same definitions every time. When the live data arrives with missing fields, renamed columns or a different meaning for a familiar term, the system that worked in the demo starts to give wrong answers. AI works like a magnifying glass: over a strong data foundation it amplifies the gains, and over a weak one it amplifies the errors.

Integration was left until last

A pilot often lives in its own interface. People are asked to open a new tab, copy text in and paste the answer out. Use falls away because the tool sits outside the work. Integration into the CRM, the ticketing system or the document store is often the largest part of the build, and it is rarely in the pilot plan.

Nobody owns it after the demo

The pilot team moves on to the next idea. There is no named person responsible for the system's accuracy, its running cost or its users. When something goes wrong, nobody is sure who should fix it, so nothing is fixed and trust falls.

Nobody agreed what good looks like

The pilot was judged on impressions. Without a written definition of a correct answer and a test set to measure against, nobody can show that the system is ready, and nobody can show whether a change has made it better or worse. Sign-off stalls because there is no evidence to sign.

The seven stages from pilot to production

Each stage below has a clear output and a common way to fail. The order matters. A later stage cannot make up for an earlier one that was skipped.

Stage one: agree the outcome and metric

Write down the business result the system is meant to change and the metric that will show it. Examples are time spent on a task, the share of tickets resolved without escalation, or forecast error. Agree the baseline before anything is built. The common failure is a vague goal such as 'use AI in support', which cannot be measured and so cannot be defended when budgets are reviewed.

Stage two: put data contracts in place

A data contract is a written agreement between the team that produces a dataset and the teams that use it. It sets out the schema, the meaning of each field, the freshness expected and who to contact when it breaks. Our guide to data contracts explains the format. For AI, the contract stops a quiet change in a source system from turning into wrong answers in front of users. The common failure is building on a one-off extract that nobody has committed to keeping current.

Stage three: build an evaluation set

Collect real examples with known correct answers, drawn from the cases the system will meet in production, including the awkward ones. Decide how each output will be scored and what score is good enough to go live. Run the set every time the prompt, the model or the data changes. Our guide on how to evaluate large language models covers the methods. The common failure is testing on the same handful of questions used in the demo.

A good measure also scopes the claim honestly. For 1NCE, the outcome was stated against a human benchmark: 60% of tickets handled to a human standard. A measure framed that way tells everyone what the system does and what it leaves to people.

Stage four: integrate into existing tools

Put the output where people already work: a suggested reply inside the ticketing system, a summary attached to the record in the CRM, a flag in the existing dashboard. Plan the authentication, the error handling and what the user sees when the system is unsure. The common failure is a separate application that only the pilot team opens.

Stage five: govern access and decisions

Decide what data the system may read, on whose behalf, and which decisions it may take without a person. Record which model version produced which output. Where models need to reach governed data, a controlled route such as Model Context Protocol (MCP) keeps access rules and an audit trail in one place, rather than copying data somewhere ungoverned. Our AI governance framework guide sets out the wider controls. The common failure is governance arriving as a late review that blocks go-live.

Stage six: deploy and monitor

Move the system into a managed environment with versioning, rollback and alerts. Our explainer on model deployment covers the release patterns, and MLOps architecture covers the components around them. Then watch the inputs, outputs, cost and response time, and rerun the evaluation set on a schedule. Model monitoring in production explains which signals to track. The common failure is treating go-live as the finish line.

Stage seven: hand over ownership

Name the business owner, the technical owner and the support route before launch. Write down how changes are requested, tested and released. The common failure is an implicit handover, where the build team leaves and the system slowly decays.

A pilot proves that something can work. Production proves that it does work, every day, for people who did not build it.

Roles and who owns what

Clear ownership is the stage most often missing. In a smaller organisation these roles do not need to be separate people, but each one needs a name against it.

• Business owner: accountable for the outcome and the metric, and decides whether the system stays, changes or is switched off.
• Delivery lead: runs the backlog, gathers feedback from users and plans changes.
• Data owner: responsible for the source data and its contract, including freshness and definitions.
• Engineering lead: owns the integration, deployment, security and release process.
• Evaluation lead: maintains the test set, scores each release and signs off quality. This is often a subject expert rather than an engineer.
• Governance and risk: approves data access, records decisions and checks the system against policy and UK GDPR.

An external partner can fill some of these roles during the build. The arrangement that lasts is one where your team works alongside the partner and owns each role by the end. If the partner holds every role, the system leaves when they do.

Cost and timeline drivers

There is no standard price or duration for AI implementation, because the work depends on what is already in place. The drivers below explain most of the difference between a short engagement and a long one.

• The state of the data: a governed warehouse with agreed definitions shortens every later stage, while scattered sources add foundation work first.
• The number of integrations: each system the output must reach adds build, testing and security review.
• The evaluation effort: subject experts need time to label examples and agree what a correct answer is.
• The level of risk: decisions that affect customers, money or regulated activity need more governance and more human review.
• Running costs: model usage, hosting and monitoring continue after launch and grow with volume.
• Change and training: people need time to learn a new step in their work, and the plan must include it.

The most economical route is usually to scope one use case on a foundation that will carry the next. A first win that pays for the second keeps the programme funded. Where the foundation is not ready, the honest answer is to fix it first, and sometimes that means saying no to the AI build for now.

The order of building decides whether the first win pays for the second.

The first ninety days after go-live

Go-live is the start of the evidence, not the end of the work. The first ninety days decide whether the system earns trust or is quietly abandoned.

Days one to thirty: watch closely

Review a sample of outputs every day. Track use, error reports and response times against the evaluation results from before launch. Fix integration problems quickly, because early friction sets the habits users keep.

Days thirty to sixty: measure the outcome

Compare the business metric against the baseline agreed in stage one. Add the hard cases users have found to the evaluation set. Check that monitoring alerts reach a named person and that the person acts on them.

Days sixty to ninety: decide what is next

Report the result to the business owner in the terms agreed at the start. Confirm the support model and the release process. Then decide, on evidence, whether to extend the system, improve it or stop. A clear result at this point is what funds the next use case.

Where this leaves you

A working pilot is a useful result, but it is not yet a system. The distance between the two is made of data contracts, evaluation, integration, governance, monitoring and ownership. Each one can be planned, and each one fails when it is left until last.

If you are about to move a pilot forward, check each of the seven stages against what you have now. Where a stage is empty, that is the work. If you want a structured starting point first, the free AI readiness assessment scores strategy, people, data and change on the spot, with no email gate.

If you need a sequenced plan for moving a pilot into production, our AI strategy service assesses readiness and builds the roadmap with the team that will deliver it. If the plan is clear and you need the build, AI development covers the data contracts, evaluation, integration and monitoring described here, on governed data, and we stay past go-live. Talk to us about the pilot you have, and we will tell you honestly what stands between it and production.

Frequently asked questions