MELLORCATalk to Mellorca
Patterns

AI, Automation & Systems Readiness

Why AI Pilots Keep Failing to Become Operations

Impressive demonstrations rarely turn into daily practice. The gap is process, ownership, data and rules, not the tool. Here is how to close it.

Mellorca Patterns2026-10-11

Observable condition

The pilot worked. In a workshop, a small team showed an AI assistant drafting customer replies, summarising supplier contracts or classifying invoices, and everybody in the room was impressed. Six months later the pilot is still described as "promising". It is used by two enthusiasts, nothing in the daily routine depends on it, and the next pilot, with a different tool, is already being planned.

This is not a story of failure in the usual sense. No money was lost dramatically, nobody was blamed, and the demonstrations were real. Nothing simply happened afterwards. The business has run several pilots and has no AI-assisted process that it could not do without.

An illustrative example: a Johannesburg insurance broker tests an AI tool for summarising claim documents. Results are good on the sample. When the team tries to use it for live claims, it turns out the documents arrive by email in six formats, nobody is sure whether client details may be sent to an outside service, the output has to be re-typed into the claims system, and no one owns the checking of its work. The pilot is quietly shelved.

Realisation

The realisation is that a pilot proves a capability, whereas operations need a process. A pilot asks whether the tool can do the task. Operations ask who triggers the task, where the inputs come from, who checks the output, where the result goes, what happens when it is wrong and who is accountable. Most pilots answer the first question and never reach the rest.

IBM's overview of machine learning operations describes the work needed to take a model from experiment into dependable use: deployment, monitoring, versioning and ongoing maintenance. The same logic applies to simpler tools bought as subscriptions. The demonstration is the short part; the surrounding discipline is the long part.

Realisation prompt: list the AI pilots your business has run. For each, name the person who owns it today, the process it is part of and the measurement that would show it is working.

Identification

This is a pattern of pilot-stage stagnation. The visible symptom is a growing list of experiments with no embedded results. The common assumption is that the technology is not ready, or that staff resist change. The structural cause is that the surrounding process, data, ownership and controls were never prepared to receive the tool.

Related Patterns describe parts of the picture. Why buying AI does not automatically create business value covers the purchase side, when employees use AI but the operating model stays the same covers individual use without change, and the hidden cost of automating a bad process with AI covers a hazard of moving too quickly. This Pattern concentrates on why pilots stall between demonstration and daily use.

Diagnosis

  • No owner after the demo. The sponsor moves on to other priorities and the pilot belongs to nobody. Without an owner, nothing is fixed and nothing is expanded.
  • The process was never defined. The pilot was run on a convenient sample, so the real work, with its exceptions, formats and handoffs, was never designed in.
  • Data is not ready. The inputs are scattered across inboxes, shared drives and individual laptops, and the tool cannot reach them without manual effort. The Pattern when poor data quality becomes an automation failure describes the effect.
  • Governance is unanswered. Questions about which data may be sent to which service, who reviews outputs and how errors are handled stay open. Under POPIA, personal information cannot simply be pasted into an external tool without thought, so these questions block rollout until someone answers them.
  • Success was never defined. Without a baseline, such as minutes per task or error rate, nobody can show an improvement, and nothing justifies the effort of embedding the tool.
  • The output has nowhere to go. The result is a draft in a chat window rather than a record in the system of work, so someone must copy it across, which removes much of the saving.

Commercial impact

The commercial value at stake is productivity, decision quality, service speed and the return on every Rand already spent on tools, licences and staff time.

The first cost is the direct spend with nothing to show for it. Licences, consultant hours and workshop time are small individually, but a sequence of pilots can add up to a substantial amount, and none of it compounds because nothing is carried forward.

The second cost is credibility. Each stalled pilot teaches staff that new initiatives fade, which makes the next, possibly better, one harder to launch. People learn to wait it out.

The third cost is uneven use. Where no sanctioned process exists, enthusiasts use consumer tools on their own, with their own prompts and their own data handling. The business gets the risk of AI use without the benefit of consistent practice. IBM's overview of AI governance notes that organisations need defined oversight, accountability and policies for how AI systems are used; informal pilots tend to have none of these.

The fourth cost is competitive. Competitors who have embedded even a modest capability, such as faster quote preparation or more consistent first-line replies, accumulate small advantages that show up in response times and cost per transaction.

Common misidentification

The first misreading is to treat the stalled pilot as a tool problem and swap vendors. A new tool run through the same pilot design will stall at the same point. The second is to treat it as a training problem and schedule more workshops. Staff who can use the tool but have no process to use it in will still not use it.

A third misreading is that the business should wait until the technology matures. Waiting does not prepare the data, define the owner or settle the governance questions. Those are the long-lead items, and they are equally necessary whichever tool is chosen later.

Possibility

In a better state, each AI use is attached to a named process with an owner, a measured baseline and a defined place for the output. The tool is one step in the process, not a separate activity. A human reviewer is named for the cases where the tool is uncertain, and a record shows what the tool produced and what was changed. After a trial period the business decides to adopt, adjust or stop, on evidence.

IBM's introduction to business process automation makes the underlying point: automation works best on well-defined, repeatable work. The more clearly the process is described, the more reliably any tool, AI or otherwise, can be put into it.

Intervention

  1. Pick one process, not one tool. Choose a repeatable task with enough volume to matter, such as classifying incoming invoices or drafting first replies to common queries.
  2. Measure the current state. Record time per item, error rate and who does it today, for at least a few weeks.
  3. Fix the inputs and the destination. Decide where inputs will come from and where outputs will be recorded, and remove any manual copying.
  4. Settle the rules first. Decide what data may be used, who reviews the output and what counts as an error, and write it on one page.
  5. Name an owner with time. Give one person the responsibility and the hours to run the trial and report results.
  6. Decide at a fixed date. At the end of the trial, compare against the baseline and choose to adopt, adjust or stop.

Diagnostic questions

  • Which of your AI pilots is part of someone's daily routine today?
  • Who owns each pilot, and how many hours a week do they have for it?
  • What was the measured time or error rate before the pilot began?
  • Where does the tool's output go, and does anyone have to copy it by hand?
  • Do you have a written rule about what business and personal data may be shared with an AI service?

Bottom line

Most AI pilots do not fail; they simply do not continue. The gap between a good demonstration and a working operation is made of ordinary things: an owner, a defined process, usable data, clear rules and a way to measure the result. Prepare those for one process, and the same tool that stalled as a pilot has a realistic chance of becoming part of the way the business works.

Sources and further reading

Related Mellorca reading

Article summaryImpressive demonstrations rarely turn into daily practice. The gap is process, ownership, data and rules, not the tool. Here is how to close it.
Explore our solutionsStart a conversation