MELLORCATalk to Mellorca
Patterns

AI, Automation & Systems Readiness

The Hidden Cost of Automating a Bad Process With AI

Adding AI to a process that is already confused does not fix it; it makes the confusion faster and harder to see. Here is where the cost hides and what to settle before you build.

Mellorca Patterns2026-10-09

Observable condition

A Stellenbosch wine distributor decides to use AI to draft replies to customer order queries. The tool is configured in a fortnight and the first demonstrations are impressive. Three months later the support lead is reading every draft before it goes out, because the replies sometimes promise delivery dates that the warehouse cannot meet, quote a discount that expired in March, and apologise for a delay that did not happen. The team now spends as long checking drafts as it once spent writing replies. The licence, the set-up fee and the consultant's invoice are real, and the saving is not.

The pattern repeats elsewhere. An AI assistant routes incoming invoices into approval queues, but the approval rules were never written down, so it routes by guesswork. A tool summarises sales calls into the CRM, but there is no agreed list of what a good record contains, so the summaries are fluent and unhelpful. A bot books service appointments against a diary that three people edit differently.

In each case the technology did what it was asked. The trouble is what it was asked to do.

Realisation

A process that is unclear, inconsistent or full of workarounds depends on people quietly filling the gaps. They know which customers are exceptions, which form is out of date and which figure to distrust. They do this without noticing, and that unwritten knowledge is part of the process. When the process is handed to AI, the gaps do not disappear. They are now filled by a model that has never met the customer, does not know the exceptions, and answers confidently anyway.

IBM's description of intelligent automation lists AI alongside business process management and robotic process automation as components that work together to streamline decisions. The point in that framing is that automation sits on top of a defined process. Where the underlying process is not defined, there is nothing for the layers above to streamline.

Realisation prompt: if you asked three experienced staff to describe how an order query is handled, would they give the same steps, the same exceptions and the same decision rules?

Identification

This is a pattern of automated ambiguity. The visible symptom is an AI tool that produces plausible output requiring heavy checking, correction or escalation. The common assumption is that the model is not good enough, or needs better prompts. The structural cause is that the process around it lacks clear rules, clean inputs and defined outcomes, so no model can be reliably right.

The Insight why automating a broken process makes the problem worse sets out the general argument, and the Impact page automating a broken process without simplifying it estimates the cost. This Pattern concerns what is different when the automation is AI: the output looks finished even when the logic underneath is not.

Diagnosis

  • Rules live in people's heads. Discount limits, exception customers, credit holds and escalation thresholds are known by long-serving staff. Once such rules are pasted into an AI prompt, they become harder to find and audit.
  • Inputs are inconsistent. Product names, customer records and prices differ across systems. A model repeats whatever it is given, so poor data becomes poor answers.
  • Nobody has defined a correct output. Without a standard for what a good reply, record or decision looks like, quality is judged by impression, and problems surface through customer complaints.
  • The process has unnecessary steps. Duplicate approvals, re-keying and legacy checks are automated along with the real work. IBM's overview of process mining describes using event data to see how a process actually runs, which often reveals steps and loops that nobody intended.
  • Exceptions are a large share of the work. The routine cases may be few. If a third of cases need human judgement, an AI tool built for the standard path increases handling of the rest.
  • Risk is not mapped. The NIST AI Risk Management Framework asks organisations to identify and manage the risks of AI systems in their context. In a poorly understood process, those risks are difficult to name.

Commercial impact

The commercial value at stake is realised savings, error and rework cost, customer trust and management attention.

The first hidden cost is checking. If every output must be reviewed because it cannot be trusted, the labour moves from producing to inspecting, and sometimes grows. Inspection is also tiring and error-prone: reviewers begin to approve what looks right, and mistakes reach customers at higher speed than before.

The second is errors at scale. A person who misquotes a price affects one customer. A tool that misquotes a price can affect hundreds before anyone notices. Refunds, credit notes and apologies are costs that arrive weeks after the project was declared a success.

The third is lost improvement. Once a flawed process is automated, it becomes part of the system and harder to change, and the people who knew how it should work have moved on to other tasks. The business has spent money to make a problem permanent. The Impact page buying AI without a defined business problem looks at the wasted spend directly.

Common misidentification

When results disappoint, the first conclusion is that the AI tool is weak, and the response is to switch vendors or upgrade the plan. The same problems follow to the new tool. The second is that staff are resisting change, so training is scheduled. People are often correctly sceptical because they can see the rules the tool is missing.

A third reaction is to keep adding prompt instructions until the tool seems to behave. That works for the cases tested and hides the rest, and leaves a long prompt that nobody can maintain. The Pattern why buying AI does not automatically create business value explores this expectation gap.

Possibility

In a better state, the process is understood and simplified before anything is automated. The steps are known, the decision rules are written down and owned, the inputs come from agreed sources, and a clear definition of a correct result exists. Automation is aimed at the stable, high-volume part of the work, with a defined route to a person for exceptions.

AI is then used where its flexibility helps, in drafting, summarising and classifying, with checks proportionate to the risk. Savings can be measured because the baseline was measured first, and the business can say whether the project paid for itself.

Intervention

  1. Map the process as it actually runs. Follow ten real cases from trigger to outcome, including the workarounds and the exceptions. The Insight before you automate, understand how the work actually moves shows how.
  2. Remove before you automate. Delete steps that exist only because of old habits, duplicate checks or system limitations.
  3. Write down the rules. Capture discounts, limits and exceptions in a short document with an owner, so that they can be applied by people and tools alike.
  4. Define a good output. Agree what a correct reply, record or decision contains, and collect twenty examples for testing.
  5. Measure the baseline. Record current handling time, error rate and cost per case, so the saving is a figure and not an impression.
  6. Start narrow, with a human route. Automate the clear cases first, send everything else to a named person, and extend only when results hold.

Diagnostic questions

  • Could a new employee follow the process from written instructions alone?
  • Which decision rules exist only in someone's memory?
  • What proportion of cases are exceptions?
  • How would you know if the AI output were wrong?
  • What was the cost per case before the tool arrived?

Bottom line

Automating a bad process with AI is expensive because the tool speeds up the confusion and gives it a convincing face. Time spent mapping, simplifying and writing down the rules before a licence is bought is less than the cost of checking, correcting and apologising afterwards.

Sources and further reading

Related Mellorca reading

Article summaryAdding AI to a process that is already confused does not fix it; it makes the confusion faster and harder to see. Here is where the cost hides and what to settle before you build.
Explore our solutionsStart a conversation