MELLORCATalk to Mellorca
Discovery

Integration & API Operations

Why manual file transfers survive inside otherwise automated processes

Downloading a file from one system and uploading it to another is often the only manual step left in an automated process. Why it survives, what it costs and how to close the gap.

Mellorca Discovery2026-10-06

The order system exports a file at 06:00. The warehouse system imports a file at 07:00. Both jobs are automated. Between them, someone logs into a portal, downloads the export, renames it, opens it to check it looks right, and uploads it to a different screen. When that person is on leave, or busy, or stuck in traffic, the whole chain waits.

This is one of the most common gaps in automated operations: a manual file transfer sitting in the middle of a process that everyone describes as automated. It rarely appears on a process map because it feels too small to mention. It is often the step that fails most.

The problem in plain language

A manual file transfer is any point where a person has to move data from one system to another as a file. The usual forms are familiar:

  • downloading a report or export from a portal and uploading it elsewhere;
  • saving an email attachment to a folder that a scheduled job watches;
  • copying a bank, payroll or supplier file from one network drive to another;
  • opening a CSV in a spreadsheet to "tidy it" before importing it.

Each system on either side may be well run. The hand-off between them depends on memory, timing and a person's judgement about whether the file looks correct. There is usually no record of who moved which file, when, or whether it was changed on the way.

What the buyer is actually trying to solve

People rarely search for "manual file transfer". They ask how to automate a download from a supplier portal, how to stop an import failing when someone forgets the file, or why a process is "automated but still slow". Operations and finance managers describe the symptom as a process that only works when a particular person is in. The real question underneath is how to make the whole process run end to end, with a reliable record of every hand-off, without replacing the systems on either side. A broader view of the same pattern is in when your staff become the integration between software platforms.

Evidence and system mechanism

File transfer is a legitimate integration style

The File Transfer pattern in Enterprise Integration Patterns describes applications producing files that other applications consume, at intervals that suit the business, with processing to convert formats between them. Files are not the problem. The pattern assumes the producing and consuming applications handle the exchange. When a person does it instead, the timing, format conversion and checking all move out of the systems and into someone's working day.

The tooling to automate the hop already exists

Managed file transfer services show what the automated version looks like. Amazon's documentation for AWS Transfer Family managed workflows describes processing that starts automatically once a file has been transferred: moving it to the right folder, decrypting it, tagging it, running custom validation and sending notifications. It also describes exception-handling steps, such as emailing an administrator when a file fails validation, and detailed logs for each step that can be audited to trace data lineage. Similar capabilities exist in other integration platforms. The point is that the manual step usually persists for organisational reasons, not because the technology is missing.

Personal information makes the hop a compliance question

Many of these files carry personal information: payroll, customer, patient or debtor data. Under section 19 of the Protection of Personal Information Act, a responsible party must secure the integrity and confidentiality of personal information, identify foreseeable risks, maintain safeguards and regularly verify that they are effectively implemented. A file that passes through someone's downloads folder, personal email or USB drive is hard to square with that obligation, because nobody can show where copies went.

Problem owner and why-now trigger

The person who suffers the problem is usually the process owner: the operations manager whose orders are late, or the financial controller whose bank file was not loaded. The person who could fix it is often in IT or an external integration partner. That split is part of why the manual step survives. Budget typically sits with the finance director or chief operating officer.

The trigger is frequently a merger, acquisition or consolidation. When two businesses combine, their systems are joined quickly and often by file, with a person in the middle "for now". Months later the temporary hand-off is a permanent part of month-end. Other triggers include the departure of the one person who knows the routine, an audit question about personal data, or a growth in volume that makes the daily download impossible to keep up with.

Economic consequence

The direct labour cost is modest per file, which is why it is tolerated. The larger costs are indirect:

  • Delay. Downstream automation cannot start until the person has done their step, so the whole process runs at human speed.
  • Key-person risk. The process depends on one or two people being available at the right time.
  • Silent errors. A file uploaded twice, or the wrong day's file, or a file "fixed" in a spreadsheet that changed a date format, can create errors that surface days later.
  • Data exposure. Copies of sensitive files accumulate in inboxes and local folders.
  • Integration debt. Every later change has to work around the manual step, which inflates project estimates.

The way this kind of half-automation hides its cost is described in more detail in manual automation triggers leave half the process manual.

Root cause

  • Automation scoped by system, not by process. Each project automated its own system's import or export and stopped at the boundary.
  • Access barriers. The source is a portal with no API or SFTP option, or the credentials belong to a person rather than a service account.
  • Distrust of the data. Someone checks the file by eye because a bad file once caused trouble, and the check was never automated.
  • Temporary arrangements that became permanent. Common after mergers and system migrations.
  • No owner for the hand-off. Each system has an owner; the gap between them does not.

Practical intervention

  1. Find the hops. Walk each core process from trigger to outcome and mark every point where a person moves a file. Ask the people who do it, not only the people who designed it.
  2. Record why each one exists. Is it access, format, distrust, or habit? The reason decides the fix.
  3. Automate the transfer where access allows. Use an API, SFTP or a managed transfer service with service credentials, not personal logins.
  4. Automate the check the person was doing. If someone looks at the file to confirm the row count, date range or totals, write that check into the workflow and route failures to a named queue.
  5. Log every transfer. Record arrival, source, checksum and outcome so duplicates and gaps are visible.
  6. Where a manual step must stay, control it. Use a single secure location, a checklist, a second-person review for financially material files, and a log entry for each transfer.

Trade-offs

Some partners only offer a portal download. Screen automation can bridge this, but it is fragile; a controlled manual step with logging may be safer until the partner offers a proper interface.

An illustrative example

Illustrative: a Gauteng logistics business with about 120 staff acquired a smaller competitor. To bill the new customers quickly, an administrator downloaded a daily delivery export from the acquired company's system and uploaded it to the billing platform. Invoicing was automated on both sides. When the administrator was ill for a week, nobody else knew the routine and invoices for the acquired customers stopped. The business moved the export to a scheduled SFTP drop under a service account, added a row-count and date check, and sent failures to a shared operations queue.

Diagnostic questions

  • Which of your "automated" processes stop if one named person is away?
  • Where do files containing personal information pass through someone's email or downloads folder?
  • Could you show an auditor who moved yesterday's bank or payroll file, and whether it was changed?
  • When someone checks a file by eye, what exactly are they checking for?
  • Which manual hand-offs were set up as temporary after a merger or migration?

What good looks like

Every core process runs from trigger to outcome without a person moving files, except where a documented reason requires it. Transfers use service credentials, run on schedule or on arrival, are validated automatically and are logged. Failures go to a named team queue with enough context to act. Where a manual step remains, it is deliberate, controlled and recorded. For the documentation that makes this sustainable, see what to document before connecting two business platforms.

Where Mellorca fits

Mellorca's systems implementation and integration work can map manual hand-offs, replace them with automated and logged transfers, and build the validation checks people currently do by eye. Where the bigger question is which processes to tackle first, a Digital Systems Audit identifies the hand-offs carrying the most delay and risk.

Method note

Product capabilities are described from current vendor documentation and differ between platforms. The legal reference is a summary of POPIA section 19, not legal advice. The example is illustrative rather than a client case. Manual steps are not always wrong; the risk lies in hand-offs that are unrecorded, unchecked and dependent on one person.

Sources and further reading

Related Mellorca reading

Article summaryDownloading a file from one system and uploading it to another is often the only manual step left in an automated process. Why it survives, what it costs and how to close the gap.
Explore our solutionsStart a conversation