Why Good Systems Get Quietly Bypassed

Three weeks after a matching system went live, its numbers looked healthy and the team's workload had not dropped. We spent a morning with the team to find out why. One of the most experienced clerks had a spreadsheet open beside the system. Every match it proposed, she re-keyed and checked by hand before accepting it.

She was not resisting change. She had done this work for years, she was accountable for its accuracy, and nothing in the system told her when it could be trusted. So she trusted it never, and checked everything. The system was working. In practice, it was also bypassed.

We have stopped calling this an adoption problem. It is a problem of calibrating trust, and unlike adoption, that can be measured and designed for.

Calibrated trust is the target

The aim is not for people to trust the system. It is for them to trust it exactly as much as it deserves, case by case. Too much trust leads to rubber-stamping, which is a problem of the approval screen. Too little leads to the spreadsheet beside the screen and a team doing the work twice, which is a problem of the organisation around it, and the subject here. Training fixes neither. People calibrate trust from evidence on their own cases, not from explanations of how the system works.

Two numbers show where trust is missing

We track two numbers from the first day of go-live, per segment of work and per team, never per person.

Check rate. How often people open the source document, the ERP record or a file of their own before accepting a proposal they could have accepted directly. A high check rate on a segment where the system is reliably right means trust is lagging the evidence. That is information, not a behaviour to stamp out: it points to exactly the segments that have not yet earned trust in the team's eyes.

Correction latency. The time between a person correcting the system and the system changing its behaviour for that kind of case. It is the number that matters most, and almost no project measures it. If an expert corrects the same mistake three times and nothing changes, she concludes, correctly, that her corrections do not count, and she checks everything. If the rule changes within days and she watches the mistake disappear, trust grows faster than any demonstration could make it.

Shadow mode ends on criteria the team agreed

Before a system acts on anything, it runs in shadow: it processes the same cases as the team, its proposals are recorded, and the team works as before. Each day the proposals are compared with what people actually did.

Shadow mode is common. What makes it work is how it ends. The exit criteria are agreed with the team and the process owner before it starts: a minimum run of cases per segment, no disagreement in a money-moving field where the system was wrong, every disagreement reviewed. Segments leave shadow one at a time as each meets the criteria, and the old way of working is retired the same way, segment by segment, never on a fixed date. The team watches evidence build against a standard it helped set, instead of hearing a go-live date announced.

Every disagreement ends in one of four outcomes

Once a week in the early months, the team and an engineer review every case where the system and a person disagreed. Each is closed with one recorded outcome.

• The system was wrong. A rule or mapping changes, with a target of days, not weeks. This is where correction latency is won or lost.
• The person was wrong. Usually a habit that has drifted from policy. The process owner decides whether the habit or the policy changes, and tells the team.
• Both were defensible. The process owner decides, and the decision becomes a written rule. These cases are the unwritten policies that lived in people's heads, and they are worth more than any other.
• The input was the problem. A supplier document or master data record caused it, and it goes upstream to be fixed at the source.

The balance between the four shows how the system is maturing. Early on, most disagreements are system errors. Later, most are policy questions. When disagreements are rare and mostly about policy, the system has earned its trust.

The experts become rule owners

The deepest cause of bypassing is a change of role nobody named. Before the system, the experienced clerk was the person who knew how to match a difficult payment. After it, she was a user of a tool doing her old job, with a vague duty to check it. That is a demotion, whatever the project plan calls it.

So roles change explicitly. The most experienced people become owners of the rules and exception causes in their area. They approve tolerance changes, decide how new supplier formats are handled, and their names are on the rules. The system's accuracy becomes their responsibility and their achievement, and the knowledge that lived in one head becomes rules the whole team can read.

The sponsor states the purpose in writing

Every operations team wonders whether a new system is there to replace them. If nobody answers, they answer it themselves. Before shadow mode, we ask the sponsor to write down three things and share them with the team: what the team will do more of, what it will do less of, and what the business intends to do with the capacity. In the work we take on, that purpose is capacity for growth. A system that depends on corrections from people it threatens will not get them.

We also ask for throughput targets to be suspended during the transition. People measured on cases per hour will rationally ignore a system that slows them down while they learn it.

Trust follows correction speed

The spreadsheet beside the screen was the most useful thing we saw in that engagement. It marked the cases the system had not yet earned, and it told us her corrections were taking too long to count. Once a correction changed the system within days, she stopped opening it. Nobody had to ask.