Autonomy Is Earned One Decision Class at a Time

An AI agent taking orders for an IT hardware provider makes about a dozen kinds of decision: whether the setup can be built, whether the customer's credit holds, whether to split a delivery when the monitors are in stock and the laptops two weeks out, which agreed terms apply. A few weeks into testing on live orders, the head of sales operations asked what every sponsor asks: it's nearly always right and the approvals slow the desk, so when can we switch them off?

Some decisions had been right for weeks and could be undone in seconds. Others had been right as often but committed the company to something a customer would hold it to. One switch treats them alike.

The head of credit control had the better model. She doesn't decide whether customers in general may buy on credit. She sets a limit per customer from payment history, reviews it on a date, and the account blocks when an invoice goes unpaid. That is how a business extends trust it can defend.

Autonomy belongs to a decision type

A decision type is narrow enough that its cases share both difficulty and cost of error: "check a setup against the build rules", "split a delivery". Types are listed on the process map before the build, and every decision is labelled with its type in the case file. Each type has its own level, recorded with the date, the evidence and who agreed it. The agent has none. Asking whether it can run unsupervised is like asking whether the order desk can accept orders. Which orders?

Five levels

1. Suggest. The agent proposes; a person does the work.
2. Draft. The agent drafts the change in the target system; a person releases it.
3. Act with approval. The agent acts once a person approves a one-screen decision card. Approving is a decision, not rework, so it beats a draft.
4. Act and report. The agent acts; a named person reads a daily report and can reverse anything.
5. Act silently. Nobody looks, except through sampling and monitoring.

Feng, McDonald and Zhang's "Levels of Autonomy for AI Agents" (Knight First Amendment Institute, July 2025) gives a careful five-level scale, but one level per agent. Inside an operation, the same agent needs several at once.

The system enforces the level, not the prompt

A level in the agent's instructions is a suggestion. So each type's level is a setting outside the agent that decides which tools it gets: read tools on suggest; a draft tool but nothing that releases on draft; on act with approval, a final action that needs an approval token only the decision card issues, tied to that case and that version of the proposal. Raising a level is a logged change through the client's change process. The agent can't talk its way up: the tool isn't there.

Moving up needs four kinds of evidence

A type moves up one level when four things hold and the process owner signs off in writing:

• Accuracy on recent cases. Necessary, and the least of the four.
• Limited damage. The worst likely error has a known cost and a known reversal. Goods released on credit and already on a truck have none, so that type stops at act with approval.
• A later check. Something catches errors no reviewer sees any more: a reconciliation, a customer confirmation. Without one, promotion just hides errors.
• An owner. A named person answers for the type at its new level.

The agent never proposes its own promotion.

How many error-free cases are enough

"Right on every case for six weeks" is weaker than it sounds. Hanley and Lippman-Hand (Journal of the American Medical Association, 1983) give the shortcut: after zero errors in n cases, you can say with 95% confidence only that the true error rate is below about 3 divided by n. Three hundred clean cases rule out one in a hundred, not one in five hundred.

The owner sets, from the cost of one error and the monthly volume, the error rate the business can carry at the next level; the type needs about three divided by that rate in clean, recent cases. A setup check, where an error is caught at picking and costs minutes, moves after a few hundred. Releasing orders held on credit may never gather enough cases in a year to pass act with approval. That is the right answer.

Delivery splits ran about 400 a month; a wrong split cost a second shipment and a call to the buyer, so the sponsor accepted one in two hundred: 600 clean cases, about six weeks if nothing changed. A wrong special term, a condition agreed with one customer, binds the company for the rest of the agreement year. One in five thousand meant 15,000 clean cases, over two years. It stayed on act with approval; the arithmetic fitted on one page.

A reviewer agreeing is not the agent being right

On act with approval the reviewer sees the proposal first, and people anchor. Approving something plausible isn't reaching the same answer independently, so approval rates overstate accuracy, most on the unclear cases that matter. Before any promotion, a small sample is reviewed blind and the answers compared. That number, not the approval rate, goes in the record. It's usually lower, sometimes much lower.

Moving down is a rule, not a meeting

A type drops a level automatically when a confirmed error slips past review, when its inputs change (a new order format, product line or market), or when its later check breaks. A model upgrade or rule change drops every type to act with approval until it requalifies. A new model is a new customer with no payment history, however good its references. And every level above draft has a review date; a lapsed review drops a level, as a credit limit lapses.

Every promotion has a price

Promotion thins the review that produced its evidence, so sampling on the upper levels is sized to show a doubled error rate within a month. For a low-volume type that can mean a third of its cases, and then it should stay where it was.

Demotion is only real if someone can take the work back. If delivery splits drop, 400 cases a month return to a queue. With no one free, the team resists the demotion or approves unread, and the rule becomes theatre. So each promotion records who takes the work back and roughly how many hours a week. That is the real price of the autonomy.

Most types settle in the middle

Most end on draft, act with approval or act and report. Act silently is for reversible decisions checked automatically, like matching a bank line to an open invoice, where a mistake shows in the next run. With a good decision card, act with approval costs a reviewer seconds per case. Going silent saves those seconds and removes the one place someone would notice the world changing. We decline a global autonomy switch, even in a pilot: it promotes the weightiest decisions with the easiest.

So, when can we switch off the approvals? For some decisions, now, and here is the evidence. For some, after a few hundred more cases. For some, never, and here is the arithmetic.