Organise AI Around the Workflow, Not the Model

In March we sat with the executive committee of a manufacturing group reviewing its AI programme. The centre of excellence had a slide of pilots that ran to two columns: contract summaries, a sales assistant, invoice reading, a maintenance chatbot, a forecasting experiment. The chief executive asked which of them the business would miss if they were switched off on Friday. After some discussion, the answer was one, and it was the one a plant manager had adopted and was paying for out of his own budget.

The pilots had not failed. Most of them worked in the sense that the demonstration did what it promised. What none of them had was a person whose results depended on it, who was staffed to handle what it could not do, and who had the authority to change the process around it. The one survivor had exactly that person. We think this is the pattern, not the exception, and that it follows from how most companies organised for AI in 2023.

The centre was built around last year's scarce skill

When ChatGPT arrived, the scarce thing was knowing what these models could do. It made sense to gather the few people who understood them into a central team, give it an innovation budget and let it run experiments across the business. The centre of excellence is organised around the model, because in 2023 the model was the hard part.

That is ending. Models are now an API call. GPT-4 and Anthropic's Claude 3, released last month, are available to any company with a credit card, and every quarter the capable models get cheaper. Andreessen Horowitz's survey of enterprise leaders, "16 Changes to the Way Enterprises Are Building and Buying Generative AI" (March 2024), found generative AI spend moving from one-off innovation budgets into permanent software budgets. That is a healthy sign. A permanent budget line is still not an owner.

The centre's own incentives make it worse. A team measured on the pilots it launches will launch pilots. It has no way to be measured on a workflow it does not run, so it cannot be blamed for the pilots that never become one, and nobody else can either.

What stays scarce is different. It is someone accountable for a workflow's outcome who is willing to change how the work is done, and engineers who can connect AI to the ERP, the bank files and the ticketing system that the workflow actually runs on. Neither of those lives naturally in a central lab.

Klarna did not have a better model

Klarna's announcement in February was striking for its numbers: an assistant handling two thirds of its customer service chats in the first month, with the company reporting average resolution time down from 11 minutes to under 2 and repeat inquiries down 25%. The model underneath came from OpenAI and is available to Klarna's competitors on the same terms.

What competitors cannot copy by signing the same contract is the rest: an operation that decided the assistant was part of how customer service runs, connected it to the order and refund systems a customer's question depends on, and measured it on customer service's own numbers. Look at which numbers Klarna chose to report. Not model accuracy, not the share of questions answered, but repeat inquiries: how often a customer had to come back. Only the owner of customer service cares about that number, and only a change in how the operation runs can move it. Whatever one thinks of the headline claims, the value appeared where a business function owned the result.

The unit of AI is a workflow with an owner

We suggest organising around the workflow. Order to cash, supplier invoice to payment, claim to settlement, ticket to resolution. Each has an executive whose numbers it moves, and that executive owns the AI in it. Ownership here is specific. The owner is accountable for four things:

• The result. Measured on the workflow's own numbers, such as time to invoice, days sales outstanding or first-contact resolution, not on model accuracy.
• The exceptions. Every case the system does not handle goes to people who are staffed and trained to handle it. If the exceptions have nowhere to go, the system will be bypassed.
• The rules. Which decisions the system may take, which need a person, and who may change either. Written down and signed.
• The run budget. The cost of running and changing the system for years, in the owner's operating budget, not the innovation fund.

A sponsor attends the steering committee. An owner is the person who gets the phone call when the system is wrong. Pilots have sponsors. Production needs owners, and a company has as many AI systems in production as it has owners willing to be called.

The engineers sit with the operation

The skill that decides whether AI reaches production is integration: reading from and writing to systems that were never designed for it, handling the file format a large supplier changed without notice, knowing that a posting in a closed period will be rejected. It is learned by sitting next to the people who do the work, watching what they do when the screen says something unexpected.

So the engineers building a workflow's AI should work in that workflow, with that team, for the months it takes to get it right. Not in a central lab taking requests through a ticket queue. Much of the talk last year was about prompt engineering as the new job. We expect the job that matters to be less fashionable: an engineer who can make an ERP accept a correct transaction exactly once, and who knows the accounts payable team by name.

What the centre keeps

This does not abolish the centre. It changes its job from running use cases to running infrastructure and standards, closer to a treasury than a lab.

• Access and contracts. Approved models, hosting arrangements, data processing terms, one place to buy capacity.
• Standards. How decisions are logged, how a system is tested against real cases before it goes live, what evidence a workflow owner must be able to produce.
• Risk policy. Which classes of decision may be automated at all, and who signs off.
• A small bench of specialists who help workflow teams with hard problems and move on.

What it gives up is the portfolio of pilots. If a use case has no owner in the business, the centre should not be running it.

Software that acts needs a manager

Last month Cognition showed Devin, a system that plans and carries out multi-step software tasks on its own. Whatever it proves to be, it shows where the research is heading: from models that answer to systems that act. Inside a company, acting means posting, paying, replying to a customer, changing a record. Ethan Mollick's Co-Intelligence (April 2024) is right that every employee should be experimenting with these tools. But individual experiments do not change an operation, and software that acts inside one raises a question no experiment answers: who manages it?

Here is a prediction we would put in writing. Within a few years, that will be the hard organisational question in most operations, and the right answer is the manager who runs the work today. Not IT, and not the centre. If software processes a third of the supplier invoices, the accounts payable manager should manage it the way she manages the people who process the other two thirds. She decides what it may handle, sees its workload and its errors each week, and changes its rules when the business changes. The centre gives her the tools. It does not do the job for her. A company that has named its workflow owners will be able to let software act sooner, because it already knows whose permission the software is acting under.

What to do this quarter

1. Put a name next to every pilot. For each item on the list, write the executive whose numbers it changes and who has agreed to own it. Stop the ones without a name. It will be most of them, and that is useful information.
2. Move the money. Fund each surviving system from the owner's operating budget, including the cost of running it for three years. What an owner pays for, an owner uses.
3. Move the engineers. Assign the builders to the workflow for its duration, sitting with the team that runs it.
4. Shrink the centre to a platform. Keep contracts, standards, risk policy and a small specialist bench. Stop measuring it by the number of pilots.
5. Change the definition of done. A system is in production when it appears in its owner's weekly numbers and its exceptions are written into someone's job description.

Count owners

A list of pilots measures curiosity. A list of owners measures how much of the business has actually changed: each name is a workflow where someone has said this is ours, this is what it changed, and this is who handles what it cannot. Count owners, not pilots.