SeriesFrom decision to deploymentPart 11 of 13
  1. AI Won't Fix a Chaotic Process. First Check You Have Something to Automate
  2. Don't Automate Your Most Important Process First. Pick One the Company Can Learn On
  3. Does Automation Pay Off? How to Build the Business Case Before You Invest in AI
  4. Do You Really Need an LLM? Picking the Right Technology for Process Automation
  5. From Excel to AI. Six Ways Companies Work With Data
  6. You Have Data and Reports. Why Don't They Lead to Better Decisions?
  7. The AI System "Works". But How Often Is It Right?
  8. 99% AI Accuracy Can Be Too Little. Or Too Much.
  9. The AI Passed 1,000 Tests. Is It Ready to Go Live?
  10. What Does It Really Cost to Run AI? What's Easy to Miss Before You Build
  11. What Can AI Do Without Human Approval? How to Set the Limits of Automation
  12. Who Is Responsible for AI Errors? How to Monitor the System and Respond to Problems
  13. How to Choose an AI Implementation Partner. What to Check Before You Sign

What Can AI Do Without Human Approval? How to Set the Limits of Automation

A customer writes to an online store: "The parcel arrived two days late and one of the products has a damaged casing. I don't want to keep it. Please refund the order, including the delivery cost."

The AI system spots the delay, finds the order and establishes that the store runs an extra compensation program for late deliveries. It can draft a reply and issue a refund of the delivery fee. Should it do that on its own? And if so, is it allowed to mark the request as closed?

Carrying out one permitted action correctly is not the same as settling the whole case correctly. That's why the limits of automation have to be set not only for the functions we make available to the AI, but also for the kinds of decisions and the range of cases the system may close by itself.

AI doesn't have to have one level of autonomy for everything

Handling a complaint involves several different actions. The AI can read the message, pull the data, draft a reply, send it, initiate a financial transaction or change the status of the request. Permission for one of these shouldn't automatically mean permission for the rest.

The system may, for example, give a customer the confirmed status of their parcel. That doesn't entitle it to share details of any order just because someone quoted its number. It may also prepare a refund proposal without being allowed to trigger the payment. And it may make one narrowly defined compensation payment without any authority to settle other claims.

It makes sense to tie the level of autonomy to four things: the consequences of a mistake, whether the grounds can be checked independently, whether the effect can be reversed, and whether the decision requires an authorized person. You can't boil that assessment down to a single monetary threshold. A small refund can be unjustified, and a harmless-looking message can misinform a customer about their rights.

Who decides that a case is simple enough?

A common assumption goes like this: the AI will handle the simple cases automatically and pass the hard ones to staff. The trouble starts with the question of who recognizes that a case is simple. It's the process owner who has to define in advance which conditions allow the system to act on its own, rather than leaving the model free to draw those lines.

In our example the model may notice the late delivery but miss the damaged product and the demand for a refund of the order. If the application treats the label "late delivery" as the full description of the request, it can correctly grant the compensation and then wrongly close the whole case.

Every financial check may work perfectly along the way. The error happens earlier, when the system works out what the customer is actually asking for.

So three things need to be kept apart: understanding what the request says, establishing the grounds for a specific decision and the right to carry out the resulting transaction. These are three different stages. Controls on the last one won't fix something overlooked in the first.

Three different stages of handling a request: understanding the message, establishing the grounds for a decision and the right to carry out the transaction. Each has its own check, and a correct check of permissions and amount won't fix a claim that was missed when the message was read

If the system can't reliably tell that it has captured everything in a free-text message, it shouldn't close the whole request on its own. That doesn't have to rule out automating a single action, though. The condition is that the action doesn't depend on interpreting the remaining claims correctly and doesn't affect how they will later be reviewed or settled. Where that independence can't be assured, you need an additional check or a human decision.

What can be checked independently of the model?

Suppose the store runs a voluntary program that refunds the delivery fee in certain cases of delay. The program is separate from the customer's statutory rights and cannot limit them.

The AI can recognize a message about a late parcel and suggest applying the program. But the model's suggestion alone shouldn't count as proof that the conditions are met. The application can pull the agreed delivery date, the actual delivery date, the fee charged and the history of earlier compensation payments. It also has to know which source is authoritative. The carrier's estimated delivery date isn't necessarily the date the store committed to. If the data is contradictory or incomplete, there are no grounds for acting automatically.

Separately, the application checks whether it's allowed to initiate this specific transaction: for the right customer, the right payment and the amount set by the program. A request written into the message, say for a transfer to a different account, can't widen those permissions on its own. The instructions given to the model describe the expected behavior, but it's the application and the target system that have to enforce the limits.

These are two distinct checks. One concerns whether the compensation is justified on the merits, the other the right to carry out a specific transaction. Even when both work properly, that doesn't mean the system has settled the complaint about the damaged product.

When can part of a case go ahead, and when does it have to stop?

Back to the customer who reports a delay and a damaged product in the same message. There are two possible scenarios.

In the first, the company's program provides extra compensation for the delay regardless of any other claims. The system confirms the conditions against the right data. It can also establish that paying this compensation won't reduce or duplicate the amount that will have to be settled for the remaining claims. Only then can this one action be allowed to run automatically. The request as a whole stays open, and the customer gets a precise message about what was done, with no statement that the complaint is closed.

In the second scenario, the compensation depends on what the message means, on the customer's consent or on how several claims are settled against each other. The store may not know, for instance, whether the planned refund of the full payment will already include the delivery fee. In that case a correct confirmation of the delay isn't enough to safely issue a separate refund of that fee. The system should hold the transaction and hand the case over for clarification.

Two scenarios for a request that combines a delay and a damaged product. When the compensation has independent grounds and doesn't change how other claims are settled, the system performs that one action and the request stays open. When it depends on the meaning of the message, the customer's consent or settling several claims together, the system holds the transaction and hands the case over for clarification

So simply leaving the request open doesn't make every partial automation safe. You also have to check that the action has independent grounds and won't change the outcome of reviewing the remaining claims.

This split lets you automate selected pieces of work without giving the AI the right to settle the whole complaint by itself. But don't confuse the system declining to perform an action with the company rejecting the customer's claim. The claim still has to be properly reviewed.

Human approval has to cover a specific decision

In an ambiguous case, the employee should see more than the AI's proposal: the text of the request, the confirmed facts, the disputed information and the effect of the proposed decision. They need to be able to refuse or change it. An "Approve" button alone doesn't give real control, as the article on handling unusual cases shows with an ambiguous order.

It also matters what exactly was approved. If an employee approved a refund of a given amount, a later change to that amount can't ride on the earlier approval. Before the transaction runs, the current state of the case and the key parameters have to be checked. If they've changed, a new decision may be needed.

People are needed most where independent checks aren't enough to decide on the merits, or where the applicable regulations require a human. That doesn't mean every action has to be approved. The aim is a conscious decision at the right point in the process, not turning the whole automation into a queue of clicks.

How to write down the limits of automation before deployment

A statement like "the AI handles simple complaints on its own" is too vague. For the program described here, the company and the vendor could agree on the following rules:

  1. Scope of action. The system may grant only the extra compensation for late delivery. It has no right to settle claims about the product on its own, or to close a request that contains other claims that are unidentified or unresolved
  2. Grounds for the decision. Automatic compensation requires confirming the agreed delivery date, the actual delay, the fee charged and the terms of the program. Without a reliable source, automatic execution is blocked
  3. Independent settlement. The payout can't depend on an unresolved reading of the message or interfere with reviewing and settling other claims. If that can't be confirmed, an employee takes over the case
  4. Execution permissions. The application may read only the data it needs and issue only the permitted type of compensation. The model can't change the recipient, the amount or the program rules by itself
  5. Human decision. Ambiguous cases go to a person who sees the sources and the effects of the proposed decision. Approval covers specific parameters, which are checked again before execution
  6. Status and communication. A message saying the compensation has been paid can go out only once the transaction is confirmed. It must not imply that other claims are closed or that the customer has lost any of their rights

The value of rules like these isn't in how many there are. It's that each one can be translated into specific application behavior and a condition that blocks an action. The limits also need checking once the system is live. The next part, on monitoring AI and responding to its errors, covers how to detect that it's closing cases with a missed claim after all.

The key question isn't whether AI can be trusted with handling complaints. It's which decisions and actions it may complete on its own, on what grounds, and what has to stay outside its permissions. Only then does "automated handling" mean something precise to both the company and the vendor.