Human-in-the-Loop for AI Agents: Where to Put Approvals So They Actually Catch Errors

A human approval step catches an AI agent's errors only under three conditions: it sits at an irreversible action, it shows the reviewer exactly what will change and why, and it arrives rarely enough that the reviewer still reads it. Approvals placed everywhere become rubber stamps. Anthropic reported in March 2026 that users of its Claude Code agent approve 93 percent of permission prompts, and named the result "approval fatigue, where people stop paying close attention to what they're approving." Decades of research on automation bias point the same way: a person shown a confident recommendation tends to accept it, including when it is wrong.

The working model follows from that. Classify every action the agent can take by how reversible it is and how far the damage would spread. Choose one of four oversight patterns for each class: no approval with logging, approval before commit, sampled review after the fact, or escalation on low confidence or a policy trigger. Design the approval request so the reviewer can judge it in seconds. Size the reviewer load so the queue never outruns attention. Then measure whether approvals catch anything, and widen the agent's autonomy only when the evidence says it is safe.

As of the week of October 12, 2026, the OpenAI Agents SDK, LangGraph, Microsoft's Agent Framework and Amazon Bedrock AgentCore all let a developer pause an agent for a human decision. The primitive is not the hard part. Deciding where to put it, and proving it works, is. This guide is for the executive who owns that decision, in any function.

Key Takeaways

• Approval is a scarce resource. Every unnecessary prompt makes the necessary ones less likely to be read. Spend approvals on irreversible, wide-blast-radius actions only.
• Automation bias is the default. In a 2023 Radiology study, inexperienced radiologists' accuracy fell from 79.7 to 19.8 percent when a purported AI suggestion was wrong.
• Four patterns cover almost every action: log only, approve before commit, sample after the fact, and escalate on a trigger. Pick by reversibility and blast radius.
• The request itself is the control. Show the diff, the evidence, the policy touched, and make rejection one click with a reason.
• Measure approvals like any other control: rejection rate, time to decide, and the catch rate on deliberately seeded errors.
• Autonomy should be earned with evidence, and taken back automatically when the model, the tools or the error rate change.

Why Most Approval Steps Fail

A person can catch what the model gets wrong, but only under the right conditions, and most deployments get the conditions wrong.

Automation bias: people agree with the machine

Automation bias is the tendency to over-rely on an automated recommendation, accepting it without enough independent checking. Raja Parasuraman and Dietrich Manzey's 2010 review in Human Factors found that automation bias produces both omission errors (missing a problem the system did not flag) and commission errors (following a recommendation that is wrong) when decision aids are imperfect. A 2012 systematic review in the Journal of the American Medical Informatics Association by Kate Goddard, Abdul Roudsari and Jeremy Wyatt identified 74 studies of the effect in clinical and other settings.

The size of the effect is the uncomfortable part. Dratsch and colleagues, publishing in Radiology in May 2023, gave 27 radiologists mammograms with a suggested rating from a purported AI system and made some suggestions deliberately wrong. Correct ratings fell from 79.7 percent to 19.8 percent for inexperienced readers, from 81.3 to 24.8 percent for moderately experienced readers, and from 82.3 to 45.5 percent for very experienced ones. Expertise reduced the effect but did not remove it.

Language models make this worse, because their output reads as confident and well-argued. A randomised trial reported in NEJM AI (preprinted in August 2025) gave 44 physicians, all of whom had completed 20 hours of AI-literacy training, diagnostic vignettes with suggestions from GPT-4o, some deliberately wrong. Physicians given suggestions containing deliberate errors scored 73.3 percent on diagnostic accuracy, against 84.9 percent for those given accurate suggestions. Training did not inoculate them.

An agent's approval request is a recommendation from a machine. Unless the request is designed against automation bias, the reviewer will mostly agree with it.

Approval fatigue: volume destroys attention

The second failure is volume. Anthropic's engineering team wrote in October 2025 that constantly clicking "approve" leads to approval fatigue, and that sandboxing its coding agent cut permission prompts by 84 percent in internal use. Its March 2026 post on Claude Code's auto mode reported the 93 percent approval rate cited above. When nearly every request is approved, approval stops being a decision.

The pattern predates agents. A 2006 review in JAMIA by Heleen van der Sijs and colleagues found that clinicians override drug safety alerts in 49 to 96 percent of cases. In Uber's September 2022 breach, an attacker holding a contractor's stolen password got in after the contractor, faced with repeated two-factor login requests, accepted one. Microsoft now enforces number matching in its Authenticator app specifically to stop such accidental approvals.

Placement: the wrong action gets the check

The third failure is placing approval where it is cheap rather than where it matters. Teams often gate the first step of a task, such as "plan approved", and then let the agent execute twenty actions unchecked. Or they gate every tool call equally, so a read query and a payment look the same to the reviewer. The guide to tool design for agents makes the point from the engineering side: a gate that prompts on every state change trains people to approve reflexively, which is worse than no gate.

Step 1: Classify Every Action by Reversibility and Blast Radius

The unit of oversight is the action, not the agent. One agent may read a ticket, draft a reply, update a record and issue a refund. Those four actions need four different levels of oversight.

Two questions classify each action.

How reversible is it? Three levels are enough:

• Reversible: an undo exists and costs almost nothing. A draft, a tag, a status change, a record in a sandbox.
• Reversible with effort: a compensating action can undo it, at some cost or delay. A refund can be clawed back, a configuration change rolled back, a purchase order cancelled before it ships.
• Irreversible: once done, it cannot be taken back. Money sent to an external account, an email or offer letter delivered to a person, data deleted without a backup, a contract signed, a public post.

How far would an error spread? Again, three levels:

• Contained: one record, one internal user, no external party.
• Wide: many records, a whole team, a shared system, or one external party.
• External or regulated: customers, regulators, the public, financial reporting, or anything with legal effect.

The agent or workflow decision guide uses the same reversibility question to decide whether a task should be autonomous at all. Here it decides how each step is overseen.

The action-classification table

Bulk deletion is often better removed from the agent's tools than gated: the sandboxing guide and the guide to AI agent identity and access management both argue that the strongest control is a permission the agent never holds. For IT operations, the guide to agentic AIOps applies the same classification action by action.

Step 2: Choose One of Four Oversight Patterns

Every action class maps to one of four patterns. Each has a cost and a failure mode.

Pattern 1: No approval, with logging

The agent acts; every action is recorded with its inputs, reasoning trace and result. This is right for reversible, contained actions, where a human check costs more than the occasional error. The log is what lets a later review reconstruct what happened; the agent observability guide sets out what to capture.

Fails when: nobody reads the logs, or the action was misclassified and is less reversible than assumed.

Pattern 2: Approval before commit

The agent prepares the action and stops. A named human sees the proposed change and approves or rejects it before anything happens. This is right for irreversible actions and for wide or regulated blast radius. It puts a person in the critical path, so it should cover the fewest actions.

Fails when: volume is high enough to cause fatigue, or the request does not show what will actually change.

Pattern 3: Sampled review after the fact

The agent acts. A human reviews a random sample, plus every case flagged by a rule, after the event. This is right for high-volume actions that are reversible with effort, such as routing, small refunds or standard replies. It does not stop individual errors. It measures the error rate, catches systematic problems early and provides the evidence for widening autonomy.

Fails when: the sample is too small to detect a rare but costly error, or reviews fall behind and stop being done.

Pattern 4: Escalation on low confidence or a policy trigger

The agent acts on its own unless a defined condition fires, in which case it stops and asks. Good triggers are deterministic and checked in code: an amount above a limit, a new payee, a customer flagged as vulnerable, an action outside business hours, a request that touches a protected record. Model-reported confidence can add to these, but it should not replace them, because a model that is wrong is often confidently wrong.

Fails when: triggers are written in the prompt instead of enforced in code, or nobody reviews whether the triggers fire on the right cases.

The decision grid

Step 3: Design the Approval Request

The approval request is the control. A badly designed request turns a careful reviewer into a rubber stamp. Four elements are required.

A diff of the change. Show what will be different afterwards, not a description of the agent's intent. "Update supplier record" is useless. "Bank account for Supplier X changes from account ending 4471 to account ending 9083; requested by email from a domain first seen yesterday" is reviewable.

The evidence. The source documents, the data the agent relied on, and the checks it ran. If the agent cannot show evidence, the request should say so explicitly, because missing evidence is itself the signal.

The policy it touches. Name the rule that made this action need approval: the amount limit, the new-payee rule, the access policy. This tells the reviewer what to check and stops approval becoming a general judgement of whether the agent "seems right".

One-click reject, with a reason. Rejection must be as easy as approval. A short reason, chosen from a list, produces the data that improves the agent and the triggers.

Further choices reduce automation bias:

• Show the evidence before the recommendation. A reviewer who reads the recommendation first anchors on it.
• Make the reviewer engage with the risky field. Microsoft's number matching works by making the person type a number shown on screen. The agent equivalent is asking the reviewer to confirm the one value most likely to be wrong, such as the destination account or the recipient.
• Bind the approval to the exact action. The approval should cover the precise parameters shown. If the agent changes anything afterwards, the approval is void.
• Default to deny on timeout. An unanswered request should expire, not proceed.
• Separate the requester from the approver. The person who asked the agent to act should not be the one approving its riskiest output, the same segregation of duties that applies to human work.

Step 4: Size the Reviewer Load

When approval demand exceeds capacity, the control fails silently: reviewers fall behind or approve without reading.

No published standard says how many approvals a person can handle per hour. Security operations offer the closest evidence. Vectra AI's 2023 survey of 2,000 security analysts found teams receiving an average of 4,484 alerts a day and unable to deal with 67 percent of them. A 2025 vendor-sponsored survey by Prophet Security of 282 security leaders put the average at 960 alerts a day, with 40 percent never investigated. Approval queues for agents will go the same way unless they are sized deliberately.

The arithmetic is simple, and worth doing before launch:

Reviewer hours per day = agent actions per day × share that need approval × minutes per review ÷ 60.

An illustrative case: an agent handling 5,000 service requests a day, with 4 percent needing approval at three minutes each, creates 600 minutes, or 10 hours, of review a day. That is more than one full-time reviewer before breaks, other work or the fact that attention degrades across a long queue. If the volume doubles, the control breaks unless something changes.

Plan the overflow before it happens:

• Hold, never auto-approve, when the queue is full. The agent waits or hands the case to a human to do manually.
• Set a per-reviewer ceiling and alert when it is exceeded, so a growing queue is visible to someone accountable.
• Move volume down the patterns, not up. If a class of approvals is almost always granted, convert it to sampled review with a tighter trigger, rather than adding reviewers to approve it faster.

Step 5: Measure Whether Approvals Work

An approval step is a control, and a control that is never tested should be assumed not to work. Four measures tell an organisation whether its approvals catch errors.

Seeded errors are the most useful test, and the least used. Insert a small number of known-bad requests into the approval queue, clearly tracked and never executed, and measure how many reviewers reject. This is the same idea as a phishing simulation. It turns the abstract question "do approvals work?" into a number.

A near-zero rejection rate can mean the agent is excellent or that reviewers have stopped looking. The seeded-error catch rate tells the two apart.

Step 6: Evolve Autonomy With Evidence

Autonomy should be widened by evidence, never by default or by deadline. A practical ladder has five rungs:

1. Shadow: the agent proposes, a human does the work, and the two are compared.
2. Approve every action in the class.
3. Approve above a threshold, with everything below sampled.
4. Sampled review only, with escalation triggers.
5. Log only.

Promotion between rungs needs stated criteria: a minimum number of decisions, an error rate below an agreed level on sampled review, and no misses on seeded errors over a set period. Demotion should be automatic on three events: an incident in the action class, a change of model or prompt (the release discipline in the LLM release management guide applies), and a change to the tools the agent can call.

Anthropic's auto mode for Claude Code, which replaces prompts with a classifier that blocks risky actions, shows the cost of that step. Its March 2026 post reported a 0.4 percent false-positive rate on real traffic and a 17 percent false-negative rate on real overeager actions, which it called "the honest number." That is the right shape of evidence: a measured miss rate that can be compared with what human approvals were actually catching.

What Standards and Regulators Expect

The OWASP Top 10 for Large Language Model Applications (2025 edition) lists "excessive agency" as a top risk and recommends human-in-the-loop control "to require a human to approve high-impact actions before they are taken." The emphasis on high-impact actions is the point: OWASP does not recommend approving everything.

The EU AI Act goes further and names the problem. Article 14 requires that people overseeing high-risk AI systems be able "to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)." In other words, the law treats a human who rubber-stamps as insufficient oversight. The timing has moved: the Digital Omnibus, adopted as Regulation (EU) 2026/1744 and in force since July 27, 2026, delayed the obligations for stand-alone high-risk systems from August 2, 2026 to December 2, 2027, and for AI embedded in regulated products to August 2, 2028.

Liability is the other reason to get this right. When an agent causes a loss, the approval record is evidence of whether the organisation exercised reasonable control, a question explored in the agent liability gap. An approval log full of three-second approvals is weak evidence.

The Approval-Design Checklist

• [ ] Every agent tool is classified by reversibility (reversible, with effort, irreversible) and blast radius (contained, wide, external or regulated).
• [ ] Each action class has exactly one oversight pattern, recorded in the tool's metadata, not in the prompt.
• [ ] Irreversible, high-blast-radius actions are approved before commit or removed from the agent's permissions.
• [ ] Escalation triggers are deterministic and enforced in code.
• [ ] Each approval request shows the diff, the evidence, the policy touched, and offers one-click rejection with a reason.
• [ ] Evidence is shown before the recommendation, and the reviewer confirms the riskiest field.
• [ ] Approvals bind to exact parameters and expire to deny.
• [ ] The requester and the approver are different people for high-risk actions.
• [ ] Reviewer load is calculated before launch, with a per-person ceiling and an alert.
• [ ] Queue overflow holds the action; it never auto-approves.
• [ ] Rejection rate, time to decide, seeded-error catch rate and errors found later are reported monthly.
• [ ] Autonomy changes follow written promotion criteria, and demote automatically on incidents, model changes or tool changes.

FAQ

What does human-in-the-loop mean for AI agents?

Human-in-the-loop means a person must approve an agent's action before it takes effect. For agents, it should be applied selectively, to irreversible or high-impact actions, rather than to every step. Lower-risk actions are better handled by sampled review after the fact or by escalation when a policy trigger fires.

How many approvals are too many?

Too many is the point at which reviewers stop reading. A rejection rate near zero, with decisions taking a few seconds, is the clearest sign. The definitive test is whether reviewers catch deliberately seeded errors.

Should an agent escalate based on its own confidence score?

Only as a supplement. Deterministic triggers checked in code, such as an amount limit, a new payee or a protected record, are more reliable because they do not depend on the model knowing when it is wrong. Model confidence can add escalations, but should never be the only gate on an irreversible action.

Does the EU AI Act require human approval of every agent action?

No. Article 14 requires effective human oversight of high-risk AI systems and explicitly requires that overseers be able to recognise automation bias. It does not require approval of every action. After the Digital Omnibus, the obligations for stand-alone high-risk systems apply from December 2, 2027.

What is the difference between human-in-the-loop and human-on-the-loop?

Human-in-the-loop means a person approves before the action happens. Human-on-the-loop means the agent acts and a person monitors, samples and can intervene or stop it. Most mature agent deployments use human-in-the-loop for a small set of irreversible actions and human-on-the-loop for everything else.