AI Agents & Automation in Practice
When Should an AI Agent Escalate to a Human?
By Mudabbir Hasan, Co-founder · Published · Last updated
Escalate based on what an action can break, not only on how confident the agent feels. Sort actions into risk tiers by reversibility: let read-only and reversible steps run with logging, route actions that touch outside systems through review or confidence checks, and require human approval for irreversible ones such as moving money. Add sentiment and looping signals on top, and log every handoff.
Why is a confidence score not enough to decide when to escalate?
Most teams start with a threshold: if the model is less than some percent sure, hand the case to a person. It is easy to build, and it fails in two directions. A confident agent can still be wrong about something that cannot be undone, and an unsure agent can be asked to escalate a harmless lookup that a person would approve without reading.
Digital Applied, in its 2026 escalation guide, says to distrust the raw confidence number and to assign oversight by reversibility and blast radius instead. It also says the approval rules belong in the workflow definition, not something the agent negotiates at runtime. That second point matters in practice: a rule the agent can talk itself out of is a suggestion.
What are the four risk tiers for agent actions?
The Digital Applied guide proposes four tiers. They are the clearest starting point we found, and the framework is the authors' own synthesis, so treat the thresholds as defaults to tune on your own process.
- Read-only: lookups, retrievals and analysis with no side effects outside the system. Run these fully autonomously, because gating them only causes confirmation fatigue.
- Reversible: drafts and internal changes that can be undone. Run autonomously, and log enough detail to reverse and audit each one.
- External or third-party: actions that touch outside systems. Use a staging queue or confidence routing.
- High-risk or irreversible: deploys, money movement, deletions, privilege changes and external communications. Mandatory human approval, no exceptions.
Four risk tiers for agent actions
Read-only
Fully autonomous. Gating adds only confirmation fatigue.
Reversible
Autonomous, with logging so each action can be undone and audited.
External or third-party
Staging queue or confidence routing.
High-risk or irreversible
Mandatory human approval, no exceptions.
Which signals should trigger a handoff?
The same guide lists six triggers, which combine well because each catches something the others miss: a confidence threshold breach, a match on the action-risk tier, a frustration or sentiment signal, an approaching SLA breach, an irreversibility flag, and an anomaly or prompt-injection signal. Tier 4 actions and injection signals call for a synchronous stop. Lower-risk breaches can queue for asynchronous review so the customer is not left waiting on a person.
Looping is worth adding to that list. An agent that retries the same step several times has told you it is stuck, whatever its confidence says. Treat a retry limit as a trigger like any other.
Six triggers that combine well
Confidence threshold breach
Action-risk tier match
Frustration or sentiment signal
Approaching SLA breach
Irreversibility flag
Anomaly or injection signal
Which human-in-the-loop pattern fits which action?
Dataiku describes four patterns, and production systems usually layer more than one:
- Interrupt and resume: the agent pauses at a preset point and waits for a person to approve, change or reject its proposed action.
- Human as a tool: the agent calls a person as one of its resources when it is unsure, then continues with the answer.
- Approval flows: a policy engine checks each action against rules and sends the ones that need sign-off to the right reviewer by role, action type and risk.
- Fallback escalation: below a confidence threshold, the agent notifies a person asynchronously instead of stopping, and may continue with a safe default.
A reasonable mapping is approval flows for Tier 4, interrupt and resume for Tier 3 actions that cannot wait for a queue, and fallback escalation for conversational work where stopping the dialogue would be worse than a slightly cautious reply.
What does a good handoff contain?
The person receiving the case should not have to re-read the whole conversation. Pass a short summary of what was asked, what the agent already tried and why it stopped, with the full history one click away. Log every handoff and escalation with a timestamp and the trigger that fired, so a reviewer can later tell whether the rules are too tight or too loose.
Put a timer on every pending approval. If nobody responds in the window you set, the request should escalate to someone else or the action should be cancelled safely. Digital Applied suggests a 30-minute approval window as a default in one open protocol it cites; pick your own number from how quickly your reviewers can realistically answer.
A handoff, step by step
- 1
A trigger fires
Confidence, tier, sentiment, retry limit or anomaly.
- 2
The agent stops and summarises
What was asked, what it tried, why it stopped.
- 3
The right person is notified
- 4
An approval timer starts
No answer in the window: escalate, or cancel safely.
- 5
The outcome is logged
With the trigger that fired.
How does this look in a shipped system?
Our public case study on autonomous support and operations agents describes the same shape: specialist agents that hand work to one another, escalation to human staff the moment a case exceeds what an agent should resolve alone, and a logged, auditable trail of every handoff. Human staff stay the last word on complex cases. The engagement is confidential, so the case page is deliberately limited to what the client has allowed us to say.
Read the case study: multi-agent support with human escalation
What should you do before you build the escalation layer?
- List every action the agent can take and assign each one a tier by reversibility.
- Write the approval rules into the workflow, outside the model's prompt.
- Choose triggers for each tier, and add a retry limit.
- Decide who receives each handoff and what happens when nobody answers.
- Log every handoff, then review the log weekly for the first month and tune the thresholds.
If you are deciding where an agent fits in an existing process, a short technical discovery is the cheapest place to settle the tiers before any code is written.
Sources
- Human-in-the-Loop AI Agents: design patterns for governance, Dataiku (accessed 2026-10-10)
- Human-in-the-Loop Escalation Design for AI Agents 2026, Digital Applied (accessed 2026-10-10)
Related guides
Talk to an engineer about your project
Start with a free project call. We learn what you are building and recommend the right next step, with no pitch.
Book a call