The Escalation Problem
An agent that never asks for help will eventually do something wrong and costly. An agent that asks for help constantly is just a chatbot with extra steps. The sweet spot is an agent that handles routine work autonomously and escalates when it genuinely needs a human, which is harder to implement than it sounds.
The difficulty is that the agent needs to assess its own confidence, estimate the risk of proceeding versus the cost of interrupting a human, and make this judgment in real time across diverse situations. There's no universal threshold that works for every task.
Confidence-Based Escalation
The most common pattern ties escalation to the agent's confidence in its plan or output. If the agent is confident it knows how to proceed and that the outcome will be correct, it proceeds autonomously. If confidence drops below a threshold, it escalates.
The practical challenge is calibrating confidence. Models are notoriously bad at self-assessment, sometimes confident when wrong and uncertain when right. Supplementing model confidence with external signals helps: does the plan match known patterns? Are the tools returning expected results? Does the output pass validation checks? These objective signals are more reliable than the model's subjective confidence.
Risk-Based Escalation
Even when the agent is confident, some actions are too risky for autonomous execution. Deleting data, sending external communications, making financial transactions, deploying code to production: these should require human approval regardless of the agent's confidence because the cost of getting it wrong is high.
This maps to the guardrail concept: certain action categories always escalate. The agent's confidence determines whether it escalates in the gray areas; the risk level determines whether it escalates in the clear cases.
Designing the Escalation Experience
When an agent escalates, the quality of the escalation matters. "I need help" is useless. "I'm trying to update the user's billing plan, but the API returned an error I don't recognize. Here's the error, here's what I've tried, and here are the options I see" is useful. Good escalation provides context, shows the agent's work so far, and offers concrete options for the human to choose from.
The human should be able to respond quickly. If escalation requires reading through pages of context, it's too costly. The agent should summarize the situation concisely and present clear decision points. Agent frameworks that handle escalation well make this easy to implement.
Reducing Unnecessary Escalations
Track escalation patterns to find opportunities for automation. If the agent escalates the same type of question repeatedly, you can probably add a rule or capability that handles it. If 30% of escalations are "I don't have permission to do X," maybe the agent needs broader permissions (with appropriate guardrails). Search for agent analytics tools that can surface these patterns.