Earlier in this series the argument was: do not let the model be the only thing standing between an agent and an irreversible action. The knowledge-base chapter gives that argument a mechanism. Before a dangerous tool runs, a callback pauses and asks a person.
DANGEROUS = {"delete_file", "send_email", "execute_sql"}
def approval_callback(context, tool_call):
if tool_call.name not in DANGEROUS:
return None # safe: run it
if input(f"Approve {tool_call.name}({tool_call.arguments})? (y/n) ") == "y":
return None # approved: run it
return "Skipped: the operator declined." # denied: do not run it
That is a genuine control, and it is the right shape. It also has a failure mode that turns it into decoration, and knowing that failure is most of using it well.
Why the model cannot approve itself
Start with why a human is in the loop at all, rather than a clever instruction telling the model to be careful.
A tool result is untrusted input, from earlier in this series, and so is a retrieved chunk from a knowledge base. Content the agent read can steer what the agent does next. So an instruction to "only delete files the user explicitly named" lives in the same buffer as text an attacker might have planted in a document the agent retrieved. The model weighs both. The gate exists precisely because the model's own judgement is the thing under pressure, so the model cannot be the one to approve passing it.
The human sits outside that buffer. Nothing an agent retrieves can reach the person deciding y or n. That external position is the whole value of the control.
The failure mode: approval fatigue
Here is where a real control decays into a fake one.
A gate that fires on every tool call, or fires with too little information, trains the person to stop reading. The tenth identical prompt in an hour gets a reflexive y. The hundredth gets a y before the message has finished rendering. At that point the dialog still appears, the log still records an approval, and no actual decision is being made. The control is theater: it looks like oversight and performs none.
The security name for this is overwhelming the human in the loop, and it is a recognised threat rather than a usability nitpick. An attacker who can make an agent generate many approval prompts, or bury one dangerous request in a stream of benign ones, is exploiting the same human limit, deliberately.
A gate that asks constantly trains a reflex yes. The dialog remains; the decision is gone.
Designing a gate that stays a gate
The fix is to make each approval rare, informative and hard to reflex through. None of it is exotic; it is the discipline that keeps the human's attention worth having.
| Control | What it protects |
|---|---|
| Gate by consequence, not by tool count | Ask only where an action is irreversible or high-impact, so prompts stay rare enough to read |
| Show the full action, in plain terms | "Delete prod_backup.sql" not "execute delete_file", so the person can actually judge it |
| Make deny the easy default | The safe choice should be the low-effort one, so fatigue fails safe, not open |
| Batch and rate-limit prompts | A flood of approvals is itself the attack; refuse to generate one |
| Log the decision, the actor and the reason | An approval without a record is not accountability |
| Keep a hard code boundary underneath | For the truly irreversible, a person is a second gate, not the only one |
Rare, informative, deny-by-default. The gate stays a decision instead of a reflex.
The last row matters most. Human approval is a strong control and a fallible one, because humans tire. For an action you genuinely cannot undo, the sturdiest design pairs the human gate with a code boundary the model cannot reach past at all, so a mis-click and a hardcoded limit are two separate lines of defence rather than one.
What this touches
For the frameworks, this sits on the MAESTRO Agent Frameworks layer. The failure mode maps to T10 Overwhelming Human in the Loop, which the pack defines as targeting human oversight by exploiting cognitive limitations. The gate itself defends against T2 Tool Misuse, an agent driven to use a tool in an unintended way. In the OWASP Agentic Top 10 the anchors are ASI09 and, for the tool side, ASI02.
On the regulatory side, framed as scope rather than a citation: human oversight and authorisation of significant actions are obligations across GCC rulebooks, written for processes where a person really does review each item. A rubber-stamped approval satisfies the letter and defeats the purpose, so a gate designed to stay meaningful is how you meet the intent, not just the checkbox. Which obligations apply depends on your regulator and architecture, and that mapping is an advisory estimate until someone checks it properly.
The part worth keeping
Human-in-the-loop is one of the strongest controls available for an autonomous agent, and building it is the right instinct. Nothing here argues against the gate.
The argument is narrower: a gate is only a control while the human behind it is still deciding. Keep the prompts rare, make them legible, let deny be the easy path, and back the irreversible cases with code. Do that and the gate stays a decision. Skip it and you have built a dialog box that logs the word yes.