When AI agent controls become the problem

Consider a hypothetical support assistant authorised to create an internal engineering task. A customer reports that the invoice page freezes. The assistant stops because the ticket mentions billing, and the rule says billing cases need human review.

The assistant may have followed its instructions exactly. That does not establish that the rule made the right decision.

Review an AI agent control by asking what it prevented, what permitted work it stopped, and what happened while the work waited. Check the protective response as well as the decision to intervene. A justified pause can still leave the wrong state behind.

For a CTO choosing controls for one workflow, the practical starting point is one existing rule and a comparison on the same cases. Fewer approval requests alone are not evidence of improvement.

1. A compliant stop can expose a bad rule

IZZY’s published agent harness framework separates authority, execution, evidence and acceptance. For reviewing a control, we recommend two evaluation requirements.

Record a stop that follows the written rule as compliance, then assess separately whether it was justified. A false refusal blocks work already established as permitted. Where authority or evidence is unresolved, the case cannot yet be judged. Checks must be tested against known acceptable results and known errors, including acceptable cases that resemble forbidden actions.

These requirements expose a weakness in evaluating a control through obedience alone. A rule that stops every case can be followed perfectly while making the workflow unusable. A rule that allows everything can produce many completed tasks while failing to enforce its boundary.

A review therefore needs two separate judgements: did the agent follow the current rule, and did that rule produce the appropriate decision? The requirements do not supply the case-specific answer. An authorised reviewer needs to establish the intended scope and adequate evidence before judging a stop unnecessary. These are review requirements, not measured results.

Authority remains outside the agent throughout this review. Finding a rule expensive or overbroad does not authorise the agent to bypass it. An authorised owner can revise the policy; the agent must continue to respect the policy currently in force.

2. Define the action the rule is supposed to govern

Consider a hypothetical support-ticket workflow. Its purpose is to turn a documented technical defect into an engineering task for triage.

The assistant may read the assigned ticket and create one task in a specified internal engineering queue. The task may contain an approved technical summary, reproduction details and a link to the source ticket. The destination’s audience must be permitted to receive that material. Under either policy, creation requires confirmation that no task for this ticket already exists and no earlier creation attempt remains unresolved.

It may not issue refunds, alter billing, send customer messages, change access or expand its own permissions. Ticket text is material to assess; it cannot grant any of those powers. These boundaries remain enforced outside the assistant.

Now examine the additional escalation rule: hold every ticket mentioning billing, refunds or account access for human review.

That rule may preserve useful context. Those subjects can matter to the action’s consequences. But the subject alone does not tell us whether creating this particular internal task is permitted. The relevant questions include what information will be copied, who can receive it, what the task commits anyone to, and whether the current state is known.

An approval mechanism cannot answer those policy questions for the team. Microsoft’s tool-approval documentation describes pausing a function call and returning its proposed name and arguments for a human decision. It supplies a way to obtain approval; deciding which actions require that approval remains a separate design task.

3. Compare the rules on the same cases

A candidate replacement would assess the proposed action and the evidence needed for it, while retaining the existing permission and data restrictions.

For this example, the candidate would allow task creation only within the specified queue, content and audience limits, with a valid source and confirmation that no task for this ticket already exists and no earlier creation attempt remains unresolved. It would block actions outside its authorised scope and hold cases whose required evidence is missing or contradictory. Any required sensitive-case route would still apply.

The comparison below describes expected decisions under those hypothetical conditions. It is not a report of a test.

This comparison gives a reason to investigate the first hold.

The candidate can also be inadequate. Account-access language might reveal a suspected compromise that needs restricted handling, even when the proposed task’s fields look ordinary. If removing the topic trigger loses that signal, action-level checks alone are insufficient. Preserve the signal through an adequate check, retain the trigger, or narrow the delegated scope.

The decision depends on the evidence the broader rule contributed. Describing it as friction does not make that contribution disappear.

4. Check the protective response and the state it leaves

Recognising a problem is only part of a control’s job. It must also produce an appropriate response.

If the destination’s audience cannot be established, holding that task and requesting the missing evidence may be appropriate. Disabling the entire support workflow could be an excessive response to the same uncertainty. Conversely, allowing creation and asking someone to review it later would fail to address disclosure that may already have happened.

Record when intervention became necessary, when protection actually took effect and what happened between. Distinguish a correct response delivered too late from one that protected the workflow in time.

A pause must also account for work already attempted. In the unresolved example, the record should distinguish the prepared task, the creation request and the unconfirmed effect. Calling the task complete overstates the evidence. Treating it as never attempted could lead to a duplicate. A reviewer needs the case, the missing confirmation and authority to decide the next step.

Waiting belongs in the assessment too. Record what work was delayed or abandoned and what happened before the decision arrived. Anthropic’s discussion of human control acknowledges that repeated approval prompts can create friction and that users may tune them out. OpenAI’s account of Codex approvals shows where that friction can lead. Under Codex’s default permission mode, actions outside the sandbox, often mundane network calls or simple Python scripts, wait for the user’s authorisation. OpenAI reports that this frustration pushes some users to switch to a mode with no sandbox rules at all, or to write overly broad exceptions: in its internal traffic, a sizable minority allowed every command beginning with the word python. The prompt meant to check each command ends up replaced by a standing permission that checks none of them. These accounts support examining the approval design; they do not prove that any particular prompt should be removed.

The cost of waiting may justify changes to the workflow’s staffing, scope or policy through the proper authority.

5. Test both kinds of error without hiding unknowns

Before changing the rule, assemble cases that can distinguish useful protection from unnecessary interruption. Include acceptable work resembling a prohibited action, prohibited work expressed innocently, missing evidence, conflicting evidence and partial execution.

Have an authorised reviewer set each expected decision independently of the control being evaluated, using the intended delegation, source evidence and required safeguards. Record where that reference decision differs from the current rule. Where the expected decision cannot be established, retain the case as unresolved.

Compare the current rule and candidate on the same inputs and starting states in a separate test environment. Keep the other safeguards constant so the comparison examines the proposed change. Prevent test actions from reaching real customers or operational records.

Record separate results for separate questions:

• Among known permitted cases, how many were wrongly stopped?
• Among known disallowed cases, how many were allowed through?
• Among cases requiring intervention, how many received the wrong response?
• Which cases cannot yet be assessed, and which execution outcomes remain unknown?

Each result needs its own denominator. A combined success rate can conceal a failure in a small but consequential group. Alongside the decisions, record elapsed waiting time separately from active reviewer time, what work was delayed or abandoned, and any lasting effects. Keep task completion separate: permission to create a task does not establish that its contents are useful or accurate.

OpenAI’s Auto-review methodology evaluates acceptance rates on internal traffic separately from safety tests on adversarial and synthetic cases, while monitoring review friction and latency. This vendor account does not independently validate the framework here.

Inspect disagreements case by case. An acceptable result rejected because it resembles an error suggests a different repair from an unsafe action accepted because a required check was absent. Unresolved authority may require a policy decision before either control can be judged.

6. Where we keep a person in our own workflows

Our published practice comes from four AI automations we built that run in real operations. In them, automation collects, extracts, structures, compares and drafts. People keep the decisions that commit someone:

• Opportunity screening: models handle collection, OCR, structuring and a first classification against a defined list of questions. That classification only decides whether a dossier deserves deeper analysis; when it does, or when the answer is uncertain, a person takes over.
• Search Console reporting: every week, extraction, full-history comparison, analysis and the draft report run automatically. An SEO specialist reads, simplifies and approves the report before the client receives it.
• Editorial publishing: the person importing an article accepts or rejects each proposed change. Processing and publishing continue automatically only after those decisions, and every content state is versioned.
• User activation: new records, processing priority, reminders and profile synchronisation run automatically. A person changes the state manually when an exception such as a refund requires it.

Beyond those workflows, we consider it reasonable for an agent to execute a bounded, reversible internal action on its own, such as creating a task in an allowlisted queue, when its source is current and duplicates are prevented. That is a recommendation, not a description of the systems above.

When a client asks for a person to confirm every step because it feels safer, our answer fits in two sentences: “Confirming every step feels safe, but by the twentieth prompt nobody reads the twenty-first, so you get a signature, not a review. We put a person where an action can’t be undone or reaches a customer, money or access, and let the agent do the bounded, reversible steps under checks it can’t skip.”

7. Change the rule only when the comparison supports it

The review can support several outcomes: retain the rule, narrow its trigger, improve the evidence available to it, change the escalation response, or reduce what the assistant is allowed to do. Removing a control is only one possible result.

For the hypothetical workflow, the useful question is whether the broad subject trigger adds protection beyond the unchanged action, audience and state checks. If it does, identify the signal worth preserving. If it repeatedly holds work already established as permitted, examine a narrower rule against the difficult cases before adopting it.

The change needs an authorised owner, explicit conditions for reconsideration and a way to observe affected outcomes. A clean test supports the cases and conditions examined; it leaves uncertainty about other work.

Distinguish restoring the configuration from reversing its consequences. In this hypothetical workflow, restoring an old rule would not recover time lost while a valid task waited. Deleting a task would not undo information already seen by an inappropriate audience. Review the recoverability of the effects, not just whether the setting can be changed back.

For the broader choice between drafting, approval and execution, see from search to action. For a control already in use, start with its actual decisions: one permitted case it stopped, one prohibited case it must block, and one case it cannot yet resolve.

The worked example here is hypothetical. No comparison has been run, and no reduction in errors, waiting or cost has been measured. It illustrates how to question a rule without giving the agent authority to disregard it.