Verifying What AI Agents Do: Permissions, Approval Gates and Risk Tiers

For the past few years, most of the effort around generative AI went into asking better questions. As models gain the ability to call tools and change real systems, a different skill starts to matter more: checking what the AI actually did. This article explains why a wrong action is a different class of failure from a wrong answer, and gives you a practical framework for scoping permissions, placing human approval, and matching autonomy to risk when you build or adopt agents.

From systems that answer to systems that act

Early chat assistants were passive. You asked, the model replied, and you decided what happened next. An agent changes that division of labour. Given a goal, it can interpret the request, plan steps, call tools, take actions, look at the results and keep going without waiting for you between steps.

The difference is easiest to see with a mundane request. Ask a chatbot how to organise a messy set of folders and you get advice you can take or ignore. Give an agent permission to do the organising and it may inspect directories, rename and move files, create new folders, update records and delete what it considers duplicates. A chatbot recommends an action; an agent performs it. That single shift is why reliability has to be rethought when agents enter the picture.

Why a wrong action costs more than a wrong answer

When a model gives you bad information, you have several chances to catch it. You might notice something is off, search again, compare with another source or simply ignore the reply. The human sits between the output and any consequence.

In an autonomous pipeline, that buffer is gone. The flow below shows the typical shape: the agent reads something, decides, calls another system, changes state and moves on. Notice that nothing in the chain pauses for review.

Reads information
↓
Makes a decision
↓
Calls another system
↓
Changes something
↓
Continues automatically

A mistake in the first step propagates through every step after it. A misread input becomes a wrong decision, which becomes a wrong API call, which becomes a changed record that later steps treat as ground truth. This is why the useful question is shifting from how capable a model is to how safely it can be allowed to act.

Least privilege: scope what the agent can touch

Consider how you would onboard a new hire. On day one you would not hand them unrestricted access to production databases, customer records, financial systems, the source code and the deployment pipeline. You would grant what their role requires and expand access as trust is established.

Agents deserve the same treatment. Grant only the access a task needs, and separate reading from writing wherever possible:

• An agent that reads documents does not need permission to delete them.
• An agent that analyses source code does not need permission to deploy production builds.
• An agent that drafts emails does not need permission to send them.

The rule is short: an agent gets only the access its job cannot be done without. In practice that usually means narrow, purpose-built tools and short-lived credentials rather than a general-purpose token, so that the worst thing a confused agent can do is bounded by design rather than by its judgement.

Human approval for actions that matter

One of the most effective patterns is also one of the simplest: let the agent do the work, but require a person to approve consequential actions before they run. The sequence below splits the pipeline into a preparation phase the agent owns and an execution phase that only starts after a human has reviewed and approved the proposed change.

AI identifies issue
↓
AI proposes solution
↓
AI prepares changes
↓
Human reviews
↓
Human approves
↓
System executes

You keep most of the speed, because the agent still investigates, proposes and prepares, while judgement stays with a person at the point where it counts. The approval step works best when the agent presents a concrete, reviewable artefact, such as a diff, a list of records to be changed or a draft message, rather than a vague description of intent.

Not everything should need sign-off, or reviewers will start approving by reflex. Summarising a hundred documents can safely run unattended. The bar should rise sharply for actions like these:

• moving money
• deleting production data
• publishing something publicly
• deploying software
• changing security permissions
• contacting customers

If you are implementing this pattern, our walkthroughs of approval-gated agents with LangGraph interrupts and checkpoints and of state, approval and idempotency for acting agents cover the mechanics.

Verification as a core skill

Prompt engineering, and more recently context engineering, get most of the attention. Verification deserves as much: can you tell whether the result is actually right?

That depends on domain knowledge. A developer must be able to read generated code critically. A finance professional needs enough expertise to challenge generated calculations. A researcher has to check that cited sources exist and say what is claimed. A designer has to judge whether a generated layout solves the actual problem. AI can speed up execution enormously, but fast execution without verification mostly means making mistakes faster.

Matching autonomy to risk

Framing the question as "can AI be trusted?" with a yes or no answer is unhelpful. Trust should be granted per task, in proportion to what could go wrong. A simple three-tier model is a good starting point.

Low risk: let it run

Summarising meeting notes is a good example. The downside of a mistake is small and easy to spot, so a high degree of autonomy is reasonable.

Medium risk: automate, then check

Changing a software feature is a typical middle case. The agent can write the change, but automated tests and a code review should confirm the result before it ships.

High risk: require explicit authorisation

A request to transfer money to an account belongs here. Strong human authorisation should be mandatory, regardless of how confident the agent seems.

The principle generalises: verification should scale with the cost of being wrong.

More autonomy is not automatically better

It is tempting to assume that the most autonomous agent is the best one. A more useful question is what level of autonomy fits this particular task. Useful levels form a ladder:

• suggest only
• prepare the work for someone else to execute
• execute after confirmation
• operate fully automatically

Well-designed systems place each capability on the right rung instead of pushing everything to the top.

Questions to ask before connecting a tool

Developers are no longer wiring models only to a text box. Agents are increasingly connected to a wide surface of systems, such as the ones listed below.

APIs
Databases
Browsers
File systems
Cloud infrastructure
Development tools
Business applications
External services

Each integration adds a capability and, with it, a new way to fail. Beyond asking whether the agent can complete the task, work through these questions for every tool you expose:

• Which data and systems can it reach?
• Which of them can it change?
• If it breaks mid-task, what state is left behind?
• Is there a way to undo what it did?
• Who signs off on sensitive operations?
• Is every action logged in a way you can audit later?
• How do you stop it quickly?

These answers can matter as much as which model you pick. A useful rule of thumb is that anything irreversible and unlogged should never be reachable without a human in the loop.

AI literacy now includes risk literacy

Knowing how to get results out of an AI tool is no longer enough; you also need to understand where it breaks. That is not an argument for fear, it is what separates a capable user from a careless one. A mature user asks not only whether AI can do something, but whether it should do it without supervision.

Human judgement, machine execution

Agents can give one person reach that used to require a team, across research, analysis, coding, content, automation, data processing and operations. That extra capability comes with extra responsibility. A workable division of labour looks like this:

• Humans define the intent.
• The agent handles execution.
• The surrounding system enforces boundaries.
• Humans verify the outcomes that matter.

Key takeaways

• The first wave of generative AI was about talking to machines; the agent era is about delegating to them, and delegation depends on trust, boundaries, verification and accountability.
• Errors in autonomous pipelines compound, so design for containment: least privilege, reversible actions and complete logs.
• Place human approval on consequential, hard-to-undo actions, and let low-risk work run unattended so reviews stay meaningful.
• The defining question is shifting from what AI can do to what you should let it do on its own, and answering it well is becoming a core engineering skill.