Writing

How to audit an AI agent.

An audit asks five things of any AI agent: what it was allowed to do, what it did, who approved it, whether it still does what was tested, and whether it can be stopped. Here is the evidence that answers each.

cvlSoft6 min read

An auditor does not ask whether an AI agent is smart. They ask whether the organization can show what it did and why. Most teams find out they cannot at the worst moment: after a customer disputes an outcome, or a regulator asks about a decision made months ago.

An audit of an AI agent comes down to five questions. For each one, the useful thing is not the answer but the evidence behind it, and whether that evidence existed before anyone asked.

1. What was it allowed to do?

The first question is about limits, and the evidence is a policy that was in force at the time of the action. “We told the agent not to” is an instruction, not a control. A control is a check that runs before each action touches a system of record, and that fails the action when the policy says no.

Ask to see the policy, who changed it last, and whether it applies the same way on every channel the agent works through. If a phone call and a scheduled job follow different rules, there are two agents, and two audits.

2. What did it actually do?

The evidence here is a record of every action, its inputs and its outputs, written as it happened and not editable afterwards. Logs that a team can tidy up are a reconstruction, not a record.

A good test is to pick one case at random from three months ago and ask for the full sequence: what was seen, what was decided, which systems were touched, in what order. If the answer takes a week of digging, the record does not exist in a usable form.

3. Who approved it?

Some steps should never run on the agent's judgment alone: a payment, a coverage decision, a message that commits the company. The evidence is a named person who saw what had been done and what was proposed next, and whose decision is recorded next to the action it authorized.

Look for two failures. Approval gates that cover everything, so people click through without reading. And approval gates that cover nothing, because the step was judged low risk at design time and never revisited.

4. Does it still do what was tested?

An agent can keep working while its behavior moves away from the system that was tested. Policies change, tools change, the conditions around it change, and nothing raises an alarm because the agent is still producing answers. We wrote about this in the agent lifespan problem.

The evidence is measurement against intent, on an ongoing basis: outcomes compared with what the workflow was meant to achieve, with thresholds that trip on their own when cost, error rate or scope leave the band that was agreed. A test report from launch day is history, not assurance.

5. Can it be stopped?

The last question is the one people skip. Can one execution be halted while it runs? Can everything be halted at once? Who can do it, and how fast?

The evidence is a control that has been used, not one that exists on a slide. Ask when it was last exercised.

What this means for how agents are built.

None of the five answers can be added afterwards by a review process. Limits, records, approvals, measurement and a stop control have to sit in the path of every run. That is the design of AIOS: every action is policy-gated, a person signs the steps that matter, every action, input and output goes to an append-only ledger, and kill switches and circuit breakers can halt one run or a whole tenant. Our page on AI agent governance sets out the controls in the order a run meets them.

Whatever you are running, the five questions travel. Ask them before someone else does.

Your processes. Autonomous. Guaranteed.

We embed until it works, then you pay for what worked. Bring the process you would most like to stop staffing.

More writing