Checking agent work: review loops and work you didn't watch

Use it when an agent has done a job while you were not watching, such as editing files, updating records or writing code, and you are about to rely on the result.

Not when you watched every step, or the result is easy to see at a glance. For an ordinary answer in a chat, Check before you trust is enough.

You need the agent's log or report, access to the files it changed, and about 24 minutes for a batch of 25 items. The steps, times and numbers on this page are our suggestion.

Example (invented). Mei asked an agent to tidy 120 customer records and draft replies to the 15 oldest support tickets.

Where to check, and how much

You can check in three places.

1. After the plan. Before the agent starts, read what it says it will do. A wrong plan is the cheapest thing to fix.
2. After the first slice. Let it do one piece, such as ten records, and check that before the rest.
3. At the end. Check the finished work with the steps below.

For a long job, the first two are the milestones in Long agent jobs. How much you check depends on the stakes:

For Mei: the 120 records are medium, so she checks 20% of 120, which is 24. If the records also carry internal tags, those are low, so 12. The 15 customer replies are high, so she reads all 15.

Steps

For step 1, paste this into the agent's session when the job ends, or add it to the end of the brief:

Before I review this, write a short hand-back report on the job:
<TASK>. Use three headings:
1. Assumed: everything you assumed that I did not tell you.
2. Skipped: every item or step you did not finish, and why.
3. Unsure: every item where you were not certain of the result.
For each line, name the file or item it applies to. Do not summarise what
went well. If a heading is empty, write "none" and say how you checked.

Replace the following:

• <TASK>: the job in one line, such as tidy 120 customer records and draft replies to the 15 oldest tickets.

Read the report as a list of claims, not as a result: steps 2 to 4 test it.

How to tell it worked

• You can say what the agent assumed, and which of its claims you confirmed for yourself.
• The sample count, the decision and your minutes are written down.
• It has failed if you accepted the work because the report sounded sure. Go back to step 2 and open the files.

Two variations: for a job with a test suite, step 2 is running the tests, but passing is a start and not the end, so still do steps 3 and 4 (Coding agents for people who don't write code has the evidence on tests that pass). For work that is not code, such as records, documents or emails, the steps are the same: replace "tests" with the checks in your brief.

Test a workflow before you rely on it

The steps above check one run. If you will run the same workflow, prompt or agent again and again, also test it once before you rely on it, and again after any change to the workflow, the prompt or the model. Allow about 45 minutes (our estimate).

1. Describe it in one paragraph: what comes in, which tools it uses, and what it can read, write, send or delete. Then run the private data, outside content and send check in Letting an agent near your email, browser and files, and redesign first if it has all three.
2. Imagine it has failed. It is six months from now and the workflow caused a real problem. Write your own 10 stories of how before any model sees the description, then, if you like, ask a model for more with the prompt below. Keep the five most likely and costly.
3. Build 2 or 3 attack inputs for each, starting from the table below, and write down the safe behaviour you expect before you run them.
4. Run them on copies and test data, and keep each output.
5. Fix each failure: add a check, cut a permission or put a person on an approval. Or write a one-line accepted-risk note: the risk, how likely and how costly (low, medium or high), why you accept it, and a date to look again.

Here is a workflow I run with AI: <DESCRIPTION>.
Assume it is six months from now and this workflow caused a serious problem
(wrong information sent, data exposed, money or time lost, someone misled).
Write 10 short, specific stories of how that happened. For each, name the
exact input or condition that triggered it and the step that should have caught it.
Do not suggest fixes yet.

Replace the following:

• <DESCRIPTION>: your paragraph from point 1: inputs, tools and permissions.

Five inputs to start with. No study ranks these.

Others to add: a very long input, a scanned page, a request outside its job, a made-up citation, and mixed units or currencies.

Example (invented). An assistant reads supplier invoices from a shared inbox, copies the total and due date into a payments sheet, and drafts a reply when something looks wrong. The check in Letting an agent near your email, browser and files finds all three parts: invoices hold bank details, they come from outside, and the assistant can send replies. The owner changes it to drafts only, sent by a person. The five cases:

The fixes: a missing date is written as MISSING, email text is treated as data, and terms come from the contract table. The draft-only change had already stopped the bank details from leaving.

The test has failed if you wrote the cases after watching the workflow work. Go back to point 2. To compare models on your own tasks rather than test one workflow, see Test any new model on your own tasks.

Why it works

• When 4 maintainers reviewed 296 AI-generated pull requests that passed SWE-bench tests, only about half would have been merged, against 68% for human-written fixes: about 24 points below the benchmark score (METR note, March 2026, field study). That is why step 3 looks beyond the tests. The study is about code; we read the lesson as carrying to other work, but it does not test that.
• On real freelance projects, automated judges overestimated success by 2.3x to 2.9x (CAIS, benchmark preprint). That is why a person, and not an automated judge, checks what a client will see.
• The better a model recognises its own writing, the more it favours it when judging (a linear link) (Panickssery et al., NeurIPS 2024, peer-reviewed benchmark). That is why step 5 uses a different checker.

Limits

Sources

1. METR. Many SWE-bench-passing PRs would not be merged into main, 2026. Field study. Link
2. CAIS. Significant increase in digital labor automation, 2026. Benchmark preprint. Link
3. Panickssery et al. LLM Evaluators Recognize and Favor Their Own Generations, NeurIPS 2024. Peer-reviewed benchmark. Link

Changes

1. v1.010 Oct 2026first version