# Script, workflow or agent: how much to automate

Eight rungs from doing it by hand to an unattended agent, five questions to pick the lowest rung that does the job, and the tasks where a plain script beats a model.

Guides · v1.1 · draft · https://hayzlabs.com/guides/script-workflow-or-agent/

In short:
- Pick the lowest rung that does the job; many agent tasks need only a script, or a script with one model call.
- Use a plain script when the answer must be exact or the same every run, such as arithmetic, dates, sorting or renaming.
- Move up only when the lower rung has shown it fails, and confirm each action that sends, pays or deletes.

## Short answer

Pick the lowest rung that does the job. Many tasks handed to an agent need less: a plain script, or a script with one model call in the middle. Higher rungs cost more tokens, are harder to check and fail in less predictable ways. The ladder, the five questions and the examples below are our suggestion and untested.

## Why start low

Anthropic describes five workflow patterns plus autonomous agents, and advises "finding the simplest solution possible, and only increasing complexity when needed" ([Anthropic, 19 Dec 2024](https://www.anthropic.com/engineering/building-effective-agents), vendor guidance).

Each rung up costs more and repeats less: Anthropic found agents use about four times the tokens of a chat and multi-agent setups about fifteen times ([Anthropic, multi-agent research system](https://www.anthropic.com/engineering/multi-agent-research-system), vendor report), and on one open model 1,000 runs of one prompt at temperature 0 produced 80 different completions ([Thinking Machines](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/), vendor test).

## The ladder

| Rung | What it is | Use when | Check by |
|---|---|---|---|
| 0 | Do it by hand | You will do it fewer than about ten times, or it takes judgment | You |
| 1 | Checklist or template | The steps are fixed and a person must decide | Ticking the list |
| 2 | Script, no model | The input format is fixed and the rule can be written down | Tests; the same input gives the same output |
| 3 | Script with one model call | One step needs reading language: classify, extract, summarise | A fixed output format, plus a spot-check of a sample |
| 4 | Workflow: a fixed chain of model calls with a check between steps | Several language steps, in a known order | The check after each step |
| 5 | Agent with read-only tools | The steps are not known in advance, and nothing it does can change anything | Read its sources |
| 6 | Agent that can act, confirming each risky action | It must send, buy or edit | The confirmation prompts and its logs |
| 7 | Agent running unattended in a sandbox | A long job that cannot damage anything outside the box | Review the result afterwards |

Some products call a fixed workflow an agent. Judge by who picks the next step: your code or the model.

## Five questions, in order

1. **Can you write the rule down in under a page?** If yes, start at rung 2. If not, start at 3 or higher.
2. **Is the input free text, images or messy formats?** If yes, you need a model somewhere, so rung 3 at least.
3. **Do you know the steps in advance?** If yes, rung 3 or 4. If not, rung 5 or higher.
4. **Does any step change something outside your machine, such as sending, paying, deleting or publishing?** If yes, rung 6 with a confirmation for each risky action, or keep a person on that step. Before you give an agent your email, files or accounts, read [Letting an agent near your email, browser and files](https://hayzlabs.com/guides/agents-near-your-email-and-files/).
5. **Can you check the output cheaply?** If not, drop a rung or add a check between steps.

> [!OUR RULE]
> Start at the lowest rung where all five answers fit, and move up only when you can show the lower rung failed, such as a pile of cases the script could not handle that you will not review by hand. A feeling that it should be smarter is not enough. If you have re-prompted the same fix three times, step down a rung or do it by hand.

Try the five questions on a task of your own:

<!-- WIDGET rungPicker -->

## When a script beats a model

Use a script when the answer must be exact or the same every run:
- exact arithmetic, dates and sorting;
- removing duplicates, renaming files and converting a format, such as CSV to JSON;
- anything that needs an audit trail, such as a task with a legal or money consequence;
- very high volume, where even a small cost per item adds up.

A model beats a script when the input is language that varies (emails, notes, receipts photographed at angles), when the rule has many exceptions, or when good enough most of the time is acceptable and errors are cheap to catch.

Try a script with one model call and a check after it first. A model can write the script for you. This prompt asks for one that makes no model calls and refuses to guess:

```text title="Build the script version · v1.0"
Here are <N> example inputs and the outputs I want. Write a Python script
that does this with no AI calls. If a case cannot be handled by fixed
rules, have the script write it to needs_review.csv instead of guessing.
Include 5 tests using my examples.
Examples:
<EXAMPLES>
```

Replace the following:
- `<N>`: how many examples you are pasting, such as `5`.
- `<EXAMPLES>`: each input and the output you want from it. Leave out anything you should not paste: [What not to paste](https://hayzlabs.com/start/what-not-to-paste/).

Run the script on a copy of your files first, and read the tests before you trust them.

## Four tasks, four rungs

An illustration: a small online shop owner with four jobs.

- **Rename 400 product photos to a fixed pattern.** Rung 2. The rule fits on a page and the steps are known. Run it on a copy and check the tests; the same folder always gives the same result.
- **Sort 400 receipts into categories.** Rung 3. A script handles the files, makes one model call per receipt with a fixed list of categories, and writes the answer to a table. Spot-check a sample and read every receipt the model marks unsure.
- **Plan a supplier visit abroad.** Rung 5. The steps are not known in advance, so an agent with live search and read-only tools suits it. Check each place and price against the official page.
- **Clear the inbox.** Rung 6, held back to drafts. Sending changes something outside the machine, so the agent drafts replies and a person sends them.

## What each rung costs to check

Each step up makes the check longer. A script check is a test you run once. A rung 3 check is a sample you read each time. Above that you are checking a process and not a result, and you may need logs. If you cannot say how you would check the work at a rung, it is the wrong rung for now. For checking agent output, see [Checking agent work](https://hayzlabs.com/workflows/checking-agent-work/).

## How to tell it worked

- You can name the rung you chose for the task.
- You can give the reason in one line, using the five questions.
- You can name the failure that would move you up a rung, such as a pile of cases the script sent to needs_review.csv that you will not review by hand.
- It has not worked if you picked a rung because it sounded smarter. Go back to the five questions.

## Limits

> [!LIMIT]
> The three sources are vendors, and two are about their own systems. Rules rot: a script that reads a bank statement breaks when the bank changes its layout, and a model may cope where the script cannot. Models improve, so the line between rungs moves; re-check a task you wrote off. A structured output tells you the shape of an answer and nothing about whether it is right.

## Try this next

1. Take one repeated task from this week and answer the five questions. (10 minutes)
2. If it lands on rung 2 or 3, ask for the script with the prompt above. (20 minutes)
3. For a big job that needs an agent, see [Long agent jobs: pieces, milestones and stop rules](https://hayzlabs.com/workflows/long-agent-jobs/).

## Sources

1. Anthropic. Building effective agents, 2024. Vendor guidance. [Link](https://www.anthropic.com/engineering/building-effective-agents)
2. Anthropic. How we built our multi-agent research system. Vendor report. [Link](https://www.anthropic.com/engineering/multi-agent-research-system)
3. Thinking Machines. Defeating nondeterminism in LLM inference. Vendor test on one open model. [Link](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/)