# When not to use AI

A six-question card to read before you open the chat box. Any no means doing the task yourself first, then using the assistant as a reviewer, or skipping it.

Guides · v1.1 · draft · https://hayzlabs.com/guides/when-not-to-use-ai/

In short:
- Before you open the chat, check that you can verify the result faster than doing it and can spot a subtle error.
- Also check that you need no skill from doing it, have seen it work, are on one task, and only you would fix a bad result.
- If any answer is no, do the task yourself first and use the assistant as a reviewer, or skip it.

## Short answer

Before you open the chat, ask six questions. If any answer is no, do the task yourself first and use the assistant to review your work, or leave it out. It takes about a minute. The six questions are our suggestion; no study tests them together.

## Why the task decides

In a 2023 field experiment with 758 BCG consultants using GPT-4 (the AI behind paid ChatGPT at the time), those on tasks that suited AI completed 12% more tasks, 25% faster, with over 40% higher quality. On a task just outside its abilities, they were 19 points less likely to be right (about 60 to 70% against 84% without AI) ([Dell'Acqua et al.](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321), field experiment). We do not claim the same gap for today's AI tools; the point that carries over is that the task decides.

Over-trusting automation is most likely when you are juggling other work. It affects experts as much as novices, and practice alone does not cure it ([Parasuraman and Manzey, 2010](https://trid.trb.org/View/1085175), review).

## The card

Yes to all six means go ahead.

| # | Ask yourself | Why it matters |
|---|---|---|
| 1 | Can I check the result faster than I could do the task? | If checking takes as long as doing, the assistant saves nothing, and an unchecked result moves its cost to someone else. |
| 2 | Do I know the subject well enough to spot a subtle error? | A result you cannot judge is one you can only trust. The consultants on a task just outside the tool's range were less often right than those without it. |
| 3 | Is the finished result all I need, with nothing for me to learn by doing it? | If the goal is to learn the skill, the assistant can take away the practice: [Learning without losing the learning](https://hayzlabs.com/guides/learning-without-losing-the-learning/). |
| 4 | Have I seen it work on this kind of task before, and written it down? | The same tool does well on one task and badly on another. A short list of what has worked and failed for you is a better guide than a feeling. |
| 5 | Am I working on this one task, and not juggling other threads? | Over-trusting automation is most likely when you are juggling other work. |
| 6 | If the result is wrong or thin, will I be the only one who has to fix it? | Unchecked output costs the person who receives it, in time and in what they think of you. |

> [!OUR RULE]
> If any answer is no, do the task yourself first and then use the assistant as a reviewer, or skip it. Ask it to criticise your draft and not to write one: [Getting disagreement instead of agreement](https://hayzlabs.com/guides/getting-disagreement-instead-of-agreement/). If only part of the task passes, use the assistant for that part.

A copy for your notes:

```text title="Before I open the chat · v1.0"
Task: <TASK>
1. I can check the result faster than I could do it: yes / no
2. I can spot a subtle error in this subject: yes / no
3. The result is all I need; I do not need to learn the skill: yes / no
4. I have seen it work on this kind of task, and noted it: yes / no
5. I am on this one task, not juggling other threads: yes / no
6. If it is wrong or thin, I am the only one who fixes it: yes / no
Any "no": do it myself first, then use AI as a reviewer, or skip it.
```

Replace `<TASK>` with the task in a few words, such as `feedback note for a colleague`.

## Three tasks through the card (illustrative)

A team lead has three tasks on a Friday afternoon:

- **Tidy the meeting notes.** Yes to all six. He checks the result against his own memory in two minutes, and nobody else depends on it. He goes ahead.
- **Summarise a report from a field he does not know.** No to question 2, because he cannot tell a subtle error from a correct summary, and no to question 4, because he has no record of the tool on this kind of task. He reads the report's findings and limitations himself first, then asks the assistant what he may have missed, and checks every point against the text.
- **Feedback for a colleague who has had a hard quarter.** No to question 3, because the thinking is the task, and no to question 6, because his colleague will be the one to read a note that sounds hollow. He writes it himself and asks the assistant only whether any sentence could be read harshly.

## When a no is not the end of it

- **A no to question 1 or 2 can be fixed.** If you can get a second check from a person who knows the subject, or by testing the result against a source, the answer may become yes.
- **A no to question 5 can wait.** Come back to the task when you are on one thing.
- **A no to question 6 is a prompt to read your work first.** If someone else will receive the result, read all of it before you send it.
- **A yes to all six is not a guarantee.** It says AI is a reasonable choice for the task. The result still needs a check that fits its stakes: [Check before you trust](https://hayzlabs.com/start/check-before-you-trust/) is the short version.

## How to tell it worked

- You ran the card before opening the chat, not after the result came back.
- For every task that got a no, you did a first version yourself and used the assistant only to review it, or skipped it.
- Your list of what has worked and failed has a new line, so question 4 is easier to answer next time.
- Nothing you sent to someone else this week needed fixing because of an unchecked AI result.

## Limits

> [!LIMIT]
> The card is a rule of thumb of ours and no study tests it. The consultant experiment used GPT-4 in 2023, models change, and the line between inside and outside a tool's range moves, so keep your own list and update it. The review of over-trust is about automation in general. For health, money or legal questions, where a wrong answer can hurt, the card is not enough: ask a qualified person.

## Try this next

1. Next time you reach for the chat box, run the card on the task. (1 minute)
2. Start a short list of tasks where the assistant helped and tasks where it did not, one line each: `<DATE> · <TASK> · helped / did not · <WHY>`. [Where could AI help this week?](https://hayzlabs.com/tools/where-could-ai-help-this-week/) is a place to start. (5 minutes)

## Sources

1. Dell'Acqua et al. 2023. Randomised field experiment with 758 consultants. [Link](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321)
2. Parasuraman and Manzey. Human Factors, 2010. Review. [Link](https://trid.trb.org/View/1085175)