# Asking questions of your data when you're not an analyst

A 34-minute routine for asking an assistant about a spreadsheet: check the row count and a known total first, test the question on a sample you worked out by hand, then reconcile the answer and keep the code.

Workflows · v1.1 · draft · https://hayzlabs.com/workflows/asking-questions-of-your-data/

In short:
- Remove personal data, then have the assistant describe the file and check its row count and a known total against yours.
- Test your question on 10 to 20 rows you worked out by hand before you run it on the full file.
- Check the parts add to the total, ask for a second method on anything you will act on, and save the code.

**Use it when** you have a spreadsheet or CSV (a plain-text spreadsheet file) and want to ask an assistant that can run code a question such as "what changed?", and you are not an analyst.

**Not when** the numbers carry real consequences, such as tax, payroll or a legal filing, and you cannot check them. Take those to a person who can.

**You need** a copy of the file with personal data removed, an assistant that runs code on uploaded files, and about 34 minutes. Remove names, contact details and health data first: [What not to paste](https://hayzlabs.com/start/what-not-to-paste/). The steps and times are our suggestion.

## Steps

| # | Step | Who | Time | Check |
|---|---|---|---|---|
| 1 | Prepare. Remove personal data. Write down the row count and one or two totals you already know. Work out by hand the answer to your question on 10 to 20 rows. | Person | 10 min | You have a row count, a known total and one hand answer. **If it fails:** you do not know the data well enough to check it yet. |
| 2 | Upload the file and ask the assistant to describe the data before any analysis. | AI | 3 min | Columns, types, row count, missing values and date range are listed, with nothing changed. |
| 3 | Compare its row count and totals with yours. | Person | 4 min | The row count and the total match. **If it fails:** ask which rows it dropped and why, and fix the file before you go on. |
| 4 | Test the question on your 10 to 20 rows, telling it which rows by row number or pasting just those. | AI | 4 min | Its answer matches your hand answer. **If it fails:** the method is wrong. Do not run the full file. |
| 5 | Ask the question on the full data, with its method and any rows it leaves out. | AI | 3 min | It names the method and the exclusions. |
| 6 | Reconcile the answer: do the parts add to the total, and do percentages come with counts? | Person | 5 min | Segments sum to the total and every percentage has its count. |
| 7 | For anything you will act on, ask for a second method and save the code and steps with the answer and any chart. | AI and person | 5 min | Two methods agree, or you can explain the gap. |

Each later question repeats steps 5 and 6, about 8 minutes.

> [!CHECK]
> Step 3 is the gate. A row count that is lower than yours means rows were dropped, and the assistant may not have told you. Find out which before you trust any total.

## The prompts

For step 2:

```text title="Describe the data first · v1.0"
Before any analysis, describe this file. List the columns and their types, the row count, missing values per column, the date range, and anything odd. Do not change the data.
```

For step 5:

```text title="Ask with the method shown · v1.0"
Answer this question about the whole file: <QUESTION>
First state your method and any rows you exclude and why. Then give the answer with counts as well as percentages. Show the code you ran.
```

Replace `<QUESTION>` with one question, such as `which month had the biggest drop in sales?`

For step 7:

```text title="A second method · v1.0"
Answer the same question a different way, for example with pivot-table logic in place of grouping, and show both results side by side. Say what would change your conclusion.
```

Useful question patterns: "Compare A and B and show counts as well as percentages." "Is this difference bigger than normal month-to-month variation? Show the variation." Treat any "why" in the answer as a list of ideas to check. The file shows what happened, and the reasons are guesses.

## A worked example (illustrative)

Asha has a year of cafe sales: 2,150 rows, and she knows the total is 38,640. The assistant reports 2,118 rows and a total of 39,120. She asks which rows are missing. It says 32 rows had blank dates and were left out. They were refunds, worth 480 in all, so 38,640 plus 480 gives the 39,120 it reported. She fills in the dates, runs the file again and both numbers match. Only then does she ask which month dropped most.

## How to tell it worked

- The row count and the known total match, and your hand answer on the sample matches too.
- The parts add up to the total, and you have the code saved with the answer.
- It has failed if you used an answer whose row count you never compared. Go back to step 3.

If the data starts as a photo or scan, read the figures yourself first: [Photos in, data out](https://hayzlabs.com/workflows/photos-in-data-out/).

## Why the checks matter

- OpenAI's help page advises users to "review the generated code, outputs, and assumptions before relying on the result" ([OpenAI help](https://help.openai.com/en/articles/8437071-data-analysis-with-chatgpt), vendor documentation). Steps 3, 4 and 6 are that review.
- In a CHI 2024 study of 22 analysts, AI-generated analysis code could be "misaligned with the analyst's intent or be seemingly correct but lead to incorrect conclusions" ([Gu et al.](https://arxiv.org/abs/2309.10947), user study). That is why step 4 checks the answer on a sample and does not rely on you reading the code.

> [!LIMIT]
> A matching row count and total do not prove the method fits your question, which is what the sample test and the second method are for. The routine will not catch a wrong column that gives a plausible number, or a bias in the file itself. A confident explanation of why something changed is a guess until you test it.

## Sources

1. OpenAI, Data analysis with ChatGPT, help page. Vendor documentation. [Link](https://help.openai.com/en/articles/8437071-data-analysis-with-chatgpt)
2. Gu et al., 2024. CHI user study of 22 analysts checking AI-generated analysis code. [Link](https://arxiv.org/abs/2309.10947)