Am I actually faster?

Time a few of your own runs both ways and this page turns them into a median saving, with a quality check beside it. It counts the minutes that people forget: reading the result through, fixing it and asking again.

What to record

One row per run. Use the same kind of task in both modes, so that the comparison is fair, and pick the order of your runs by a coin flip so you do not give AI the easy tasks.

The hands-on tool below starts with the worked example further down. Change the rows or add your own. It runs in your browser and nothing you enter is stored or sent.

Input	Unit	Hint
Mode	AI or by hand	Pick the order by a coin flip.
Minutes	whole minutes	From starting to a version you would use. Time it on a clock.
Rework minutes	whole minutes	Reading it through, fixing it, asking again, anything after "done".
Quality OK	yes or no	Would you send it as it is?

Reading the answer

What you see	What it means
Fewer than 3 runs in a mode	Not enough data. Keep logging.
3 or 4 runs in a mode	The answer is provisional.
Saving of zero or less	No saving.
Saving, and AI failed more often than by hand	Saving, but quality slipped.
Saving, and AI failed no more often	Saving.

Prompt

time          = minutes + rework minutes
saving        = median(by hand) - median(AI)
speed-up %    = saving / median(by hand) x 100
failure rate  = runs where quality was not OK / runs, for each mode

A median is the middle value when the times are in order (with an even count, the average of the middle two). One odd run, such as an hour lost to a broken session, moves a mean a lot and a median hardly at all. In a spreadsheet, =MEDIAN(range) does this.

Worked example (illustrative)

These runs are invented to show the arithmetic.

By hand: 32, 28 and 35 minutes. The median is 32.
With AI: 14, 22 and 18 minutes. The median is 18. The 22-minute run did not pass the quality question.
Saving: 32 - 18 = 14 minutes, which is 14 / 32 = 43.75%, about 44%.
Failure rate: 0 of 3 by hand and 1 of 3 with AI.
Verdict: saving, but quality slipped. With only 3 runs each, it is also provisional.

Rework changes the result. Say the three by-hand runs take 10 minutes each, and the three AI runs take 6 minutes each plus 5 minutes of fixes. The AI runs are 11-minute runs, so the saving is 10 - 11 = -1 minute. The verdict is no saving, although the AI part felt quick.

Felt speed and measured speed

Before you read the answer, write down how much faster you think AI made you. The difference between that guess and the measured figure is worth keeping.

In a 2025 trial, 16 experienced open-source developers (people who build software whose code is public) using early-2025 AI tools took about 19% longer on real tasks. Afterwards they still believed AI had made them about 20% faster, and beforehand they had expected 24% (METR, 2025, randomised trial). It says that felt speed and measured speed can disagree, and it does not say which way yours will go.

Sources

METR. Randomised trial of early-2025 AI tools with experienced open-source developers, July 2025. 16 developers. Link

Changes

v1.010 Oct 2026first version