AI detectors: what they can and can't tell you

Short answer

A detector flag is a statistical guess about a piece of text. It is no proof about the person who wrote it. Independent tests found wrong flags and unreliable results, and tools differ a great deal. Treat a flag as a reason to ask, and decide on drafts, version history and a conversation about the content.

What the tests found

• Seven AI detectors wrongly flagged on average 61% of 91 TOEFL essays (an English test for non-native speakers) written by people; 97.8% were flagged by at least one detector and about 20% by all seven. Simplifying the wording of native speakers' essays raised false positives (human writing wrongly flagged) from about 5% to 57% (Liang et al., Patterns 2023, peer-reviewed benchmark).
• OpenAI: "As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy." On its own challenge set it caught 26% of AI-written text and wrongly flagged 9% of human text (OpenAI, vendor).
• Testing 12 public tools plus Turnitin and PlagiarismCheck, Weber-Wulff et al. (2023) concluded "the available detection tools are neither accurate nor reliable"; they leaned towards calling text human-written and did worse when text was disguised (Weber-Wulff et al., preprint).
• Vanderbilt disabled Turnitin's AI detector in August 2023, noting that at a 1% false-positive rate the 75,000 papers it submitted in 2022 would have meant around 750 wrongly flagged (Vanderbilt, institution guidance).
• Turnitin said (June 2023) its detector has a "higher incidence of false positives" when less than 20% of a document is flagged as AI, added an asterisk to such scores and raised the minimum length from 150 to 300 words (K-12 Dive, reporting the vendor).
• In an NBER working paper (Sept 2025), Pangram had near-zero false positives and 2-4% false negatives (AI text it missed), GPTZero under 1% and 0-2%, Originality.ai under 1% and 10-40%; a free, publicly released detector built on RoBERTa was "unsuitable for high-stakes applications" (Jabarian & Imas, preprint; per-tool rates as summarised by the Booth Review).

So we cannot say every detector fails. We also cannot say any of them settles whether one essay was written by AI. Each result is for the version and the texts tested, and tools change.

If you are the one flagged

The steps below are our suggestion. Sending the request takes about 15 minutes (our estimate); gathering your drafts may take longer.

1. Gather your process before you reply: drafts, notes and version history with dates. In a shared document, open its version history and note the dates. Keep these from now on for anything that is graded or checked.
2. Send the request below by email, so it is in writing. It asks which tool was used, what score it gave, what the written process is, and for a person to review the decision.
3. If they agree to meet, prepare to talk through the content: why you made each point, where the sources came from.
4. If you write in English as a second language, say so and point to the Liang result above. Writing in a second language, or with dyslexia has a routine that keeps your draft and your edits on record.

Subject: <ASSIGNMENT> - request for details of the AI detection result

Dear <NAME>,

I understand <ASSIGNMENT> was flagged by an AI detector. I wrote it myself and
would like to resolve this. Please could you tell me in writing:

1. Which detection tool was used, and which version if known.
2. The score it gave, and which parts of the text it flagged.
3. The written process for a case like this, and what happens next.
4. That a person will review the decision, not the score alone.

I can share my drafts, notes and dated version history: <EVIDENCE>.
I am also happy to meet and talk through the content, why I made each point
and where my sources came from.

Detectors are known to make mistakes. OpenAI withdrew its own classifier "due
to its low rate of accuracy", and Liang et al. (Patterns, 2023) found seven
AI detectors wrongly flagged on average 61% of 91 TOEFL essays written by
people. <ESL_LINE>

Thank you,
<YOUR_NAME>

Replace the following:

• <NAME>: the teacher, editor or officer who told you.
• <ASSIGNMENT>: the title and date of the work.
• <EVIDENCE>: what you have, such as two drafts, my notes and the document's version history from 2 to 9 March (illustrative).
• <ESL_LINE>: if English is your second language, English is my second language.; otherwise delete it.
• <YOUR_NAME>: your name, and your student or account number if you have one.

If the case is serious, such as a misconduct finding, take it to a qualified adviser.

If you are holding a detector result

1. Treat it as a reason to ask, not as a finding. Do not act on the score alone.
2. Look at drafts and version history, and ask the writer about the content.
3. Be especially careful with writing by people who use English as a second language, and with low scores or short texts, where the vendors' own caveats apply.
4. Remember the base rate. An illustration with invented numbers: a detector catches 90% of AI-written essays and wrongly flags 1% of human ones. Of 1,000 essays, if 50 are AI-written it flags 45 of them and about 10 human ones, so roughly one flag in six is wrong. If only 10 are AI-written, it flags 9 and still about 10 human ones, so about half the flags are wrong.

Move the sliders below to see the same sum as 1,000 essays, one square each.

For setting clear rules on AI use at home before anything is checked, see Kids, homework and a family AI plan.

How to tell it worked

• You have a written reply naming the tool, the score and the process.
• A person, not the score, made the decision, after looking at drafts or talking about the content.
• From now on, your graded or checked work has dated drafts or version history you can show.

Sources

1. Liang et al. AI-text detectors and non-native English writing, Patterns, 2023. Peer-reviewed benchmark. Link
2. OpenAI. AI classifier announcement and withdrawal notice, 2023. Vendor. Link
3. Weber-Wulff et al. Testing of AI-text detection tools, 2023. Benchmark (preprint). Link
4. Vanderbilt University. Why it disabled Turnitin's AI detector, 2023. Institution guidance. Link
5. K-12 Dive. Turnitin's false-positive caveats, 2023. Trade press reporting a vendor. Link
6. Jabarian & Imas. NBER working paper w34223, 2025, with the Booth Review summary. Benchmark (preprint). Link · Booth Review

Changes

1. v1.010 Oct 2026first version