How accurate are AI detectors?

The short answer is: not accurate enough to accuse anyone with. But the more interesting question is why, because once you see why, you see that better detectors won’t fix it.

Short answer: AI detectors wrongly flag human writing often enough to matter. OpenAI shut down its own detector for being inaccurate, and a Stanford study found popular detectors flagged most essays by non-native English writers as AI. Some newer detectors test much better, but every detector gives you a guess, not evidence.

What the studies found

In 2023 a group at Stanford ran 91 essays from the TOEFL, the English test for non-native speakers, through seven popular detectors.[1] Every essay was written by a person. On average, the detectors called 61% of them AI. At least one detector flagged 89 of the 91. The same detectors were near-perfect on essays by US-born eighth graders.[2]

That same year OpenAI took its own detector offline. It caught only 26% of AI-written text and wrongly flagged 9% of human text.[3] The company that makes ChatGPT couldn’t reliably tell when something was written by ChatGPT. A separate team tested 14 tools, Turnitin among them, and concluded they were “neither accurate nor reliable”.[4]

Why detectors can’t be fixed

Here’s the problem in one sentence: a detector has to decide whether a person could have written a piece of text, and a person can write almost anything.

AI models are trained to write like people, and every improvement makes their output harder to tell apart from ours. Meanwhile any AI text a detector does catch can be run through a paraphrasing tool, and researchers have shown that this defeats many detectors.[5] So detectors are squeezed from both sides. Catching more AI means flagging more humans. Flagging fewer humans means catching less AI.

Some detectors are much better than others, and it’s worth being fair about that. A 2025 University of Chicago working paper found that Pangram made almost no errors on medium and long passages.[6] That’s real progress. But notice what even a perfect detector gives you: its opinion of the words. It can’t give an honest student anything to show back.

Who pays for the errors

• Non-native English writers. Across the seven detectors in the Stanford study, 61% of their essays were wrongly flagged, against about 5% of essays by native writers.[7]
• Black students. In a Common Sense Media survey, about 20% of Black teens said a teacher had wrongly flagged their work as AI, compared with 7% of white teens.[8]
• Anyone who writes plainly. Clear, simple writing is exactly what detectors find predictable.

And small rates become big numbers at scale. Turnitin checked 38 million submissions in its first six weeks.[9] Vanderbilt worked out that even a 1% error rate could mean about 750 wrongly flagged papers a year on its campus, and turned the detector off.[10]

What works instead

Stop asking “does this text look human?” and ask “how was this text made?” The first question can only be answered with a guess. The second can be answered with a record, and that’s what Huma gives you. Huma records every key press as you write and gives you a link and a PDF Report showing what was typed and what was pasted, which anyone can check in a browser. See how Huma compares with Grammarly Authorship, or what to do if Turnitin has already flagged you.

Questions

Can AI detectors be wrong?

Yes. OpenAI shut down its own detector after it flagged 9% of human text, and a Stanford study found seven popular detectors flagged 61% of essays by non-native English writers as AI on average.

Why did OpenAI shut down its AI detector?

OpenAI took its AI text classifier offline in July 2023 because of its low accuracy. It caught only 26% of AI-written text and wrongly flagged 9% of human text.

Are AI detectors biased against non-native English speakers?

A 2023 Stanford study found seven popular detectors flagged 61% of TOEFL essays by non-native English writers as AI on average, while being near-perfect on essays by US-born eighth graders.

Notes

1. Liang, Yuksekgonul, Mao, Wu and Zou, “GPT detectors are biased against non-native English writers”, Patterns, 2023. arxiv.org
2. Stanford HAI, “AI-Detectors Biased Against Non-Native English Writers”. hai.stanford.edu
3. The Decoder, “OpenAI discontinues AI text detector due to lack of accuracy”, July 2023. the-decoder.com
4. Weber-Wulff et al., “Testing of detection tools for AI-generated text”, International Journal for Educational Integrity, 2023. doi.org
5. Sadasivan et al., “Can AI-Generated Text be Reliably Detected?”, 2023. arxiv.org
6. Jabarian and Imas, “Artificial Writing and Automated Detection”, Becker Friedman Institute, University of Chicago, 2025. bfi.uchicago.edu
7. MIT Sloan Teaching & Learning Technologies, “AI Detectors Don’t Work. Here’s What to Do Instead.”. mitsloanedtech.mit.edu
8. EdWeek, “Black Students Are More Likely to Be Falsely Accused of Using AI to Cheat”, September 2024 (Common Sense Media survey). www.edweek.org
9. Campus Technology, “Turnitin Says Its AI Detector Analyzed 38M Submissions in Six Weeks”, May 2023. campustechnology.com
10. Vanderbilt University, “Guidance on AI Detection and Why We’re Disabling Turnitin’s AI Detector”, August 2023. www.vanderbilt.edu