Epoch's work is free to use, distribute, and reproduce provided the source and authors are credited under the Creative Commons BY license.
Learn more about this graph
We measure the performance of three AI text detectors (Pangram, GPTZero, and Originality.ai) under three conditions: human writing, AI written from a basic few-word prompt, and AI that imitates a specific author’s style. For human text, we report the false-positive rate (the share of human passages a detector flags as AI), and for AI text, the false-negative rate (the share of AI passages it does not flag as AI). We take each detector’s own document-level verdict at its default setting, counting only a clean “AI” result as correctly flagging a passage. Two of the detectors (Pangram and GPTZero) sometimes return a “Mixed” verdict, where a document is judged to be part human and part AI. This is also counted as a non-detection, since none of our passages are of this nature. On human text, a “Mixed” verdict would likewise be an error — counting toward the false-positive rate — but neither detector returned one on any of the 495 human passages.
The pattern across conditions is consistent: false-positive rates are low overall, with only Originality.ai mistaking any human-written text for AI. Likewise, when given a basic prompt (no instruction to mimic or reference text), AI is almost always caught. However, a substantial share of style-imitated AI goes undetected, especially in scientific writing. Data and figure code are in the project repository, linked here.