AI detectors: why the score is not evidence

An AI detector does not detect AI. It measures how predictable a piece of writing is, compares that against patterns from a training set, and returns a number. The number describes style, not origin. It carries no defined error rate for your document, your subject or your first language, and it has no access to any record of how the text was produced. People have been failed and fired on the strength of one of those numbers, and some universities switched the feature off rather than keep defending it.

Detectors are not worthless: a flag is a reasonable prompt to look closer. The damage comes from treating it as a finding of fact.

How a text detector actually works

Almost every text detector rests on one of two mechanisms, and many combine them.

The first is perplexity, a measure of surprise. Run a document through a language model one word at a time and ask, at each position, how likely the model considered the word that actually appeared. If the text keeps choosing the word the model would have predicted, perplexity is low. Generated text sits low, because sampling favors high probability words, which falls out of the way a model picks each next word. Detectors add burstiness: the variation in that surprise from sentence to sentence. People write a long winding sentence and then a short one. Machine output is smoother.

The second mechanism skips handcrafted statistics. You take a pile of documents labeled human, a pile labeled machine, and fine tune a model to tell them apart. It learns whatever separates the piles, including subject matter, formatting and the decade the human samples come from.

Notice what is missing from both. Neither has any information about where the text came from. Ordinary generated text carries no hidden signature, no log, no receipt. There is only the words on the page and a model of what typical words look like. A detector can say a document is unusually predictable. Whether that is because a machine produced it or because the author writes plainly is a distinction the method cannot make.

The percentage is not a probability

A detector reporting 92 percent is not saying there is a 92 percent chance a machine wrote the text, nor that 92 percent of the words are machine written. It is a classifier score squeezed into a range from zero to one hundred, and where the vendor drew the line between clear and flagged is a product decision about how many false alarms customers tolerate.

Two things matter more than the number and neither appears on screen. The first is length: below a few hundred words a single unusual sentence swings the result, which is why short answers and abstracts produce nonsense scores. The second is the base rate, where most accusations quietly go wrong.

Work an example. Suppose a tool wrongly flags 1 in every 100 human documents, catches 8 of 10 unedited machine written ones, and is pointed at 100 essays in which 5 students used a chatbot. It flags 4 of the 5, plus about 1 innocent student: one accusation in five is wrong. Now make the false positive rate 4 percent rather than 1 percent, a difference no marker could notice. It flags about 4 cheaters and about 4 innocent students, and the flag is a coin toss. Nothing on screen tells you which world you are in, and the rarer the cheating, the worse the ratio.

Who gets flagged and why it is not random

False positives are not scattered evenly. They concentrate on writers whose prose is, by the detector's definition, predictable:

• People writing in a second or third language, who sensibly stick to common words and safe sentence patterns. This is the best documented failure of the category and follows directly from what perplexity measures.
• Students taught a rigid structure: introduction, three body paragraphs, restate the thesis. The structure is the point of the exercise and it flattens variation.
• Technical, legal and scientific writing, where conventional phrasing is required and an inventive sentence is a defect.
• Anyone who runs a draft through a grammar or rewriting assistant, which smooths exactly the variance burstiness looks for.

The tool does not sort honest from dishonest. It sorts conforming from idiosyncratic, and someone then reads that as a moral result. The same trap catches people judging by eye, which is why the stylistic tells in how to tell whether text was written by AI are weak evidence rather than a checklist.

Image and audio detectors fail differently

A picture is not predicted word by word, so perplexity has no equivalent. Image detectors look for fingerprints a generator leaves in the pixels: frequency patterns from upsampling layers, unnatural noise statistics, texture that is too consistent, lighting that disagrees across the frame. Others are classifiers trained on real and synthetic collections. Both work on the generators they were built against.

Two problems follow. A detector tuned on last year's artifacts does not transfer to a model built differently. And the artifacts are fragile: a screenshot, a crop, or a platform re encoding your upload scrubs the traces, while leaving the picture perfectly convincing to a human. The checks in how to tell whether an image was generated survive that better than any score. Voice detection fails the same way, with phone compression destroying the cues.

The two methods that carry real information

The bottom two rows are different in kind: they look for something a producer put there on purpose. A statistical watermark nudges the model's sampling with a secret key, so the output carries a faint pattern a key holder can measure. That is genuine evidence when it fires, but it covers only models whose makers implemented it, usually only the key holder can check it, and a translation or an aggressive rewrite wipes it out. Signed provenance instead attaches a manifest to the file listing the tool and the edits, the approach in how Content Credentials record where a file came from.

Both share one limit that matters enormously. They can support "this came from there", and neither can ever support "this came from nowhere". Absence of a watermark or a credential is not evidence of human authorship, because most files have neither.

What a fair process looks like

If you run a classroom, a hiring process or a team, the question is what role a score may play. A defensible answer: it can open an inquiry, never close one.

1. Write down that the score is not evidence before a case arrives. A policy written under pressure reads as a justification.
2. Require corroboration from outside the text: version history, drafts, notes, the sources cited, a conversation about the argument.
3. Tell the person what was found and let them respond before a grade, offer or contract is affected.
4. Never ask anyone to prove they did not use a tool. That cannot be done, and demanding it punishes the least confident person present.
5. Apply it uniformly, or you have simply automated your hunch.

The better fix is upstream, in how work is set: tasks needing personal specifics, an in person component, a visible drafting process, or a stated rule on what help is allowed. That framing is the subject of using AI to study without cheating yourself, and a team's equivalent is a written statement of allowed use in your workplace AI policy.

What to do if you are accused

Stay calm and stay specific. Confessing to end an uncomfortable conversation is the worst move available, and rewriting the piece to look more human concedes the premise.

1. Ask in writing what the accusation rests on: which tool, what score, what threshold, and what evidence exists besides the score.
2. Ask for the policy on detector output and the appeal route. Many already say a score alone is insufficient.
3. Produce process evidence. Version history in a cloud document shows text growing over hours with deletions and reordering. Offer drafts, outlines, notes, file timestamps, the sources you read.
4. Offer to talk it through. Ten minutes explaining why you cut a section and what you would argue next beats any counter test, and it is the check a serious assessor wants.
5. If it applies to you, say plainly that these tools over flag writers whose first language is not English, and ask for that to be recorded.

Protect yourself cheaply from here on: draft where version history is kept, keep your notes, and paste research links into the document as you go. Check every source you cite exists, because a fabricated reference is the one thing that turns a weak suspicion into a real case, for the reasons in why AI invents books, cases and studies.