TELL, an AI-Text Detector That Shows Its Work

At the healthcare company where I used to work, the cost of writing a fourteen-page appeal with citations to the plan document, the state statute, and three prior determinations dropped to about zero in a single year. Every one of those pages still had to be read by a person with a caseload and a regulatory clock. So the AI conversation in most meetings was about using AI to detect AI, and the question was always "can we tell?" My answer was "sort of, and you won't like the error rate."

My friend Jing Xu was leading a team looking into AI detection at the time, and most of what I knew about the literature came from comparing notes with her: a new zero-shot method, a new benchmark, a new paper showing last year's detector falling over on this year's model.

I never got to build the thing there, so I built it at home. TELL (as in a poker tell, a signal the writer didn't mean to give) puts the methods from that literature into one engine with one rule: it returns evidence, never a verdict.

The techniques

Nearly every method rests on one fact: a language model predicts the next word, and text a model wrote is text a model finds easy to predict. Here is what each one measures and where it came from.

Perplexity
measures	How surprised a reference model is, word by word. Feed it the text, record the probability it gave the word that actually came next, average the surprise, exponentiate. A perplexity of 8 means the model was as uncertain as a choice among eight equally likely words.
reads machine	Low. The model that wrote the text was picking its own most likely words.
weak spot	Careful technical prose and second-language writing are also low. Seven commercial detectors flagged 61% of human-written TOEFL essays as AI and almost none written by US eighth graders.
source	GLTR, Gehrmann, Strobelt and Rush, ACL 2019, which colored each word by its rank in GPT-2's prediction. GPTZero productized it in 2023. The bias result is Liang, Yuksekgonul, Mao, Wu and Zou, Patterns 2023.

Burstiness
measures	How much the surprise jumps from sentence to sentence, rather than its average.
reads machine	Smooth. People write a plain sentence and then a strange one; models keep a steady hum.
weak spot	Needs enough sentences to measure variance, and a disciplined human editor flattens it too.
source	Popularized alongside perplexity by GPTZero (Tian, 2023); the underlying idea is per-sentence perplexity variance from the stylometry literature.

DetectGPT
measures	Curvature. Make a hundred small paraphrases of the text and score every one against the original.
reads machine	When the original sits on a peak: nearly every paraphrase scores worse. Human text sits on a slope, with paraphrases scoring better about as often as worse.
weak spot	A hundred model calls per document.
source	Mitchell, Lee, Khazatsky, Manning and Finn, ICML 2023.

Fast-DetectGPT
measures	The same curvature, estimated by sampling alternative words from the model's own next-token distribution instead of paraphrasing.
reads machine	High conditional probability curvature; the paper reports machine text near 3 and human text near 0.
weak spot	Still depends on how close the reference model is to the one that wrote the text.
source	Bao, Zhao, Teng, Yang and Zhang, ICLR 2024. One pass instead of a hundred.

Binoculars
measures	Two models from one family, an observer and a performer. The observer scores perplexity as usual. Then: at each position, how surprising would the observer find the word the performer would have chosen? That cross-perplexity captures how predictable the text is for reasons that have nothing to do with authorship, like topic. Divide the first by the second.
reads machine	A low ratio. The text is more predictable than even a model would expect it to be.
weak spot	Thresholds move with the model pair, and the pair has to share a tokenizer.
source	Hans, Schwarzschild, Cherepanova, Kazemi, Saha, Goldblum, Geiping and Goldstein, ICML 2024. Over 90% of generated samples caught at a 0.01% false positive rate, with no training data.

Watermarks and provenance
measures	Nothing about the words. A watermark biases the model's sampling toward a secret subset of tokens so the text carries a statistical fingerprint. Content Credentials attach a signed manifest to the file.
reads machine	When the fingerprint or the manifest is present. Near-zero false positives.
weak spot	Gone the moment someone pastes the text into a new document. Absence proves nothing.
source	Kirchenbauer, Geiping, Wen, Katz, Miers and Goldstein, ICML 2023; SynthID-Text, Dathathri et al., Nature 2024, deployed in Gemini. Manifests are the C2PA specification.

Stylometry
measures	The authorship-attribution toolkit from before any of this: sentence-length distribution and its kurtosis, paragraph uniformity, punctuation habits, type-token ratio, function-word profiles, transition words at paragraph openings.
reads machine	Too regular. Models under-produce very short and very long sentences and are too consistent about punctuation.
weak spot	Each feature is weak alone and false-fires on restrained, careful writers.
source	Mosteller and Wallace settled the disputed Federalist Papers with function-word rates in 1964. No GPU required then or now.

What TELL does with them

Every one of those methods fails, and they fail in different ways. Perplexity fails on careful writers. Lexical tells fail the week the vendor tunes them out. Watermarks fail on copy-paste. If the failures are uncorrelated you can run all of them and let the agreement be the signal. Detection is a portfolio problem, not a model problem.

So TELL is an ensemble. Every instrument implements one small contract: take the document, return a log-likelihood ratio, a confidence, the spans that drove it, and a sentence of rationale a reviewer could argue with. A regex and a GPU scorer look identical to the fusion layer. Loaded today:

• Statistical: perplexity, a Binoculars ratio, and burstiness, scored by a local Qwen2.5-0.5B base model as observer and its instruct sibling as performer. Text the instruct model generated scores a Binoculars ratio around 0.72; a human essay scores around 1.0.
• Stylometric: sentence-length variation and kurtosis, paragraph uniformity, punctuation profile, windowed type-token ratio, and the rate of sentences opening with "However," or "Moreover,".
• Rule packs: YAML files of lexical tells, each rule weighted, each pack versioned with a half-life because tells decay. Four packs: Claude-isms, GPT register, chatbot voice, SEO slop.

The unit is what makes the portfolio add up. A +1 shifts the odds about 2.7 to 1 toward machine, a +3 about 20 to 1, and independent evidence in that unit sums. TELL starts from a skeptical prior of about 7 to 1 that a document is human, adds the evidence, and shows the arithmetic.

The arithmetic has three corrections, and they are policy, not tuning. Instruments that partly measure the same thing (sentence uniformity, low perplexity, a pile of register words) are grouped, and within a group the strongest counts fully while the rest are discounted, 0.62 for the second and 0.45 for the third. No single instrument may contribute more than +3. And if only one signal fired, the total is held just under the "elevated" band no matter how loud it was. Real humans use em-dashes. A single tell is worth almost nothing; the signal is unrelated tells co-occurring. Every report also says, in the header, that false-positive rates for the profile have not been measured. Until they are, TELL answers "does my draft read like a machine wrote it, and where?" and nothing else.

The obvious giveaways

David Sedaris's Me Talk Pretty One Day is a book about being bad at French, and it is one of the funniest things I have read, because the broken grammar is the voice. Style is how a person gets through to you. So it bothers me when my AI tells me something "is the spine" or "the substrate." Me no talk like that. Nobody I know talks like that. The Claude-isms pack is where those go. Some of it:

• "has been quietly transforming," "quietly became," "silently picked." Nobody quietly or silently does anything at the rate models do.
• The metaphor vocabulary: shape, spine, substrate, load-bearing, surface, texture, scaffolding, seam, hinge, lever, throughline, guardrails, north star, blast radius. Physical nouns doing conceptual work. Marked context-required: a literal spine is innocent, and with no judge yet the engine shows the span and refuses to score it.
• "Surface" as a verb. Unlike the noun, almost always the metaphor.
• The antithesis family: "not just X but Y," "it isn't A. It's B," "not X, not Y, but Z."
• "Here's the thing," "the real question is," and the rest of the colon reveals.
• Announced directness: "I'll be honest," "let me be direct," "the short answer is."
• "That's not nothing." "Which is exactly why."
• Sincerity words: genuinely, honestly, thoughtful, intentional, worth noting, to be clear.
• Structure: a bold lead-in phrase on every bullet, lists of exactly three, anaphora in triples ("It didn't refuse. It didn't hedge. It didn't throw."), a full sentence then a two-word fragment for punch, bullets of identical length. Humans straggle.
• Paired em-dashes for an aside, and arrows in running prose.

The GPT pack carries the 2023 vocabulary everyone knows (delve, tapestry, testament, pivotal, robust, seamless, leverage), "whether you're a," emoji in headings, and the "Furthermore" that opens a paragraph. The chatbot-voice pack is tiny and nearly certain: "Great question," "I hope this helps," "Let me know if you'd like." Providers tune words faster than they tune manners.

Every rule in those packs fires in human writing. That is why the sample in the first screenshot scored +9.43 with 76 flagged spans, and a 288-word essay about rebuilding a fence scored +0.03 with two.

What a 115,000-word document gave away

The last thing I ran through all of this was a planning document I produced with a coding agent over a few days: 115,000 words in 55 Markdown files, bound into a 320-page PDF. It had been written under rules that forbid the obvious tells and had been through two cold-read reviews by agents given no context. I wanted to know what was left.

Mild, at +2.90. Only one instrument fired above the noise floor, so the ceiling pinned the total under elevated no matter how loud that pack was. The vocabulary tells were gone: zero of 2,225 sentences opened with a stock transition, the GPT pack found almost nothing, the Binoculars ratio came in at 1.043 on the human side, and perplexity at 30.9, well above the floor. What remained was structural:

what fired	count	rate
arrows in prose	404	3.5 per 1k words
bold spans	820	7.1 per 1k words (human p95 is about 8)
backticks around short phrases	2,878	24.9 per 1k words
sentence, then a two-word fragment for punch	204	in 2,225 sentences
lists with near-uniform bullet length	12
semicolons		26.7 per 1k words (human p95 is about 4)

Backticks around identifiers in a spec are not emphasis, and the arrows lived in tables; the report weights both low and says so. The GPT pack counted "harness," a term of art in that document, as a delve-family verb 142 times, and one rule can't move the total. But the semicolons were real, at six times the human 95th percentile, and so were the 204 punch fragments and the twelve lists where every bullet was the same length. The lexical tells had been edited out. The rhythm hadn't.

Then the part TELL couldn't see. Over a long document a model falls in love with a word. Not a tell word, usually. A plain one, used in three senses and leaned on in every section, because the model has no memory of using it 200 pages ago. So the newest check does nothing but count: the most frequent content words across a folder, and for any one word, the phrases it appears in. A word with one dominant phrase has one meaning. A word spread across many phrases is carrying several and wants a glossary entry or a rename.

"Never" is the one I would not have caught by reading. Three words from that document, with their rate against the two posts on this site I wrote by hand:

word	in the document	in my posts	what the phrases showed
never	201, or 2.4 per 1k	0.5 per 1k	"users never see," "is never stored," "never a," "never put": an absolute every 575 words
shows	204, or 2.5 per 1k	0.5 per 1k	63 of them were "figure shows"
gate	90, or 0.8 per 1k	0	"gate registry," "hard gate," "gated in," "gate pages," "date gate": one word, at least four meanings

Each "never" sentence was fine. Two hundred of them turned a plan into a list of prohibitions. "Gate" was worse, because every reader would have picked a different meaning and none of them would have known. The fix was a vocabulary rule in the document's own style guide: a gate is an external, non-code condition between code-complete and live, with an owner and a date. Merge-time conditions are checks. Latency limits are ceilings. Sign-in is access control. Then a rename pass, and the count fell to the uses that meant the one thing.

That is the tool I wanted at the healthcare company and didn't have. Not a percentage. A list of what fired, how often, where, and how much each one is allowed to count, so the person with the caseload and the clock can decide what to do about it.

I enjoy having AI as a partner, not a crutch. It helped me with this post. Can you tell?