No, AI Writing Doesn't Have a "Surprise Arc": Here's What Actually Gives It Away

The idea we tested

There's a seductive analogy between music and language: beauty is managed surprise. A great composition isn't monotonous and isn't noise; it shapes your expectation into a deliberate arc that builds and resolves. So maybe writing works the same way, and maybe that arc is the hidden fingerprint that separates a human author from a machine. It's a lovely theory. We ran it against real data on independent, public benchmarks. It does not hold up.

What we found

• The "arc" is a dead end. The shape of a text's surprise curve scores essentially at chance for telling human from AI: ROC-AUC of 0.55 on short answers and 0.51–0.53 on long-form text. Adding it to a simple detector improves nothing.
• The boring statistics win. What separates human from machine writing is the level of surprise (perplexity) and its variance (burstiness). On raw machine text these are near-flawless: AUC 0.97 and 0.95. Raw AI writing is simply too even and too predictable; in the jargon, it overshoots the human tendency to spread information evenly.
• A paraphrase attack is selective, not all-or-nothing. Rewriting machine text to disguise it collapses perplexity (from 0.80 to 0.58 AUC: it restores a human-looking average) but barely dents burstiness (0.80 to 0.78). The lived-in unevenness of real sentence rhythm is the harder thing to fake, which makes burstiness the more durable tell.
• More uniform writing scored higher, not lower. Among graded student essays, lower-perplexity, more even prose earned slightly better human scores: the exact opposite of the "good writing has a dramatic arc" intuition. The arc shape added nothing to predicting quality.
• Even novels have no arc. We checked 30 full-length public-domain books. Book-length surprise is the flattest writing of all, and book arcs don't reduce to a few shapes the way emotional arcs do (the top two components explain just 22% of the variance). Surprise arcs are not emotional arcs.

Why it's solid

Everything ran on public, multi-author benchmarks (HC3, MAGE and ASAP-AES) with no in-house data. Every result replicates on a second, larger surprise model, so it isn't a quirk of one model. And the whole pipeline is open: a single command reproduces every number.

What it's good for

The most useful takeaway is about robustness. The average predictability of a text is the easy half to disguise; the rhythm, the unevenness from sentence to sentence, is the half that survives a rewrite. If you want to actually look at a passage with that in mind, you can run text through our AI text detector. And none of this says detectors are infallible; they read statistical texture, not truth, which is exactly why a single score should never be the final word.

Read the full study

The complete analysis (every statistic, all the figures, the second-model replication, the book-length result and a frank limitations section) is on our research page: There Is No Aesthetic Surprisal Arc. The preprint is openly available on ResearchGate, and the full analysis code with a reproducible pipeline is on GitHub. For the companion read on how a real-corpus approach reads these same human-versus-machine textures, see the 2,590-essay corpus deep dive.