What Actually Changes as English Proficiency Grows

Ask how someone's English gets better and you will hear a familiar story: they learn longer words, build longer sentences, fold in more clauses. It is an intuitive picture, and it is mostly wrong. We took 3,300 learner essays graded from beginner to near-native, plus a small set of essays by native writers, and measured what actually changes as English proficiency grows. One axis dominates everything: accuracy. Errors fall almost five-fold across the ladder and, at the top, land right on the native baseline. Two quieter changes ride alongside it: vocabulary slowly gets rarer, and writing gets structurally more elaborate. But the elaboration is phrasal, not clausal: advanced writers pack more into their noun phrases, they do not pile up subordinate clauses. And the one error that refuses to go away is the humblest of all: punctuation.

What data shows how English writing changes with proficiency?

The data is the W&I+LOCNESS corpus, the public dataset behind the 2019 BEA shared task on grammatical error correction. It collects 3,300 learner essays labelled by CEFR level, the six-band scale that runs A1 (beginner) → A2 → B1 → B2 → C1 → C2 (near-native), plus 50 essays by native English writers as a reference point.

Asking how writing changes with proficiency takes a lot of real writing, graded consistently, by authors across the whole range. Crucially, each essay carries gold human error annotations as well as the text itself: a trained annotator has marked every error and its type. That lets us measure accuracy directly, from human judgements, rather than guessing at it with a machine.

One caution sits over everything that follows, so we put it up front. This is cross-sectional, not longitudinal: the essays at each level are written by different people. We are watching how these measures co-vary with proficiency across many writers, not following one writer as they improve over years. That is a real limit, and we say so plainly throughout.

What changes most as English proficiency improves?

Accuracy changes most, and it changes steeply. Beginner (A1) essays carry about 20.7 errors per 100 words while near-native (C2) essays carry about 4.0, an almost five-fold drop that is the single most pronounced pattern in the data. Error density is the cleanest trend of every measure we computed, and it points one way: down.

What makes it more than a trend is where it lands. The native essays sit at roughly 5.0 errors per 100 words, and by C2 the learners have essentially converged on that baseline. (Yes, the most advanced learners score slightly cleaner than the natives, but the native set is small and a different genre, so we read that as "they have caught up," not "they have overtaken.") If you had to name what "getting better at English" means in this corpus, in one number, this is it: you stop making errors, until you make about as few as a native writer does.

Which English errors are the hardest to get rid of?

Punctuation and prepositions are the hardest to shed: punctuation shrinks only about 2× between beginner and advanced writing and prepositions about 2.5×, while orthography collapses roughly 8×. Accuracy improves overall, but not uniformly across error types, and the uneven part is the interesting part.

We grouped the CEFR levels into broad beginner (A), intermediate (B) and advanced (C) bands and watched each error category shrink at its own pace.

The pattern is clean. The surface-mechanical mistakes (getting letters and capitals right, spelling words correctly, choosing the right tense) fall away fastest, because they are the most teachable and the most rule-bound. What lingers are the judgement-heavy choices: which preposition a verb takes, where a comma belongs. These do not reduce to a rule you memorize; they take a feel for the language that arrives late.

There is a striking consequence. Because the total error budget shrinks so much while punctuation shrinks so little, punctuation becomes the single largest error category for advanced learners. It rises from about 15% of a beginner's errors to roughly 22% of an advanced learner's, not because advanced writers punctuate worse, but because they have fixed almost everything else. For a teacher, that is an oddly precise piece of guidance: with a strong student, the highest-yield place to look is the comma.

Do advanced writers use longer, more complex sentences?

Advanced writers do not stack up subordinate clauses. Their complexity grows inside the noun phrase instead: noun-phrase modification and average dependency distance both climb steadily with proficiency, while clausal subordination stays essentially flat, the weakest trend of everything we measured.

Here is where the intuitive story breaks. We tracked noun-phrase modification (how much detail gets packed around nouns: adjectives, prepositional phrases, relative clauses) and average dependency distance (a measure of how far apart grammatically-linked words sit, which rises as sentences get more intricately wired). Writing does get more structurally complex as proficiency rises, but only along one of two possible routes, and stacking subordinate clauses inside one another, the thing most people picture when they say "complex sentence", is not the route it takes.

This is not a fluke of one dataset. It is an independent confirmation of a well-known argument by Biber, Gray and Poonpon (2011), who proposed that advanced academic writing grows dense not by subordination but by phrasal compression: loading information into elaborated noun phrases. Their claim was based on register comparison; our data shows the same shape emerging across learner development. Mature writing says more per noun, not more clauses per sentence.

One popular shortcut takes friendly fire here. Mean sentence length (the go-to proxy for "complexity" in countless rubrics and tools) barely tracks learner proficiency at all in this data. It only jumps once you reach the native essays, and stays nearly flat across the entire A1-to-C2 learner range. A writer can grow enormously in skill without their sentences getting meaningfully longer. If you are judging development by sentence length, you are mostly measuring nothing.

Does vocabulary change as English proficiency grows?

Vocabulary changes too, on a slower schedule. Diversity rises as advanced writers reach for a wider range of words, and the words themselves get rarer: the share of lower-frequency words climbs from about 10% to 16% across the ladder.

It is a real, monotonic signal, just a gentler one than accuracy. Proficiency reads partly as precision (fewer errors) and partly as range (less reliance on the most common few thousand words).

Why did a humanizer lab study learner English?

Learner English earns the attention because "longer sentences mean better writing" quietly drives so much teaching and automated scoring, and the claim turns out to be testable. Our own training data is 2,590 real student essays, so telling one kind of human writing from another, across levels and registers, is the work we already do.

There is a sharper reason too, and it is uncomfortable. AI detectors disproportionately misflag non-native English writers. The same careful, formal, slightly-conservative prose that earns a high CEFR band (even, controlled, low in surprise) is the texture detectors mistake for machine output, and second-language students pay for it. Understanding what advanced learner writing really looks like is part of why we built our free AI detector to be read with humility rather than treated as a verdict. The same instinct drove the two siblings to this study: It Isn't Delve, which found that the famous "AI vocabulary" is a passing fashion rather than a reliable tell, and There Is No Aesthetic Surprisal Arc, which found that the rhythm of writing doesn't give AI away: only its flatness does, the very flatness a fluent non-native writer can have honestly.

What are the limits of this proficiency study?

The first limit is that the study is cross-sectional: different writers sit at each level, so it observes how proficiency co-varies with these measures across people, not one person changing over time. We would rather under-claim, so here is exactly what it does and does not show.

A genuine longitudinal study could, in principle, find a different developmental path inside individuals.

The CEFR labels are holistic (a single overall judgement per essay, not a precise measurement), so the bands are coarse by nature. The native baseline is small (just 50 essays) and a somewhat different genre, which is why we treat the native comparisons as suggestive rather than settled. And it is a single corpus: a clean, well-annotated, public one, but one source. The honest reading of every number here is "this is what the data shows, robustly, in this corpus," not "this is a law of English." The point of putting the code and data in the open is that anyone can test how far it travels.

Can anyone reproduce these measures?

Anyone can. The dataset (W&I+LOCNESS) is public and the analysis runs from a single command, so the same 3,300 essays can be pulled and re-measured rather than taken on faith.

Each measure, the figures behind it and a candid account of what this corpus cannot show live in the preprint on ResearchGate (DOI 10.13140/RG.2.2.35861.90086), and the complete analysis code with a one-command reproduction is openly available on GitHub. The tidy story that better English means longer, more clause-heavy sentences does not survive contact with the data; what survives is duller and more useful: accuracy converging on natives, complexity growing in the noun phrase, and the comma quietly outlasting every other mistake.