Inside Textmetrics: the work behind a fair reading assessment passage

When we first wrote our assessment passages, over a year ago now, we calibrated them as best we could using the curriculum information we had at the time. In the interim; from research, from seeing real children's data, from a deeper understanding of what our texts should contain, we have begun to build a better, fairer way to construct them, so that they sit as close as we can get to what children at a given stage should actually be reading. We call it Textmetrics.

What this is really about is simple. When a child sits down to read for one of our checks, we count the words they get right. But the words we hand them do a lot of the work before they even begin. Schemes in the US often use a mathematically based schema for deciding text difficulty. In the UK, our tendency tends more towards a qualitative judgement. Textmetrics exists to ensure a fairness across the assessment texts within and between each stage, so that they are as close to similar in difficulty as we can get.

Step one: we decided what "harder" actually means. We fixed on the small set of features we know change how a passage reads aloud. How long the words are. How common they are. How many long words pile up close together. How the sentences run. We measure each of those on its own, and each one has to be up to scratch on its own terms. We don’t weight one as more important than any other.

Step two: we built a ruler. We took a large body of children's language and ranked every word in it by how often children actually meet that word. This was from the ones a five-year-old trips over every day to the ones they may only meet a handful of times in a year. That ranked index is what we measure every passage against. It gives us an honest yardstick instead of a hunch about whether a word is "hard".

Step three: we locked a target for every stage. Birch, Willow, Hazel, Yew Entry, Yew, Chestnut, Oak, Sycamore, Cedar – each stage on the ladder got its own framework expectations with its own numbers written on it: the average word length it must hit, the commonness of its words, the density of long words, how long the passages should run in words, and which should be story and which should be fact. Six passages to a stage, four fiction and two non-fiction, with one held up as the benchmark the others are measured against.

Step four: the writing, then the improving loop. We write a passage, score it against the index and the framework, see which way it is drifting, and change it – longer words here, a rarer word there, a sentence broken in two – then score it again. Round after round, until the set matches the framework. And we insist each stage sits a clear step above the one below it.

And finally: we read them ourselves. Because the ruler only sees part of it. You can score a passage until every number is perfect and hand a child something that still reads like it was assembled rather than written - the words the right length, the right commonness, the right density, and somehow not like anybody talking.

So before anything goes into a check, we read each passage aloud, the way we'd read to a class. Adult practitioners - teachers who've stood in front of a child and listened to them read; that's the whole qualification. Have you ever listened to a child read something that was technically right and somehow dead? That's what we're listening for - the stumble the index can't see. We ask whether it sounds like a story a child would actually want to read. If it doesn't land, we change it - and if that moves the score, it goes back around the loop until the ruler and the ear agree.

That's the qualitative gate, and it sits on top of every number above. The framework keeps us honest that the passages are fair against each other - that one stage isn't harder than the one below it by accident. The read-aloud keeps the framework honest that harder never has to mean worse to read. The two of them are what 'fair' means to us: fair to score, and fair to read.

These are the passages we then use in our checks. If you'd like to see how we put them to work with a child, how to assess reading fluency takes you through the assessment from start to finish.