In case you didn’t know, there are established algorithms out there that will estimate the grade level of any piece of writing in the English language. A couple of them are even embedded in Microsoft Office. Taken singly, they’re very limited in value, but there are some big eye openers to be had by looking at the various grade-level algorithms in combination.
Is reading level in our society going down?
Apparently, yes. The US government measures reading level every once in a while on large population samples and the trend of the last 20 years doesn’t look good. A study back in 1992 showed that adults were reading between 8th and 9th grade level, but the US Dept. of Health reported in 2008 that the average for adults was just about 7th grade. Meanwhile, scores on the critical reading portion of the SAT fell in 2011 to its lowest level since that portion of the test began in 1995. Also, the National Assessment of Educational “Progress” showed 12th graders in 2010 had lower reading scores than did the class of ’92. So, here we have three different measures comparing roughly the same time points and all pointing to the conclusion that Millennials are reading at a lower level than Boomers and Gen-X’ers did.
Is there a standard measure we can get our hands on?
One caveat in the aforementioned studies is that each one used a different formula to measure the grade level of a text. Most of these measures revolve around the same parameters of word-length, sentence-length, and vocabulary usage. However, when it comes to calculating a final score, there is a great deal of variation between the measures.
I’ve personally tested nine of them and found that a single text can vary from 4th to 14th grade (that’s right — a ten-grade difference) depending on which algorithm you choose. Worse, the same algorithm is not always the high-scoring one. The algorithm that delivers the highest grade estimate in one case, might turn out to give the lowest estimate on a different text.
So, which one is right? The academic community is not much help in answering that. Educators and scholars have not reached a consensus whatsoever on which algorithm is best to use, and they are still at work inventing new ones.
Is measuring reading level an art or a science?
Like most things in the “human sciences,” grade-level estimation cannot be made completely scientific, neither in the sense of being 100% empirically-derived nor in the sense of being logically deducible from self-evident premises. This is because there are deep-rooted choices to be made in what should matter most in this kind of measurement.
In designing a grade-level algorithm, one must answer questions such as: Is handling of complex connectives and long sentences more important than knowing a lot of fancy words? At what grade level should a writer start to use a lot of subjunctive mood constructions (e.g., hypothetical statements such as, “If Hitler had escaped the bunker we would all be speaking German”)? These and many similar questions have no clear answer today and each one of them can (and does) spark numerous conference papers, academic debates, and doctoral dissertations.
In an effort to make a practical, usable test, we’ve implemented a handful of the different algorithms and updated their vocabulary lists (some of which have not been officially updated since the 1970’s). Today, we run these in parallel, then discard the highest and lowest scores of the bunch before taking the average of the rest. This is a standard method in the social sciences whenever you are confronted with several diverse indicators of a single element that you are trying to measure: simply check the central tendency after discarding the outliers. Based on this we can get a more reliable, useful, and consistent grade-level score for texts. When we’ve finished tuning this meta-score, we’ll make it available here for you to try.
What grade level are publishers aiming for?
It’s been said that writing in the mass media is often aimed at about 6th-grade level, with hard news a bit higher, at around 8th-grade level. Much of the so-called “teen” literature being devoured today by high-schoolers, such as the Hunger Games novels, are actually around 5th grade level, meaning that high-schoolers enjoy reading at a level beneath themselves — and that book publishers are catering to this preference.
Nonetheless, I find that many writers think they are writing on a higher grade level than they really are. In my career I’ve had the privilege to meet and/or work with quite a few professional journalists (including folks from CNET, HuffPost, San Jose Mercury News, PCWorld, MacWorld, Red Herring, Dow Jones, etc.). In the handful of cases where I’ve run the grade-level algorithms on their texts and told them personally what their scores are, they’ve been surprised, even taken aback. Every single one. The conversation goes something like this:
Journalist: “What? My writing is only 7th or 8th grade? No way!”
Tim: “Look, 97% of the words you used are included on a list of words known to 80% of 4th graders. Yes, 4th graders. Your average sentence length and your average word length are both around what we see in a middle-school history text book. Still think the measure is way off?”
Eventually I’ll get a reluctant, dissatisfied acquiescence. Why the perception gap? I think journalists confuse their reading level with the maturity level of their audience.
J: “But I’m not writing to 8th graders!”
T: “Right, you are writing for adults who are high school or even college grads. But that doesn’t mean you’re writing at higher than 8th-grade level, nor should you. The adult public reads at an 8th-grade level, and indeed you are writing at that level. Congratulations, you got it right.”
J: “But most 8th graders couldn’t understand what I write about!”
T: “Ah, don’t confuse the level of background knowledge required to absorb your subject matter, with the vocabulary and grammar skills needed to read your text. Competent young readers might not possess the world-knowledge in 8th grade to absorb your material fully, but they have no problem reading it.”
Does reading level matter, other than to educators?
You bet it does. In a previous study, I found that you can predict the social amplification level on Facebook and LinkedIn by looking at the grade level of a post. In one of these social networks, writing at a higher grade level can increase sharing, while on the other, it has the opposite effect. (You can guess which is which!) And a study done a few years back at Youngstown State University showed that consumer medical information (on sites such as WebMD) is actually written at a little bit too high a grade level for the average consumer to absorb, meaning that public health education might actually improve by writing at a lower grade level!
Grade level can help match the right text to the right readers, while helping authors determine whether their writing is well-tailored to fit their target audience. That’s why my team is really looking forward to rolling out our own version of a more reliable grade level measure. Right now, early beta testing of it is available to our partners and other developers who want to run experiments. If you’re interested, just drop us a line.
And, oh yeah, the grade level of this post? 8.8