What do you think?
Rate this book
Measuring Up: What Educational Testing Really Tells Us
How do you judge the quality of a school, a district, a teacher, a student? By the test scores, of course. Yet for all the talk, what educational tests can and can’t tell you, and how scores can be misunderstood and misused, remains a mystery to most. The complexities of testing are routinely ignored, either because they are unrecognized, or because they may be—well, complicated. Inspired by a popular Harvard course for students without an extensive mathematics background, Measuring Up demystifies educational testing—from MCAS to SAT to WAIS, with all the alphabet soup in between. Bringing statistical terms down to earth, Daniel Koretz takes readers through the most fundamental issues that arise in educational testing and shows how they apply to some of the most controversial issues in education today, from high-stakes testing to special education. He walks readers through everyday examples to show what tests do well, what their limits are, how easily tests and scores can be oversold or misunderstood, and how they can be used sensibly to help discover how much kids have learned.
368 pages, Hardcover
First published May 1, 2008
24 people are currently reading
214 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 16 of 16 reviews
May 12, 2013
What my local school district doesn't understand about educational testing could fill a book. Oh wait! It does!
Harvard Professor and Psychometrician (measurer of abilities or intelligence) Koretz tells us why we shouldn't take standardized educational testing nearly as seriously as we do. There's nothing wrong with testing, as long as we know its limitations. Testing is fraught with pitfalls, and believing that standard test results are the be-all end-all measure of a good education will lead you down the wrong educational path and harm your students. For example, Koretz says "if students score well on math tests but appear bored to tears in math class, take their high scores with a grain of salt, because an aversion to mathematics will cost them later in life, even if their eighth-grade scores are good."
Amen.
Harvard Professor and Psychometrician (measurer of abilities or intelligence) Koretz tells us why we shouldn't take standardized educational testing nearly as seriously as we do. There's nothing wrong with testing, as long as we know its limitations. Testing is fraught with pitfalls, and believing that standard test results are the be-all end-all measure of a good education will lead you down the wrong educational path and harm your students. For example, Koretz says "if students score well on math tests but appear bored to tears in math class, take their high scores with a grain of salt, because an aversion to mathematics will cost them later in life, even if their eighth-grade scores are good."
Amen.
May 23, 2011
An incredibly readable view of standardized testing and the need for careful use of scores and tests to give a true (rather than politically popular) view of student learning in the U.S. Each chapter covers a different testing difficulty - like bias or accomodations for students with disabilities - and sneaks in some test-creator vocab and some really simple analogies to make things clear. Koretz is not an anti-test zealot, just a Ed prof trying to point out the positives and negatives in testing interpretation, and the real importance of getting all this right.
I weep to think how little use the average educator can make of the good data in this book. Seems to me education is more and more in the hands of politicians and administrators who are less interested in hearing about appropriate uses of tests than they are in what will look best in the media.
I weep to think how little use the average educator can make of the good data in this book. Seems to me education is more and more in the hands of politicians and administrators who are less interested in hearing about appropriate uses of tests than they are in what will look best in the media.
March 7, 2013
Insightful and very fair-minded book on the ins and outs of educational testing. "Measuring Up" is also as readable a book on testing as you'll encounter. Koretz is a clear writer who is adept at explaining very complex topics in understandable terms. I can't say I enjoyed this book but I can say I learned a lot and recommend to anyone who wants to think more deeply about current debates about how testing and accountability.
One could say the author is "pro-testing" in the sense that he makes his living researching and teaching about testing. But, Koretz is also quite blunt about the misuses of testing and the very real limitations of what we can safely conclude from any given test or assessment.
One could say the author is "pro-testing" in the sense that he makes his living researching and teaching about testing. But, Koretz is also quite blunt about the misuses of testing and the very real limitations of what we can safely conclude from any given test or assessment.
July 15, 2019
There is a fun irony to assigning a grade to a book about educational assessments. Measuring Up is an attempt to explain and make sense of the phenomena of standardized testing in schools - what information can we reliably gain from testing and what are its limitations. The main points summed up:
*While the phrase “standardized testing” is often used pejoratively, it simply means a test that should in theory be designed uniformly enough to produce information that is reliable.
*Just because the results of a test are reliable, it does not mean the conclusions are valid.
*The original design of testing in schools was low stakes with the intent of collecting good data. Political influences since the Cold War continuing into the era of No Child Left Behind have resulted in a shift from low stakes testing to high stakes testing with consequences to teacher salaries and school funding.
*Test questions can be biased for a plethora of reasons. Test results favoring certain groups do not necessarily indicate bias in test design, but this is not sufficient reason to become complacent in eliminating the possibility that bias exists.
*Evaluating growth in LEP and special needs students using standardized metrics is enormously challenging, but it needs to be addressed nonetheless if we want to adequately serve these populations.
*There is something called Campbell’s Law in social sciences which states that the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. As low stakes testing becomes high stakes, we are sacrificing good data for inflated scores because teachers and principals are given heavy incentives to either outright cheat or teach to the tests.
So is this a worthwhile book to read? I’m not sure I can answer that. Ideally, this book should give a clear explanation to parents, educators, journalists, and voters of what they need to know about educational testing. I am a teacher with a math background, so while I was impressed with Koretz’s explanations, I am an easy choir to preach to. I’d be curious to hear from parents and teachers without a math background if they found this easy to follow.
The 2 obvious criticisms I could make are that while he doesn’t shy from giving his overwhelmingly negative opinions about the potentially arbitrary nature of aligning core curriculum to standards based grading, he fails to make a convincing argument for why normative (percentile based bell curve) scoring is more fair or informative.
The 2nd critique is that nearly every anecdote he tells in the book smacks of “This is the time I was the smartest person in the room and I told this person off” or worse yet “These are all the times I was smarter than my son’s teachers.” I don’t enjoy interacting with people irl whose stories all cast them as a witty hero, and I suspect that kind of smug personality might not be attractive to a person who was not comfortable with their own math skills trying to understand a complex topic. Then again, rolling my eyes every 20 pages or so probably kept me engaged with what could be seen as a somewhat dry topic, so maybe it’s a good thing? Read it and decide for yourself.
*While the phrase “standardized testing” is often used pejoratively, it simply means a test that should in theory be designed uniformly enough to produce information that is reliable.
*Just because the results of a test are reliable, it does not mean the conclusions are valid.
*The original design of testing in schools was low stakes with the intent of collecting good data. Political influences since the Cold War continuing into the era of No Child Left Behind have resulted in a shift from low stakes testing to high stakes testing with consequences to teacher salaries and school funding.
*Test questions can be biased for a plethora of reasons. Test results favoring certain groups do not necessarily indicate bias in test design, but this is not sufficient reason to become complacent in eliminating the possibility that bias exists.
*Evaluating growth in LEP and special needs students using standardized metrics is enormously challenging, but it needs to be addressed nonetheless if we want to adequately serve these populations.
*There is something called Campbell’s Law in social sciences which states that the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. As low stakes testing becomes high stakes, we are sacrificing good data for inflated scores because teachers and principals are given heavy incentives to either outright cheat or teach to the tests.
So is this a worthwhile book to read? I’m not sure I can answer that. Ideally, this book should give a clear explanation to parents, educators, journalists, and voters of what they need to know about educational testing. I am a teacher with a math background, so while I was impressed with Koretz’s explanations, I am an easy choir to preach to. I’d be curious to hear from parents and teachers without a math background if they found this easy to follow.
The 2 obvious criticisms I could make are that while he doesn’t shy from giving his overwhelmingly negative opinions about the potentially arbitrary nature of aligning core curriculum to standards based grading, he fails to make a convincing argument for why normative (percentile based bell curve) scoring is more fair or informative.
The 2nd critique is that nearly every anecdote he tells in the book smacks of “This is the time I was the smartest person in the room and I told this person off” or worse yet “These are all the times I was smarter than my son’s teachers.” I don’t enjoy interacting with people irl whose stories all cast them as a witty hero, and I suspect that kind of smug personality might not be attractive to a person who was not comfortable with their own math skills trying to understand a complex topic. Then again, rolling my eyes every 20 pages or so probably kept me engaged with what could be seen as a somewhat dry topic, so maybe it’s a good thing? Read it and decide for yourself.
July 28, 2021
A nice introduction to measurement issue in general.
May 15, 2025
Solid book on the theory behind assessments. Great-read for psychometricians, teachers, and policymakers.
April 15, 2015
It's educational, but not very engagingly written - a handful of entertaining anecdotes aside, my eyes glazed over every few sentences, and the desire to skip ahead was ever-increasing (much like high-stakes test scores, ha!)
Also, he never really looks some possible counterarguments to many of his points*, such as e.g. using subcomponents of a single test to measure different bits of achievement - so you have a few straight-up arithmetic problems to see if they know arithmetic, and then a more complicated word problem or two to see if they can solve word problems, thus giving you both experimental and control items in the same test. (you still won't account for the kid not feeling well or the day being unusually nice outside, but at least it'll give you a much clearer idea of what portions of the overall knowledge kids have trouble with)
* At least in this book. At least I think so - I may have nodded off here and there while reading.
Also, he never really looks some possible counterarguments to many of his points*, such as e.g. using subcomponents of a single test to measure different bits of achievement - so you have a few straight-up arithmetic problems to see if they know arithmetic, and then a more complicated word problem or two to see if they can solve word problems, thus giving you both experimental and control items in the same test. (you still won't account for the kid not feeling well or the day being unusually nice outside, but at least it'll give you a much clearer idea of what portions of the overall knowledge kids have trouble with)
* At least in this book. At least I think so - I may have nodded off here and there while reading.
October 29, 2020
Textbook for my graduate class. Well respected and clearly written. Not a 'fun' read so much, but well written (narratively) to the point where it's not really a 'textbook.' Lots of no-nonsense analysis of the mind-bogglingly complex discipline of educational assessment.
November 9, 2011
Everyone seems to be obsessed with test scores so this is a great read for anyone interested in education. Actually it's a must read for anyone who wants to be able to speak intelligently about educational testing. The more you know about this topic you more you realize that almost everyone talking about it doesn't know much. I took Koretz's class at HGSE 9 years ago about this topic (loved it!) and I read the book last year and it was a great reminder of the complexities of testing, problems with high-stakes and questions about predictive validity. Koretz is a super nerd (I mean that in only the best way sir) who does a great job of helping us regular nerds understand this complex topic. I wish politicians and school board members had to read this. Oh, and journalists too!
July 1, 2010
Chapter 10 (Inflation of Test Scores) gives the best evidence I have ever encountered about the effects of high-stakes testing on the selection of subject matter taught (teaching to the test). When a school begins using a particular test, scores rise during the next few years. If the school switches to another achievement test, scores drop.
"...students replace mastery of material emphasized by the old test with mastery of material emphasized by the new one, not really reaching a higher level of achievement in the larger domain from which the test samples."
We need to stop fooling ourselves that high test scores indicate real education is taking place.
"...students replace mastery of material emphasized by the old test with mastery of material emphasized by the new one, not really reaching a higher level of achievement in the larger domain from which the test samples."
We need to stop fooling ourselves that high test scores indicate real education is taking place.
July 20, 2009
Dan is a wonderful teacher...I took his class in grad school, and found myself pining for lectures and powerpoints as I read the book. Unfortunately, most of the concepts are better explained graphically rather than in narrative form. The medium just isn't suited to this topic.
April 3, 2010
This is a great introduction to assessment and measurement in education. I just have too many awesome novels to read for this book to keep my attention.
January 1, 2012
Anyone who claims to be data-driven or data-informed needs to read this book. It's an eye opener. One of my key sources in my dissertation.
July 3, 2013
I wish all professors could write as well as Prof. Koretz.
December 7, 2016
It's a text book, so not a riveting read, but very well researched and written. He's most definitely up to speed on the latest solid research on educational testing.
Read
December 10, 2009I feel a bit more vindicated about greatly disliking the WASL, SAT, and GRE, et al.
Displaying 1 - 16 of 16 reviews