Pulling Back the Curtain
For those unafriad of arousing the wrath of state education departments everywhere, the Education Sector has just released Making the Cut: How States Set Passing Scores on Standardized Tests. The short "explainer" (it's only 10 pages) will tell you just about everything you want to know about the different methods states use to set the cut scores for their standardized tests.
Cut scores definitely deserve more attention than they get from the media (it's much sexier to report the results of the cut scores: "Scores plunge, thousands fail!" or "Governor lauds, claims credit for spike in test scores") so this report is a welcome first step in educating students, parents, and the public at large about how these seldom spoken of numbers have a huge impact on the tone of the education news we receive.
It's really about time someone took states to task over their cut scores and pulled back the curtain on the whole enterprise. Rotherham (author of the report) calls for all states to put information up on the web that explains what their current cut scores are and how they were developed. According to Ed Sector's analysis, only 60% of states currently offer such information on their websites. While it's certainly important for the public to know that information, as Rotherham himself notes in the report, knowing a cut score in the abstract is totally meaningless--it's just a number. A low cut score doesn't mean it's easier to pass a test, it might just mean the test is (justifiably) harder; and a high score doesn't mean it's harder to pass a test, it could just be an easy test. The states can post all the cut scores they want on their websites (here I'll throw out a few too: 45, 69, 32. Feel better?) but without having the test they are associated with they are worse than meaningless and may be harmful--they're creating the illusion of full disclosure.
When I was at the Commission on NCLB hearing last week, Checker Finn wondered aloud about when the lawsuits to compel states to release copies of the state tests developed and paid for with our tax dollars would start being filed (the tests are currently kept in a lockbox somewhere (maybe the same one as our Social Security dollars) and way from prying eyes). Not to sound like a broken record, but the problems stemming from the secrecy surrounding tests and cut scores are exacerbated by the fact that there are 50 tests with 50 different cut scores. At least if there were a single test (even if you never saw it), you would be able to compare your state's cut score to that of another state and you would at least get the benefit of a relative assessment. Oh well. For now, you'll have to settle for just learning about where cut scores do and don't come from.