« October 2009 | Main | December 2009 »

November 30, 2009

Pop Quiz: What's Worse Than Standardized Tests?

The answer depends on who you ask, of course, but there is growing opinion based on a Virginia state-wide experiment in portfolio assessment that standardized testing may not be the worst way to measure student achievement after all.

The Virginia Grade Level Alternative, as the portfolio assessments are called, offers schools a second way to determine whether students are reaching proficiency that is based not on standardized test scores but rather compilations of student work over the course of a school year.

The idea is one which has received attention in various circles, notably during the campaign trail when then-candidate Obama suggested that portfolio assessments could replace standardized tests at least in part.

In Virginia, the portfolio assessments are available to some special education students and students for whom English is a second language. Instead of asking these students to demonstrate proficiency over reading, writing, math, and science using the same tests as other students, teachers document these students' learning throughout the year in a binder of class work, including worksheets, quizzes and writing samples. Decisions are made at the end of the year whether the binder merits a passing mark.

In theory, the idea sounds reasonable, since it is often unfair to ask students with severe learning disabilities and students who are just learning English to complete the same tests as their peers. It's also true that not every student is able to fully demonstrate their mastery of important concepts through tests, and that multiple measures of assessment are typically more accurate than single measures like standardized tests.

The problem is in practice. In Virginia, the number of students taking the alternate method of portfolio assessment is growing rapidly without much proof yet that a "proficient" score on the portfolios stands for actual mastery of skills. At Lynbrook Elementary School in Fairfax, VA, for instance, the number of portfolio assessments has increased from a handful in 2007 to nearly 100 this year and passage rates have skyrocketed from 41% to 100% among students with disabilities and 69% to 97% among English language learners.

More stressing is the fact that these students are not participating in portfolio assessments in addition to the standardized tests, they are doing them instead of the traditional measure. The result, sadly, is that the state's astronomically improved passage rates among students taking portfolio assessments may mean, at worst, that the portfolios are rubber stamps given to move any student along regardless of how much they have learned--or at best that we don't know how well schools are teaching these students.

The absence of this independent source of verification--i.e. if students were doing well on portfolios AND also demonstrating some growth on standardized tests (using read-along and other accommodations where necessary)--means that there is reason to be skeptical of statistics like the Washington Post's finding that in Fairfax County alone, proficiency rates were higher among students with disabilities than among ordinary students.

To be sure, it would be terrific news if it were actually the case that disabled and ELL students were actually outperforming their peers. And Virginia deserves applause for spending the extra dollars and effort to develop and to train educators to use the portfolio assessments to begin with.

But where the state can so easily institute a small reform--requiring portfolio students to also take standardized tests and perhaps to use a combination of the two measures to determine whether a student has met proficiency--but fails to do so, advocates of special education and ELL students are right to be worried that what is really happening is a sleight of hand where schools are giving passing marks to all students without regard for how much they have been taught.

November 18, 2009

So Much For That Great Teacher Shortage

Researchers have been predicting a precarious outlook in American classrooms for years on account of a looming teacher shortage.

Just as a severe drought has major downstream impacts on numerous aspects of life, so too has it been argued that a shortage of teacher candidates would wreak havoc on student learning in America. The shifting demographics of the baby boom era, comparatively increasing salaries in other professions, and a general decline in the quality of workplace conditions for teachers, it's been argued, would lead to a shortage that in turn would impact student achievement as principals and school districts struggle to find any warm body to stand in front of a classroom.

So much for that doomsday scenario: the economic downturn has led to not a shortage of teacher candidates, but a surplus.

What will the impact of this surplus be? One might be tempted to conclude that if a teacher shortage would lead to lower teacher quality and reduced student outcomes then a surplus of available candidates might logically lead to increased teacher quality and student outcomes, as principals and districts suddenly have their pick of the litter in terms of which teachers to hire. As one district official pointed out, "It is a tougher job market, and you get applicants that you might not normally have because of the economy."

And in general, that labor market logic should work. After all, if you're the boss of a company and you need to hire an employee, much better to have 20 people to interview and pick from than 5. Even if you don't have any job openings, your new-found ability to shop around might enable to you to replace a previously under-par employee with a better one since so many candidates are newly on the market.

But in U.S. public education, I'm afraid that the shift from a teacher shortage to a teacher glut operates only in a one-way ratchet for students: even when schools in theory have more candidates to hire from, it's highly likely that there will be little measurable improvement in the quality of a child's teachers in the aggregate.

The main reason is that it is so hard to replace ineffective teachers. To be sure, schools that have new openings to fill will likely have greater success these days in finding a good teacher than five or ten years ago, as the AP article discusses. But administrators in schools with bad teachers who might want to replace those teachers are out of luck to the extent that removing bad teachers, even for cause, is a nearly impossible enterprise due to teacher tenure rules.

A Kansas deputy superintendent, John Black, put it best, "Now we have these great applicants wanting to teach, and we don't have jobs to offer them." With due respect to Mr. Black, I'd point out that they do have jobs to offer them, but those jobs just happen to be filled currently by low-performing educators who are all but impossible to remove.

November 11, 2009

Message To Michigan Lawmakers: Do Your Job

With $212 million in state budget cuts slated to hit Michigan's schools next month, a gathering of roughly 1,500 parents, students, and concerned citizens coalesced in Lansing yesterday to demand that law-makers take action.

The rally yesterday, with video below, was organized by Save Our Schools Michigan.

The $212 million in cuts, which amount to a funding reduction of $127 per student, are scheduled to come as a result of an order signed by Governor Jennifer Granholm late last month in response to falling state tax receipts. At the time of her order, the Governor called on students and parents to pressure legislators to raise taxes in order to make up the shortfall, a power that she as the state's executive does not possess.

But those aren't the only cuts that Michigan schools will be dealing with this year--an additional $165 in per student spending was cut earlier in the year by the legislature itself, for a grand total of $292 lower per pupil spending. That amount means, absent a legislative solution to raise revenues, that the state will guarantee schools 4% less per student than it did in 2008-2009, when the state minimum was $7,316 per pupil.

How much does the money matter? This is often a subject of great debate among scholars in the area, as there is some evidence that indicates that more money alone is not a good predictor of improved student success.

But while it is certainly true that more school spending is not a sufficient condition for school improvement, it's hard to argue with the proposition that sufficient resources are a necessary condition. After all, Michigan's budget cuts mean that an elementary school of 400 students will have roughly $120,000 less to spend this school year; in a high school of 1,000 student the shortfall grows to $300,000.

Where will those savings come from in ways that don't hurt students, especially at a time when our schools are struggling to keep up with international competition and the rising demands of our global economy?

Here's hoping that Michigan lawmakers heed the call of yesterday's protesters and come up with a way to bridge the gap, even if it means raising taxes. After all, what could be more important to our nation's future than the quality of education we provide to our future doctors, teachers, and leaders?

November 04, 2009

Source Confusion: What's Wrong With DC Schools

Amidst calls for critically-acclaimed Chancellor of Schools Michelle Rhee to defend her decision to fire 266 teachers last month, the national headquarters for the city's teacher union issued an interesting advertisement in Friday's Washington Post (full ad pictured below; click on it for a zoom-able view).

The conflict over Ms. Rhee's leadership decisions and style is much a debate about style as it is about substance; most of the hard questions from D.C. council members at a hearing last Thursday reflected a concern over her autocratic decision making process and not about her goals and intentions. Which is what makes the American Federation of Teachers advertisement so interesting.

If you look at the ad, the first thing to notice is that the message is not readily apparent--it takes at least some careful inspection to decipher the specific meaning and criticism against Rhee. In today's era of fast-paced, hard-hitting media I wonder how many people even bothered to figure out the whole meaning of the ad.

But setting that aside, one has to wonder about the merits of the point that the AFT is making, on at least two fronts.

First, is the AFT saying that the only thing stopping DC's students from making significant progress is the fact that Rhee and the district's administrators are not collaborating with teachers in a respectful manner? If that's the case, one has to wonder whether the AFT would describe the city's schools as successful in the pre-Rhee years, when more union-friendly school chancellors like Clifford Janey, Arlene Ackerman and others.

An honest response from the AFT would have to concede that the schools were no better during those years where collaboration and respect existed. It simply cannot be the case that the fate of student learning rises and falls with how nicely superintendents treat teachers and their union reps. As a simple example, a district would be remarkably "collaborative" and "respectful" if it cut teacher work hours in half and refused to fire any teachers even if they were negligent--but it's hard to see how students would benefit from those changes.

Second, and perhaps more important, I think the AFT ad ignores the fundamental question with DC student achievement--and in doing so, it accidentally sends a boomerang attack at Rhee that bounces back with equally forceful criticism against the union itself.

You see, Michelle Rhee isn't autocratic and stubborn in her interactions with teachers just for heck of it; she's not taking a hard line position with the union just to give them a hard time. Relations are strained because Rhee and the union disagree about key areas of policy concern. Should chronically bad teachers be fired? Should good teachers be paid more than bad ones? Should the city allow teachers who are inspiring remarkable learning gains among their students to earn in excess of six figures?

Rhee says yes to all of these questions; the union so far has said no. And therein lies the problem: "collaboration" and "respect" are a two-way street. In identifying the DC public school reform equation as lacking the two ingredients of teacher collaboration and respect, the AFT national office is as much criticizing DC teachers for failing to meet Rhee halfway in a respectful manner as much as it is criticizing Rhee!

The problem for the union, of course, is that this hide-the-ball advertisement is more palatable than a straight up response to Rhee's substantive policy suggestions. An AFT full-page ad saying, "All teachers should be paid the same regardless of how well they teach" would not win over many hearts and minds...