The evaluations themselves are typically of little value — a single, fleeting classroom visit by a principal or other building administrator untrained in evaluation wielding a checklist of classroom conditions and teacher behaviors that often don't even focus directly on the quality of teacher instruction. "It's typically a couple of dozen items on a list: 'Is presentably dressed,' 'Starts on time,' 'Room is safe,' 'The lesson occupies students,'" says Michigan State University professor Mary Kennedy, author of Inside Teaching: How Classroom Life Undermines Reform, who has studied teacher evaluation extensively. "In most instances, it's nothing more than marking 'satisfactory' or 'unsatisfactory.'"
It's easy for teachers to earn high marks under these capricious rating systems, often called "drive-bys," regardless of whether their students learn. Raymond Pecheone, co-director of the School Redesign Network at Stanford University and an expert on teacher evaluation, suggests by way of example that a teacher might get a "satisfactory" check under "using visuals" by hanging up a mobile of the planets in the Earth's solar system, even though students could walk out of the class with no knowledge of the sun's role in the solar system or other key concepts. These simplistic evaluation systems also fail to be remotely sensitive to the challenges of teaching different subjects and different grade levels, adds Pecheone.
Unsurprisingly, the results of such evaluations are often dubious. Donald Medley of the University of Virginia and Homer Coker of Georgia State University reported in a comprehensive 1987 study, "The Accuracy of Principals' Judgments of Teacher Performance," that the research up to that point found the relationship between the average principal's ratings of teacher performance and achievement by the teachers' students to be "near zero."
Principals fared better in a recent study by Brian Jacob of Harvard's Kennedy School of Government and Lars Lefgren of Brigham Young University (2005) that compared teacher ratings to student gains on standardized tests. Principals were able to identify with some accuracy their best and worst teachers — the top 10 or so percent and the bottom 10 or so percent — when asked to rate their teachers' ability to raise math and reading scores.
Principals use evaluations to help teachers improve their performance as rarely as they give unsatisfactory ratings. They frequently don't even bother to discuss the results of their evaluations with teachers.
But principals don't put even those minimal talents to use in most public school systems. A recent study of the Chicago school system by the nonprofit New Teacher Project (2007), for example, found that 87 percent of the city's 600 schools did not issue a single "unsatisfactory" teacher rating between 2003 and 2006. Among that group of schools were sixty-nine that the city declared to be failing educationally. Of all the teacher evaluations conducted during those years, only 0.3 percent produced "unsatisfactory" ratings, while 93 percent of the city's 25,000 teachers received top ratings of "excellent" or "superior."