Teacher Evaluations: Keeping Score

On Friday, a firestorm was created when the NYC Department of Education released the Teacher Data Reports for 18,000 teachers.

These DOE rankings are based on a formula which relies heavily on student test data to make a determination of a teacher’s effectiveness. 

A New York Times Schoolbook article summed it up in this way: “In simple terms, value-added models use mathematical formulas to look at the past and forecast the future. A computer predicts how a group of students will do in next year’s tests using their scores from the previous year and accounting for several factors, like race, gender and income level.

If the students surpass the expectations, their teacher is ranked at the top of the scale — 'above average' or 'high' under different models used in New York City. If they fall short, the teacher receives a rating of 'below average' or 'low.'"

While I believe that there is definitely something to be said for transparency and trying to provide parents with information surrounding the effectiveness of schools and teachers, data like this is only useful if it is accurate and timely.

By almost all accounts, the teacher data reports fail to meet these basic criteria. 

The wide margin of error (anywhere from 25-50 percentage points) resulting from the small number of tests included in the sample, the errors resulting from the wrong teachers being matched to the wrong classes, or to students whom they’ve never taught, added to the fact that much of this data is almost 5 years old, make the publication of the data reports irresponsible.

It is irresponsible not only because it provides parents with at best, useless information and at worst, erroneous information, but more importantly because it frames the very critical, complex and multi-layered discussion of teacher effectiveness through the most narrow of lenses: summative test results in math and English.

By making this data public, the DOE is essentially saying that despite their caveats, this is a reliable method of determining a teacher’s value. No one in the general public will read all of the fine print and disclaimers and you can be sure that none of the newspapers and websites will be concerned with the flaws when they begin creating the inevitable “Best and Worst teachers in NYC” lists, despite the fact that these reports only rank English and math teachers in grades 4-8.

The whole brouhaha over these data reports is evidence of a disturbing trend that is reflective of what we value as a society and of the direction in which we are headed.

We are living in a time of simplistic and short-sighted thinking. We will opt for the simple and most expedient choice instead of the best choice more times than not. We can see this most clearly in our decisions around public education in urban communities of color:

• A school is not giving us good test scores? Close it down and fire all of the teachers and administrators.
• Students need more time to prepare for tests? Eliminate drama, band, choir, athletics, recess, chess clubs, French clubs, civic groups, science clubs, debate teams, student government, school newspaper, Junior Achievement, girl scouts, boy scouts, etc. and use the time and resources to prep students for math and English tests. 
• Students arrive to schools with issues far beyond the capacity for schools to address alone within the time frame of a testing cycle? (Homelessness, lack of parental involvement, inadequate access to quality health care, mental health issues, poverty, and limited fluency in English, etc.) Just pretend that none of these things matter, and label anyone who says otherwise as having “low expectations."

Until we commit to developing a holistic and transparent system of assessing teachers that takes into account all of the variables that are attached to children, schools and communities, and ensure that all schools and children are provided with the necessary supports and resources needed for success (while making appropriate timeline adjustments to account for where the child is starting from academically and socially), we cannot in good conscience resort to publishing meaningless data that will only mislead.