Dallas Value Added Study — More Analysis

A few weeks ago, I wrote about how I did my own analysis of the 1997 study which is always quoted by Rhee about how 3 effective teachers in a row vs. 3 ineffective teachers in a row is life changing.  Now, as someone who considers himself an effective teacher, and someone who has been taught by effective teachers and also by ineffective teachers, I’m very aware that there is a difference.  The question is whether or not this difference really shows up in standardized test scores accurately enough so that districts can use them reliably as part of evaluations which can lead to teachers getting fired over them.

Here’s an example of one of the stunning graphs that still, all these years later, shows up in various forms in power point presentations by the corporate reformers.

This shows three groups of students with about 25 students in each group who all started with about a 56% passing rate.  Then the graph on the right shows how three of those groups fared three years later.  The 555 group had three great teachers in a row, the 324 had average teachers, and the 112 had ineffective teachers.  Pretty dramatic right?

Well, one thing to notice is that the 555 group, while way higher than the 112 group had only a marginal gain of about 4 percentage points.  Now though the report had the raw data for all 125 combinations of teachers from 111 to 555, they only provided the percentiles for the three groups in this graph and the three groups in another graph.  This makes it hard for others to crunch their own numbers.  But what I managed to do is take the six percentages that I knew and search through the other 122 groups to see if any others had this 56% starting point and an ending point three years later equal to any of the six percentiles I knew about for that test.  I managed to find 11 such groups.

So compare the dramatic 3 bar graph above to my less dramatic 14 bar graph based on this analysis

So, as you can see the top group did get the best, a 4 point gain.  Two other groups 345 and 235 got 3 point gains.  The rest, aside from the 112 group got 6 point losses.

Another thing to notice is that nearly all the groups had some kind of a loss, meaning that Dallas would be wise to help improve their average teachers, rather than worry about firing their poor teachers.

If having three bad teachers in a row is that bad (but getting some good ones in the mix seems to make things about average) then why doesn’t Dallas use this information as Sanders suggested to make sure that students don’t get three poor teachers in a row.  Since it seems that this combination can be avoided with some planning, this is a way they can use this data to improve student outcomes.  This study was 14 years ago and Dallas has the 7th highest dropout rate in the country, and the six cities below it are all under a million students.

This entry was posted in Research. Bookmark the permalink.