TNTP releases odd report about progress in D.C.

For people following ed reform in this country, Washington D.C. is the most significant place to track.  Though New Orleans is the place where the most experimentation is going on, the people doing the reforms there also control the data so it is tough to get a realistic picture of what is going on down there.  But D.C. is a bit more typical of an urban district and it is the place where Michelle Rhee was chancelor and where Kaya Henderson has replaced her, but continues the policies, helped by many other TFA reform leaders in high positions there.

As the D.C. model is getting replicated throughout the country, spurred by support for candidates from Rhee’s StudentsFirst, it is always worthwhile to see if there are any signs of success in this small petri dish for school reform.  Blogger Guy Brandenburg is the guru of D.C. and exposing their lack of progress.  From his posts I’ve learned about how D.C. still has the worst achievement gap in the country and hasn’t made any progress in closing it since the aggressive reforms started with Rhee.

TNTP is short for The New Teacher Project, though the TNTP doesn’t officially stand for that anymore.  It is just TNTP.  It is like when Kentucky Fried Chicken changed their name to just KFC.  I guess that they are now so much more than just new teachers.  They have gotten into research and policy advising, beginning with their report ‘The Widget Effect’ a few years ago which is often quoted when reformers discuss the potential of merit pay and of ending LIFO.  Then they wrote something recently called ‘The Irreplaceables’ about how schools are retaining their bad teachers and losing their good ones.  A few days ago they came out with a new report called ‘Keeping Irreplaceables in D.C. Public Schools.’

One thing that critics of D.C. reforms point out is that the teacher retention rate is just 79%, which is lower than other similar districts experience.  This new paper looks more deeply into that attrition rate and then suggests that there is finally something to cheer about in D.C..  Though their achievement gap is as wide as ever, they have managed to find one statistic where they beat their neighboring districts:  While the other districts retain about 88% of their ‘high performing’ teachers and about 85% of their ‘low performing’ teachers.  D.C.’s 79% retention rate, when you break it down by ‘high’ and ‘low’ retains 88% of their ‘high performing’ teachers but just 45% of their ‘low performing’ teachers.  The conclusion is that this is something to celebrate.  This report was touted in a column yesterday and in an editorial today in The Washington Post.

Before getting into the actual paper, note that TNTP used to be run by Rhee and now they are writing research papers that justify the decisions Rhee made and suggest that they ramp up these efforts.  If the only way to get something positive to say is to write it yourself, essentially, then the results do need to be eyed critically.

The main conclusion that D.C. is doing something good because they are retaining their ‘good’ teachers and losing their ‘bad’ is based on their controversial IMPACT evaluation model.  This is the one that was, for some teachers, based 50% on value-added.  When a teacher gets a low evaluation, he or she can get fired so of course when they fire people with low ratings, their retention rate for those people will drop.  That’s just common sense.  Whether or not those ‘bad’ teachers were really ‘bad’ is another story.  If a good teacher is rated ‘bad,’ he may quit even if he isn’t fired since he will be frustrated by the system that rated him inaccurately.  This will make the retention rate for low performers go down even more.

This statistic gets even less relevant when we consider the potential bias, which the paper admits several times, in the rating system.  According to the paper, only 11% of teachers in high poverty schools were ‘high performing’ compared to 42% of teachers in low poverty schools.  On the flip side, only 3% of teachers in low poverty schools were ‘low performing’ compared with 36% of teachers in high poverty schools.  On page 2, they speculate this could reveal a flaw in the IMPACT model on which this entire study is based:

Irreplaceables appear less likely to teach in the schools that need them most.
In DCPS, highly rated teachers are much less likely to teach in schools with
high concentrations of poverty than in other schools, and that disparity is greater than what we found in other districts.3 We believe there are two possible explanations: either the district’s best teachers are simply distributed unequally, or a flaw in the design or implementation of the IMPACT evaluation system is making it easier for teachers in low-need schools to earn high ratings. More analysis is necessary to find and address the underlying problem, and DCPS should work quickly to do both.

On page 12 they explore in more depth:

Irreplaceables are less likely to teach the students
who need them most.
Irreplaceables in DCPS appear to teach significantly fewer high-poverty students than low performers do. Top teachers in DCPS reported that only 60 percent of their students come from high-poverty backgrounds, compared to 90 percent of the students taught by low-performing teachers.15 This is a troubling finding and one that stands in stark contrast to the other districts we studied, where Irreplaceables were about as likely as low performers to report teaching high-poverty students.

There are two possible explanations for this trend. The first possibility is that top-performing teachers are inequitably distributed across DCPS schools, with more Irreplaceables working in lower-need schools. The second possibility is that a flaw in the design or implementation of IMPACT makes it easier for teachers working in low-need schools to earn top ratings.
We found evidence suggesting that inequitable distribution is a real problem regardless of the existence of design or implementation flaws in the evaluation system. When
we performed our analysis using value-added results instead of IMPACT ratings—a method that controls for student poverty levels—high-need schools still had many more low-performing teachers and many fewer Irreplaceables (Figure 9). However, more analysis is necessary to confirm this pattern and determine whether other factors are involved.

In my research I’ve found that teacher evaluation systems that rely on value-added generally are pretty random and you do see the same distribution of ‘high’ and ‘low’ teachers at high poverty and low poverty schools.  I always felt that this at least backed up something I felt to be true, even if it got to that result through randomness.  But since this system somehow favors the low poverty schools, it is very unfair.  Why would anyone want to stay in a high poverty school in D.C. and miss out on the bonus pay and promotions that are available to 42% of the teachers in the low poverty schools?  It seems that this system will rob the better teachers who might have been willing to stay in the school with the higher needs.

Another irony is that TNTP and TFA train many of the teachers who work at these high poverty schools so this statistic that there are so few high performing teachers at these schools (just 11%) is in stark contrast with their PR about how good the new teachers are.  It seems that the TNTP and TFA teachers are getting low IMPACT ratings.

Another one of their ‘findings’ is that principals are not doing enough to retain their ‘irreplaceable’ high performers.  As proof of this they write on page 8 that “Principals do not consider smart retention a top priority. In fact, more than two-thirds of DCPS principals do not consider “retaining effective teachers” one of their top five priorities.”  But this is meaningless since we don’t see how many things they were choosing from when making these priorities.  If they had a list of twenty things and one of them was “keeping students safe” for example, I could see why “retaining effective teachers” might not be in the top five.  It doesn’t mean that the principals don’t consider it important.

This paper is one that would not survive any sort of peer review process.  The main conclusion they try to make is obvious and meaningless.  Much more important is the repeated suggestion that the system by which the evaluations are made is skewed to benefit the teachers who teach at the schools with the fewest needs.

This entry was posted in Research. Bookmark the permalink.

9 Responses to TNTP releases odd report about progress in D.C.

  1. JDM says:

    No ethics. She and her various husbands are swine

  2. Steve M says:

    Hmm, my principal to-do list for today:
    1) Ensuring that our students are in school.
    2) Ensuring that my school is safe.
    3) Ensuring that our students are in class.
    17) Retain high performing teachers.

  3. meghank says:

    I have been posting the following article on TNTP’s Facebook posts about this report:

    They deleted my comments and blocked me from further commenting, so I asked others to post that same article.

    Gary, can you point the way for me to find out the salaries of charter school CEOs? Is that information publicly available? Specifically, I am looking for the salary of James Wilcox, CEO of Aspire Public Schools.

  4. Pingback: That Deeply Flawed Report About Rhee’s IMPACT Program: UPDATE « Diane Ravitch's blog

  5. Pingback: Problems with D.C. teacher retention report

  6. skepticnotcynic says:

    Look at the arrogance of this leadership team.

    A team of 17 leaders with a combined experience of 150 years in education, which is actually very little. They cleverly cover up their per person average of 8.8 years of experience in education and organizational management, which is probably no more than 2 years on average of actual classroom teaching per person, yet they are experts 🙂

    Any real educator can see through this sham. These are individuals who were too proud to teach, probably weren’t very good at it, and now feel that the public should listen to their faux-expertise and that they deserve a voice at the ed-policy table.

    TNTP is a joke and should be put out of business. What a waste of tax-payer money.

  7. Pingback: NCTQ Letter Grades and the Reformer Agenda–Part VIII « deutsch29

  8. Pingback: John White’s Great Out-of-State Contract Giveaway | deutsch29

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s