Two years ago I first heard about a new Charter School in New York City called The Equity Project, founded by a TFA alum named Zeke Vanderhoek. There was an article in The New York Times about how they were going to pay their teachers $125,000 in return for more work and accountability. Teachers could also earn bonuses of up to $25,000. They were also featured on 60 minutes. I have to admit that I considered applying. That’s a lot of money. Even veteran teachers in New York City with 30 years experience make just about $100,000. With 8 years in New York City, I’m up to about $75,000.
According to their website they have based this paradigm on ‘research’ including the famous 1996 Sanders and Rivers value-added study which ‘demonstrates’ that having three great teachers in a row vs. three not-so-great ones can be a difference of 50 percentage points in student achievement. Knowing what I do about education, after 20 years of teaching and analyzing it , I was skeptical that their 8 teacher million dollar ‘dream team’ would get the results they expected.
Here is their ad on page 5 from the most recent TFA alumni magazine.
When I heard about this experiment, I was very interested to see how it turned out. Despite all the ‘research’ that proves that having three great teachers in a row can dramatically increase test scores, nobody has actually conducted the very simple experiment of giving a group of kids three ‘highly effective’ teachers in a row. There have been studies where they take three thousand kids and after three years analyze the results of the thirty of those kids who happened to get three effective teachers. There have been studies where a group of kids have had one effective teacher and then the gains were multiplied by three. But, as far as I know, this is the first time that someone actually recruited a ‘dream team’ among thousands of applicants and had the same kids learn from these teachers. If the kids performed as well as those studies predicted, it would definitely support the thesis and also give some leverage to corporate reformers who believe the problem with education is that there are not enough great teachers and too many awful ones.
This progress report was released a few months ago and I haven’t heard much media coverage. So I downloaded the 2010-2011 progress report and was not really that surprised to learn that this school is performing slightly below average, even compared to its ‘peer group’ with similar demographics.
The New York City progress reports are based on three categories: 15% based on ‘environment’ which comes from teacher and parent surveys. The school got a solid A in that category — no surprise that the teachers gave the school high ratings. Then, 25% is based on student ‘performance’ — which is the achievement level of the students as compared mostly to their 40 school ‘peer group’ of schools with similar demographics. On this category, they got an unimpressive C. Student ‘progress’ — how much their students ‘progressed’ when compared to students around the city who had similar starting scores counts for 60% of the report. On that one they got a B, which gave them a B overall, and that put them at the 69% percentile of all middle schools in the city. Keep in mind that all the scores are relative to schools that have similar demographics, so they can’t make the excuse that they had low starting scores.
Here is how they got the C in performance. Note that this is their performance relative to schools with similar demographics. Their forty school peer group only had 49% proficient in English which was way better than TEP’s 31%.
You don’t have to understand a lot about statistics to know that those gray bars would fill the entire left side of these rectangles if the school had average performance for their peer group. They have only 31% proficient in English and 49% proficient in math. And this is after having ‘highly effective’ teachers for two years in a row, so far.
Here is how they got their B in ‘progress
It turns out, I learned, that 60 minutes did follow up last year. At that time the scores at TEP were pretty low. According to the CBS website:
“Some people watching this might be thinking, ‘Hey, they’re paying teachers $125,000 a year. They’ve attracted the best and the brightest. These results don’t really add up,'” Couric pointed out.
“We don’t have a magic wand. We’re not gonna take kids who are scoring below grade level and bring them up in a year,” Vanderhoek said.
But he doesn’t mention that the progress reports are supposed to account for that. Their performance is not measured against an absolute standard but against their ‘peers’ which include several KIPP schools. Their progress is also something that is not supposed to punish low starting scores. According to the CBS site, two of the eight ‘dream teamers’ were fired at the end of the first year!
Anyone who has read my blog in the past year knows that I don’t think very much about the New York City progress report calculations. I’m sure that this school is quite a good school, despite their low test scores. I’m sure the teachers are dynamic and work very very hard. The lesson to be learned here is that this ‘dream team’ is not doing much better, and actually worse, on average, than the ‘average’ New York City teachers because, on average, teachers are good.
I wonder how corporate reformers will react to this story. They are stuck, really. Either they have to admit that the ‘three great teachers’ thing has been proved wrong, once and for all, and is not something to base policy on. Or New York City corporate reformers will have to admit that their rating system is flawed, even though many ‘failing’ schools have been shut down on the confidence of the accuracy of those reports.
And as far as TEP goes, they are going to have to make a big decision very soon. The only way they will get their test scores up to a level that will satisfy their funders is if they do what most of the ‘high performing’ charters do — kick out the kids that are lowering their test scores. It is unfortunate that if they choose to be honest and do the best they can, knowing that what they are accomplishing is tough to measure, they are likely to lose their funding in 2013 when their charter expires.