In a bizarre statement, somewhat akin to Arne Duncan’s famous utterance about Katrina being the ‘best’ thing that ever happened to New Orleans schools, is something that NY city Chief Academic Officer, Shael Polakow-Suransky, said about the latest round of New York City school closings.
Last year 116 elementary and middle schools had gotten an ‘F’, a ‘D’, or three consecutive ‘C’s which made them candidates for closure. Ultimately 14 of those schools were closed, though it required a court order to rescue a bunch of others. This year there are 217 schools that have passed the threshold. With regard to which of those 217 will be targeted for closure, Polakow-Suransky in the Daily News said things will be looked at on a case by case basis because “This is an art. It’s not a science.” An art? I don’t know if that was the best choice of words. Ballet is an art. Oil painting is an art. Ventriloquism is even an art. But shutting down schools based on flawed metrics? That’s just cruelty.
The way the grading system goes in New York City, 30% of the schools get ‘A’s, 35% get ‘B’s, 25% get ‘C’s, 7% get ‘D’s, and 3% get ‘F’s. No matter what the scores are, even if everyone does well, 10% will get a ‘D’ or an ‘F’. I’ve found the rating system to be very unscientific and have written about it extensively. To show how unreliable the statistic is, I compared the percentile rankings for schools in 2011 against the same school’s percentile ranking in 2012.
In fairness, this does reveal a slight correlation. The correlation coefficient is about .4, which for some situations borders on ‘moderate’. But even if there were a very strong correlation, this would not necessarily mean that this is an accurate measure of school quality. it could just be biases that are favoring certain schools and will favor them, or hurt others, in the same way in two consecutive years.
The NYC DOE offers a statistic to claim the stability these scores. In the DOE press release they said “Grades remained stable across the city and for individual schools, as 86 percent of schools did not change more than one grade from 2011; 97 percent of schools were within two grades.” This is another example of ‘how to lie with statistics.’ The number 86% is only meaningful when compared to what would be ‘expected’ if the scores were completely random. Even if the scores were completely random, there would still be about 70% of schools either staying in the same category or moving just one category up or down.
Here’s why: Suppose there are 100 schools and in 2011 30 got an A, 35 got a B, 25 got a C, 7 got a D, and 3 got an F. For the 30 A schools in 2011, even in a randomized system, 60% of those 30, or 18, would get an A or a B in 2012. For the 35 who got B in 2011, 90% of those 35 would get an A, B, or C in 2012, which is 31.5. Of the 25 who got a C in 2011, 67%, or 17, would get a B, C, or D. Of the 7 who got a D in 2011, 35% of those, or about 3, would get a C, D, or F. Finally, of the 3 who got an F, 10%, or about .5 would get a D or F in 2012. When you add those numbers up, 18+31.5+17+3+.5=70.
For moving up to two grades away, a pure randomized method would still yield 92% staying the same or moving at most two categories. This is the kind of thing that can happen when there are really just three categories that dominate the five altogether. For example, everyone who gets a C will move at most two categories since there are only two above and two below it. Also 97% of people who get a B will have to stay within two categories, since only the 3% F would be beyond two moves.
Perhaps more informative is the fact that only 41% of the schools stayed in the same category from 2011 to 2012, which is better, but not by that much, than the 28% that would if the grades were assigned randomly.
The six schools that had the biggest drop from 2011 to 2012:
P.S. 041 Gun Hill Road 94% to 12.5%
P.S. 108 Assemblyman Angelo Del Toro Educational 93% to 17.5%
The 47 American Sign Language & English Lower School 91% to 20%
P.S. 183 Dr. Richard R. Green 85% to 19%
Eagle Academy for Young Men 81% to 14%
Goldie Maple Academy 80% to 10%
Today’s Daily News Reported that the first 36 schools have been named and have been in ‘discussion’ with Marc Sternberg as the process for trying to close them down gets underway. Apparently 5 out of the 7 schools that were targeted for closure before a judge prevented it were not on the list again. Also, 7 of the schools on the list are schools that had replaced other schools that had been shut down previously.
Will this be Bloomberg and Polakow-Suransky’s Requiem? We can only hope it is a symphony that remains unfinished.
Gary Rubinstein's Blog 
Your blog has provided a great articulation of all of the issues I have had with TFA throughout my time as a corps member. As a second year corps member, I appreciate your honest critique of TFA, and you so clearly put into words all of the things that I struggle with by being part of this organization. From TFA’s problematic relationship with charter schools (I am not working in a charter school but over half of my corps does) to their inadequate preparation of teachers like me in a specialized position (bilingual elementary education), I feel conflicted every day that I am part of this organization. On the one hand, I am thankful to TFA because it allowed find my passion for teaching, which I plan to make my career, but more often than not I feel frustrated by the direction in which the organization is going. Your blog reassures me that I am not just a bad teacher or a bad fit for TFA (which I am often made to feel for not strictly following the TFA line). It reminds me that these issues with TFA I have discovered for myself as a corps members are not only real but are having a serious impact on the future of education reform in our country. Thank you for lending a very public voice to these issues. Your blog has made me realize that I am not alone in feeling the way I do.
Thanks for the comment. Many alumni, particularly the ones that went on to law or medical school and really moved on from education, probably aren’t very aware of what is going on with the TFA-led ed reform. I think that if they knew what was happening, they’d be opposed to it too. I’ll keep plugging away and maybe catch the eye of some main stream media eventually.
For these people, closing neighborhood schools is neither science nor art: it’s private interest-driven policy.
Yeah, Shael is a real sensitive guy, in the same sense as Joel Klein and Michael Bloomberg. That he graduated from one of the most progressive public high schools in Michigan (Community High School in Ann Arbor) is beyond comprehension. His work in NYC belies every principle and practice of that school.
Has anyone investigated those schools where the scores have plummeted? Did the faculty all quit last year? Did the science lab burn down? Did a couple of Westinghouse winners graduate?
Speaking of art Gary, I thought you’d appreciate this:
The micro-managing, teacher-attrition promoting, and autonomy inhibiting charter school employer had a PD today in which they required us to drive to see Won’t Back Down with them and fill out a graphic organizer about the causes of the problem in the movie and the solutions. I felt kind of like I was in A Clockwork Orange. Unfortunately it’s hard to laugh at the irony of the whole situation when your principal and superintendant are sitting in the row behind you.
Aren’t you usually criticizing Gary and defending charters?
What do you note for the solutions? TFA temps with ukuleles? Get rid of unions and turn all schools over to privatizers?
And they called this professional development?
How embarrassing!
Did you get to create your own graphic organizer?
I criticize because I too am skeptical of many aspects of charter schools and school choice, but often do not think these movements are critiqued in a convincing and compelling way.
What was ironic about the movie is, the implicit solution is that teachers need to have some control in what their school organization/curriculum is like. Obviously, charters are no more likely to do this than public schools in most cases. But yes, they ask for causes to the problem in the movie, and I did have to write out “evil unions” “lazy teachers” blah blah. You see, I got them on my side, and then I tried to include an M. Night Shyamalan twist with my solutions box “teacher autonomy.”
On a side note, why the hell was this school failing? Apparently all the teachers there are great except for one. They just…decided not to care because of the existence of unions (who apparently sweep the building at 3 to make sure no one is still there)? Until later they decided to care?
Yoteach,
Are you actually abandoning the dark side? 🙂
Haha, if you look through my posts, you won’t find much that is super pro-charter schools, just unsatisfied with most arguments against them. I will say that spending tons of time in a charter school that fits every negative stereotype ascribed is informative and scary. Hopefully I’ll keep oscillating between inciting agreement and outrage.
My first instinct is to laugh, but then I just get sad that this ACTUALLY happened.
This supports those of us who support the notion that NCLB and RTTT were designed to automatically classify a ridicules portion of our schools as under performing. Then the corporate reformers come in and are picking off those schools of their choosing from the list. You do this long enough and eventually you take over a district and weaken the teachers’ union. It really is a long term, coordinated strategy that the real education reformers, like Diane, have to keep up the good fight. I personally have had enough and want to get out and become a spectator.of this mess.
Your graph looks like art. You are the Jackson Pollack of VAM!
A NYC teacher was suspended for using the DOE’s policy to grade her students http://studentslast.blogspot.com/2012/10/teacher-suspended-for-using-same.html #satire
Just out of curiosity did you use Spearman’s rank correlation or just Pearson’s correlation. Though I suppose it might not matter for a sample this large.