The Tennessee ASD: Booted or Re-Booted?

Since 2011 I have been following the biggest, and most predictable, disaster of the education reform movement — the Tennessee Achievement School District (ASD).  It was formed in a perfect storm of reform theory.  First, Tennessee won Race To The Top money.  Then they hired a TFA-alum and the ex-husband of Michelle Rhee, Kevin Huffman to be their state commissioner.  Then he hired TFA-alum and charter school founder Chris Barbic to design and run the ASD.  The initial promise of the ASD was that they would take schools in the bottom 5% and convert them into charter schools in order to ‘catapult’ them into the top 25% in five years.  They started with 6 schools in 2012 and grew to over 30 schools within a few years.

They completely failed at this mission.  Chris Barbic resigned, Kevin Huffman resigned, Barbic’s replacement resigned, Barbic’s replacement’s replacement resigned.  Of the 30 schools they nearly all stayed in the bottom 5% except a few that catapulted into the bottom 10%.

The new education commissioner of Tennessee is also a TFA alum with ideas similar to Huffman.  She promised, however, to get a handle on the ASD and what to do about its failure.  After a listening tour around the state she made, it seemed at first, a decision that was long overdue.

Chalkbeat TN recently had a post with the enticing title ‘All 30 schools in Tennessee’s turnaround district would exit by 2022 in a massive restructuring proposal.’  It would seem like this is good news.  The ASD was such a costly failure, costing about $100 million over the years I think, the only thing to do was to put it out of its misery and dissolve it completely.

But I’ve been studying reformers enough over the years not to get too excited about this.  The headline would make the most optimistic readers think that the 30 schools going back to the district would again become public schools.  The charter schools supposedly traded flexibility for accountability so their failure to deliver on their promises should result in them being sent packing.

But according to the article, it is not clear yet if being returned to the district means that they will become public schools again.  Also they say that there still will be an ASD after this.  Now there can’t be a school district with zero schools, so what’s going on?

I think, and I hope I’m wrong about this, that with the failure of the ASD there was no way that they could justify adding more schools to it.  But by ‘returning’ the 30 schools back to their districts, and probably keeping them as charters, there will now be room to add more schools in the bottom 5% to the re-booted ASD.  If this is what happens, the ASD won’t be disappearing or even shrinking, it will be expanding.  There will be the 30 schools that are still charters, but just operating as part of the district they have been returned to.  And then there will be another 20 schools, maybe, that are in the new ASD.  (They actually call it the ASD 2.0 in the state slide show)

Basically, this is like when a businessman declares bankruptcy yet finds a way to get out of debt that way and actually profits off of it.  Without those original 30 schools that are making their stats look so bad over the years, they will start fresh with other schools.  Then they can spend another 8 years with those schools and say “You can’t expect us to fix these schools overnight, we need more time.”  But this is just a shell game.

That’s what I think is going on.  Maybe I’m wrong, but don’t be surprised if I’m right.

Posted in Uncategorized | 6 Comments

More Reform Funded Research: KIPP Graduates Persist In College At The Same Rate As Their Mothers

Education Reform propaganda at The74 would try to make you believe that while low income students generally graduate from college at a rate of about 9%, charter school graduates complete college at a rate of 3 to 5 times that.

The main flaw in any comparison between the college graduation rates of charter school graduates to low-income students, in general, is that the charter school students do not represent a random sampling of the general population of low-income students.

In The Alumni, Richard Whitmire says that charter schools that have 5 times the expected college completion rate are ones that only counted their students who persisted until 12th grade in their charter schools.  Since for some charter schools, this only represents about 25% of the students who started in that charter school, this even more of a biased sample.  But, Whitmire explains, the one network that has the most valid way of doing a fair comparison is the famed KIPP network.  Since KIPP counts, in their data, any students who enrolled in KIPP, even if they left soon after starting.  And he says that KIPP students, including ones who didn’t persist at KIPP, graduate college 3 times the expected rate.

Reform supporting billionaire John Arnold commissioned Mathematica, a data analysis company, to study the college enrollment and college persistence of KIPP students.  Instead of comparing KIPP students to the general population, they compared KIPP students to students who had applied to the KIPP lottery but did not get into KIPP through the lottery.  This is a much more valid way of measuring the impact of KIPP.  The big takeaway, as I wrote about in my previous post, was that students who applied to KIPP, whether or not they got into KIPP, had a college persistence rate of about 3 times the general low-income population and that students who applied but didn’t get into KIPP had about the same college persistence as students who applied and did get into KIPP.  So students to apply to the KIPP lottery are the ones who, on average, were much more likely to persist in college — something that Whitmire never mentions in The Alumni.

But this Mathematica report includes some other relevant data that I didn’t pick up on when I wrote the last post.  Fortunately there was a discussion among some readers who commented on the last post which pointed this out.

In 2018 the National Center For Education Statistics published a report called ‘First-Generation Students College Access, Persistence, and Postbachelor’s Outcomes.’  In it they say that about 70% of students who have a parent who completed college also complete college compared to about 35% of students who do not have a parent who completed college.  This confirms what most people would expect for so many reasons and this is why we celebrate when students are the first in their family to graduate college.  It means that the descendants of those students will also be more likely to go to college.

In the Mathematica study, they collected statistics about the pool of students who applied to the KIPP lottery.  Among those statistics was the level of education attained by the mother of the KIPP lottery applicant.  Here’s what it says (page 6 of the report):

Screen Shot 2020-01-12 at 11.11.20 PM

Notice that last line.  It says that of the students entering the lottery about 27% of them had mothers who finished college.  This makes the fact that about 30% of the students in the study (which includes students who got into KIPP and also students who did not get into KIPP) have persisted in college through four semesters even less surprising.

Someone like Richard Whitmire suggests in his analysis that has been quoted in USA Today and The Wall Street Journal, that had these students not gone to KIPP they would have only persisted in college at around 9%.  But it can be clearly seen now that even without going to KIPP, these students should be expected to have about a 30% college persistence rate.

Data, like what percent of charter school applicants have parents who are college graduates, are so important but nearly impossible to actually learn.  Thanks to the John Arnold Foundation for commissioning this study and shedding light on a truth that we already knew but didn’t yet have hard data to support.

Posted in Uncategorized | 3 Comments

Reform Funded Research: Winning KIPP Lottery Does Not Significantly Increase Chance Of Persisting In College

The way reformers misuse data follows a very simple and predictable plan:  First they get some skewed data, then pick a ‘researcher’ to interpret the skewed data.  The ‘researcher’ then writes a report which gets touted in The74, EduPost, and eventually even makes it into more mainstream publications like USA Today and The Wall Street Journal.  Since the report is filled with nonsense and half-truths, within a few weeks the truth comes out and the report is discredited, but not before the damage was done and the spin has made it into folklore.  When this happens, the reformers will then ‘move the goalposts’ and get some more skewed data and start the process over again.

An example of this is the July 2017 report by Richard Whitmire called ‘The Alumni‘.  Whitmire has written books about both KIPP and about Michelle Rhee so I think you get the idea of what his point of view is.  In this poorly researched project he concludes that “Data Show Charter School Students Graduating From College at Three to Five Times National Average“.  The national average he is comparing to is the 9% of low-income students that graduate college, according to the Pell institute.

This was probably the easiest report I ever debunked.  The biggest flaw was that for most of the charter schools, they were only counting the percent of graduating seniors who persisted in college and then comparing that percent to the overall percent of all low-income students — an apples to oranges comparison.  Whitmire acknowledges this in another post about the methodology in which he says that only KIPP counts students who leave the school before they graduate and that their numbers are much lower, but still at 38% which is at least triple the expected graduation rate for low income students.

A second flaw, and this one is very difficult to compensate for, is that charter school students are not a random sampling of all students since many families choose not to apply to them.  So you get a biased sampling even if you do count all the students who get into the charter school and not just the ones who make it to graduate from the charter school.  And even though I and others have discredited his report, it is something that still gets quoted in the main stream media.

Just recently, however, I learned of a report generated by Mathematica and funded by the John Arnold Foundation.  I think that Mathematica is a very reputable company and even though reformers often hire them to produce reports, sometimes those reports reach conclusions that reformers were not expecting.

In this case, the report called “Long-Term Impacts of KIPP Middle Schools on College Enrollment and Early College Persistence” , reached a result that completely contradicts Whitmire’s claim that “Charter School Students Graduating From College at Three to Five Times National Average”.

What Mathematica did was follow 1000 students who applied to KIPP schools that required the use of a lottery since they had more applicants that open slots.  500 of them were offered spots and KIPP and 500 of them were not.  Comparing the fates of lottery winners to lottery losers is more accurate than comparing KIPP students to students who never even applied to a lottery since the KIPP students are not a random sampling.  The study compared the college persistence rate (what percent of each group completed four semesters of college) among other comparisons.

Here’s the relevant summary of what they found:

Screen Shot 2020-01-04 at 5.07.25 PM

This needs a lot of unpacking:  So 30.4% of the group who were offered a spot at KIPP were still in college compared to 25.6% of the group who were not offered a spot at KIPP.  Remember that Whitmire claimed a 3x to 5x comparison for charter chains.  Well, even if you just take these two numbers without any other context they provide, this is 30.4 is 1.1875 times as big as 25.6.

The big headline here should be that from this study in which all of the students involved are ones who entered the KIPP lottery (and only half of them actually attended a KIPP), about 28% of them completed four semesters of college.  Where is Whitmire’s post about how this shows that simply applying to KIPP (whether you get in or not) increases your chances of persisting in college by 3 times?  Really this shows what everyone already knows, but which anyone reporting about ‘The Alumni’ report ignores, that students who apply to charter lotteries are not a random sampling and therefore any comparison between college enrollment and college persistence for charter schools vs. all schools is going to be a flawed comparison.

Now since 30.4 is greater than 25.6, it might seem at first that winning a spot in a KIPP lottery causes a slightly higher chance of completing four semesters of college.  But the report also says that this difference “is not significantly different from zero (p-value 0.135).”  What does that mean?

Most people don’t understand what the difference between “large” and “significantly different” when it comes to statistics.  Here’s an analogy that demonstrates this a little.  Suppose there were 100 students altogether and 30 of them completed four semesters of college and 70 did not.  If you took those 100 people and split them into two groups by some sorting rule, like one group is people who have an even social security number and the other group is the people who have an odd social security number and, it turns out, there are 50 people in each group.  You would expect, since there is nothing about having an even or odd social security number that would cause one group to do better than the other group, that each of these groups would have about 15 students who completed four semesters of college, since that would be 30% for each group.  But would you be shocked if it turned out that the group with even social security numbers had 16 students (32%) who completed four semesters of college and the other group had 14 students (28%) who did this?  Would you feel confident in saying that having an even social security number somehow causes people to be more successful in college?  Of course not.  It was still very close to the expected 15 / 15 split.  Well this is what is meant by “not significantly different from zero (p-value 0.135),” in a nutshell.  The larger the p-value, the less significant the difference is with a ‘good’ p-value being very small, less than 0.05.  So this report cannot say with appropriate confidence that winning the KIPP lottery is associated with any increase in college persistence rate.  This is surely not what The Arnold Foundation would have hoped would come from this and I seriously doubt that The74 or EduPost will write about this.

There were three other comparisons that they did for this report and those three look better for KIPP so I want to explain about those also.  Some of the students who won the lottery to get into KIPP did not, for whatever reason, go to KIPP.  Maybe they didn’t want to go, maybe they were discouraged by KIPP not to accept their offer, who knows?  So in addition to the comparison I just mentioned, there is a second comparison where they compared students in the study who did not go to KIPP (lottery losers and lottery winners who did not go for whatever reason) to students in the study who got an offer and also did end up going to KIPP.  For that, the difference in the percent of students completing four semesters of college was 9% (24% vs 33%).  This was still not considered a ‘significant’ difference by the authors of the report and for me, I would consider this statistic biased anyway.  When you remove the lottery winners who did not go to KIPP for whatever reason, you would need to also remove the lottery losers who would not have gone to KIPP had they won the lottery.  Since it is not possible to know who from the lottery losers would have not gone to KIPP, this is a comparison that I consider to be flawed.

Screen Shot 2020-01-04 at 5.09.38 PM

There were two other comparisons where instead of college persistence, they compared just college enrollment.  For this one there was a 7% difference which they considered ‘significant’ for lottery winners vs. lottery losers and a 13% difference for the other way where only the students who got into KIPP and enrolled in KIPP were part of the treatment group.

Screen Shot 2020-01-05 at 9.34.16 AM

Maybe there is a benefit to going to college for a few semesters and then quitting before completing the fourth semester.  I think that since charter schools talk so much about “To and through college,” getting into college and leaving so soon seems like something that they should not be celebrating.

The big takeaway, though, from this recent report is that it is an excellent counter to the 3x to 5x propaganda claim spread by the reform blog sites.  This suggests that the number is “not significantly different from zero” when a more valid comparison is done.

Posted in Uncategorized | 8 Comments

Success Academy Class Of 2020 Sheds 239 Scholars Along The Way

The New York Post recently ran an editorial about the SAT scores of the Success Academy senior class of 2020.  Of all the different numbers they referenced, one that I took note of was 114 — the apparent number of students in the senior class.

The class of 2020 is the third graduating class of Success Academy.  The class of 2018 had 17 seniors out of a cohort of 73 first graders in 2006-2007.  The class of 2019 had 26 seniors out of a cohort of 83 kindergartners in 2006-2007.  Some of the class of 2019 were students who had been held back from the class of 2018 — probably in a comparable number to the number of 2019 students who will graduate this year.  So the 26 out of 83, or 31% persistence rate probably accounts for students who take an extra year to graduate.

For the class of 2020, things get a bit more complicated since in 2008 Success Academy did its first expansion and grew from one school, now called Harlem 1, into four schools now including Harlem 2, Harlem 3, and Harlem 4.  Some of the past records are incomplete for these schools, but when the 2020 cohort was in 2nd grade in 2009-2010, I find that there was a combined 353 students in the cohort.  By 6th grade, they were down to 263 students and by 9th grade it was 191.  In 10th grade they were 161 students and in 11th grade, 146.  And now, according to the New York Post article based on a Success Academy press release, they have 114 seniors.  So only 32% of the students who were there in second grade made it through their program.  And even more startling is that of the 191 9th graders that had been at Success Academy for 10 years, only 59% of them are on track to graduate three years later.

SA2020

[all data gotten from https://data.nysed.gov/ ]

This 32% persistence rate doesn’t even include the students who ‘backfilled’ some of the empty spots for students who have left over the years.  Without access to more granular data, this isn’t something I can study right now.  From Robert Pondiscio’s book about Success Academy we learn that the backfill process is somewhat corrupt.  Families that get off the wait list to backfill vacated spots are sometimes told that if they come to Success Academy their children will have to redo the grade they had just passed in their other school.  Surely many of these families choose to forfeit their place off the waiting list and, in that way, Success Academy makes sure that the backfill students are generally the higher performing students which serve to inflate the school’s test scores.

According to Success Academy, the demographic data for their students are:  74% receive free or reduced-price lunch, 16% have disabilities, and 8% are English language learners.  But the most recent data (From 2017-2018) about the 2020 cohort from the New York State public site is that 66% are economically disadvantaged, 11% are students with disabilities, and 0% English language learners.

Success Academy and The New York Post love to claim that Mayor de Blasio is out to get Success Academy.  I really don’t think so because de Blasio has access to the type of data that could so easily expose the various ways that Success Academy tips the scale in their favor.  Just using publicly available data, I’ve been able to uncover so much about their massive attrition rate.  Imagine how much can be learned from the data that the New York City DOE can access.

Posted in Uncategorized | 13 Comments

Does Louisiana Really Lead The Nation In 8th Grade Math Gains?

The latest National Assessment of Educational Progress (NAEP) scores for the 2019 tests were released on October 30th.  Unlike state tests for which the cutoff scores can be manipulated for political purposes, the NAEP does seem to be somewhat unbiased.  So the NAEP, sometimes called ‘The Nation’s Report Card’ does offer an interesting amount of data that I believe is worthy of analysis.

Often the NAEP results are, intentionally or unintentionally, interpreted to see if it is possible to find some kind of correlation between the education policies a state has enacted and the corresponding NAEP results.  In Obama’s 2014 State Of The Union address, he mentioned that D.C. and Tennessee were improving — as evidenced surely by their NAEP gains from 2011 to 2013 — to show that his Race To The Top recommendations, which were followed closely by those two regions, were working.

So when the 2019 results came out the other day, things looked bad for the reformers.  From 2017 to 2019, the average scale score for 4th grade reading was down 1 point, 8th grade reading was down 3 points, 4th grade math was up 1 point, and 8th grade math was down 1 point.  Though it is not clear to the public whether or not one ‘point’ is a lot or a little, everyone can agree that it is better if the scores go up rather than down.

Only one region had an increased score in 4th grade reading while 17 states decreased.  Only one region increased in 8th grade reading while 31 states decreased.  For 4th grade math, 9 states increased while 3 decreased, and for 8th grade math 3 states increased while 6 states decreased.

Former Secretary of Education, Arne Duncan, wrote an article for The Washington Post called What we can learn from the state of our nation’s education.  He begins by writing:

The latest results from the National Assessment of Educational Progress (NAEP) are prompting some soul-searching about the limited gains over the past decade, but there are outliers worth saluting. More important, we should be analyzing what successful states and school districts are doing differently so that others can learn from them.

He then tells us about some of the ‘bright spots’ which, of course, happen in places like D.C., Tennessee, Louisiana, and Denver — all places that have followed the Race To The Top playbook with charter schools and using value-added to rate teachers.  About Louisiana he says “Louisiana posted nation-leading gains in eighth-grade math” and later credits Chief of Change John White, a former TFAer who has been Louisiana’s State Superintendent of Education since 2012.  He concludes by warning us not to look at the overall lack of improvement as an excuse to rethink the reform agenda he promoted.

The one thing the United States cannot do is use these results as an excuse to go backward to the days when standards and expectations were low. We cannot return to a time when achievement gaps around race and poverty were hidden. We cannot pretend that talent strategies will happen on their own without intentional efforts to recruit, support, retain and hold accountable educators.

I’ve been following Louisiana’s John White for some time and always like to catch him cherry picking data to make it seem like he has helped Louisiana to improve in education.  Debunking this recent falsehood about Louisiana leading the nation in 8th grade NAEP growth was one of the easiest ones to uncover.

Imagine you have a friend who has been on a diet for 6 months.  You ask him how his diet is going and he proudly asserts that he lost 5 pounds in the past month.  But he looks a bit nervous when he is telling you this so you ask the important follow up question:  How much have how lost over the past six months since you started the diet.  He confesses that he hasn’t lost any weight in the six month period and actually gained a pound in that time.  It’s a good thing you knew what sort of follow up question to ask.

So while, yes, Louisiana’s 8th grade math NAEP in 2017 was 267 and their 8th grade math NAEP in 2019 was 272 which was a 5 point gain in that two year period and while that was the highest gain over that two year period for any state, if you go back instead to their scores from 2007, way before their reform effort happened, you will find that in the 12 year period from 2007 to 2019, Louisiana did not lead the nation in 8th grade NAEP gains.  In fact, Louisiana went DOWN from a scale score of 272.39 in 2007 to a scale score of 271.64 in 2019 on that test.  Compared to the rest of the country in that 12 year period.  This means that in that 12 year period, they are 33rd in ‘growth’ (is it even fair to call negative growth ‘growth’?).  The issue was that from 2007 to 2015, Louisiana ranked second to last on ‘growth’ in 8th grade math.  Failing to mention that relevant detail when bragging about your growth from 2017 to 2019 is very sneaky.

Screen Shot 2019-11-03 at 4.55.41 PMScreen Shot 2019-11-03 at 4.56.24 PM

 

Screen Shot 2019-11-03 at 5.15.36 PM

This is just one small concrete example of how reformers will cherry pick data to claim that there are bright spots in this NAEP data that show that we need to continue following the lead of people like Arne Duncan.

I think 2007 is a good benchmark year, in general, to look at how much the country ‘grew’ on NAEP.  For 4th grade reading, the scale score decreased by one point between 2007 and 2019.  For 8th grade reading, the scale score stayed the same from 2007 to 2019.  For 4th grade math, the scale score increased by one point between 2007 and 2019, and for 8th grade math, the scale score increased by one point between 2007 and 2019.

There is a lot more to be said about the NAEP results and how to interpret them.  For example, it is hard to compare growth between two states that had different starting points.  Like when someone is 100 pounds overweight it might be easier to lose 10 pounds while if someone is 10 pounds overweight it might be more difficult.  I’ve seen analysis that higher growth (as measured in points) correlates with lower starting scores, which helps explain the so-called ‘Mississippi Miracle’ reformers are now talking about.

Three of the places that Duncan touts for their ‘gains’: D.C., Louisiana, and Mississippi are three of the lowest scoring NAEP regions.  Why should we be looking to them for things to emulate?

Reformers will always look to cherrypick data that they can twist to make it look like they should continue to have the power to influence education policy.  Usually they have to stretch so far to make their claims that they are fairly easy to uncover.

Posted in Uncategorized | 2 Comments

Nevada ‘Abolishes’ Its Achievement School District

The ‘portfolio’ model of education reform was pioneered in New Orleans after Hurricane Katrina and the creation of the Recovery School District (RSD) in 2005.  Then Tennessee, fueled by Race To The Top money, brought in TFA alum and Michelle Rhee ex-husband Kevin Huffman who hired TFA alum and YES prep founder Chris Barbic to create a similar thing called the Achievement School District (ASD) in 2012.

Even though the Tennessee ASD has been a complete failure by every metric possible (unless you count Huffman and Barbic getting high paying consultant jobs after resigning in shame as one of your metrics), other states around the country have followed their ‘lead’ and proposed or created ASDs of their own.  There was one created in Michigan, North Carolina, and Nevada and others proposed in Georgia and Pennsylvania.

Not surprisingly, the ASDs have all been utter failures.  And since it is hard to keep such failure a secret, states are starting to wake up to this fraud.  So it is somewhat good news that another ASD has bitten the dust, this time in Nevada.

On June 3, 2019 Nevada passed SB321 abolishing their ASD.  The official description is that it is “An act relating to education; abolishing the achievement school district; requiring an existing achievement charter school to convert to a charter school under the sponsorship of the State Public Charter School Authority or cease operations; and providing other matters properly relating thereto.”  So even though their ASD is officially ‘abolished,’ it is not like those schools are converted back to public schools.  Instead they are now under the State Public Charter School Authority which, I’d guess, has even less accountability than there was under the ASD.

But my sense is the fewer ASDs the better.  The fewer there are, the lower the chance that some of them will lie about their success and that those lies convince other states to replicate the model.

As far as the Tennessee ASD, it has been the biggest trainwreck in education reform history.  But now even the Tennessee media which has been really slow to realize the scam that was going on  for so many years is finally writing about this.  Here is a recent article from The Tennessean exposing the failures of the ASD.

The new commissioner of education of Tennessee is a TFA alum, much like Kevin Huffman was.  She has recently announced a seven city ‘listening tour’ where they will get community input on what should be done next.  When the Tennessee ASD started, the plan was to ramp it up to about 100 schools.  They started with six schools and grew to over 30 schools in the first four years.  But they have stalled at that number and are not taking on any new schools this year.

On the latest available Tennessee state data, the ASD had not just the lowest test scores , an 11.6% success rate, but also the lowest growth score of a 1 out of 5 on their Tennessee growth measure invented in Tennessee by William Sanders on which most other growth measures are based.

Screen Shot 2019-10-05 at 11.03.07 PMScreen Shot 2019-10-05 at 11.04.22 PM

Things have changed a lot in the past four years.  It was just a little over four years ago that Michael Petrilli moderated a panel called ‘Turnaround Districts:  Lessons from Louisiana, Tennessee, and Michigan.”  Since that time the Louisiana RSD was mixed with the other New Orleans schools, most likely so they can make it more difficult for the public to analyze the data from the original RSD schools.  The Michigan ‘EAA’ was dissolved a few years ago, and the Tennessee ASD is fighting for its life.  For sure this panel discussion and the fanfare around these supposedly visionary leaders has not aged well.

 

 

Posted in Uncategorized | 6 Comments

How The Other 1/300th Learns

Robert Pondiscio’s new book “How The Other Half Learns” (Avery September 2019) answers the age old question:  Can a bunch of twenty-something teachers who know nothing about education, nothing about child development, and nothing about what it is like to be a parent, get a non-random sampling of students of color to pass standardized tests?

In addition to answering that question in the affirmative, Pondiscio skillfully paints the most thorough picture of what exactly goes on behind the closed doors of Success Academy charter schools.  We get to meet the teachers, the administrators, the ‘scholars’, and the parents. I found the book very illuminating and recommend it highly.

There is a pretty short list of writers who Eva Moskowitz might trust enough to allow them to observe in a school for a year.  I’m glad that it was Pondiscio. He is one of the few reformers I have any respect for. We disagree on some things and we agree on some other things.  I even once went out to lunch with him and had a very pleasant time.

Before the book came out he wrote a blog post on The74 called “I Just Wrote a Book About Success Academy Charter Schools. It Does Not Support Your Preferred Narrative. I Hope You Hate It.”  Not to dash his hopes, but I, for one, did not hate it. Overall I thought it was great which means that only reformers who are willing to be honest with themselves will actually hate this book.

One premise of the book is that the fundamental secret to Success Academy’s amazing standardized test scores, mentioned throughout the work is the filtering of the right families.  On page 266 he writes “The common criticism leveled at Moskowitz and her schools is that they cherry pick students, attracting bright children and shedding the poorly behaved and hardest to teach  This misses the mark entirely. Success Academy is cherry-picking parents.” Parents must go through a series of tests and hoops to jump through for their children to get into and to stay in a Success Academy school.  First there is, of course, the lottery. But winning the lottery is just the first step. Described in great — and frightening — detail in chapter 20 “The Lottery”, lottery winners have to attend a mandatory informational session where they are told how much work it is to be a parent of a child at the school — how lateness is not tolerated and there is a 7:30 AM start time.  How there is no transportation provided. How every Wednesday is a half day and there is no after school program. How absences require a doctor’s note. Many prospective lottery winners give up after that meeting. Then there are several other steps like extensive paperwork and uniform fittings and a dress rehearsal. Even Pondiscio is shocked to watch how a student who is deep on the waitlist eventually get admitted to the school.  But having families who are this willing and able to comply with the demands made by Success Academy leads, predictably, to high standardized test scores. He doesn’t say this so bluntly, but let’s face it — this is a kind of cheating.

But if you look at the back of the book, you see that it was well reviewed by various reformers including former NYC schools Chancellor Joel Klein.  How can this be? Well even though Pondiscio says the test scores need to be seen in the context of the family selection process, he also argues, several times throughout the book, that it is OK that they do this.  The argument is that wealthy families use their resources to get their child into a school that is a good fit for them so why shouldn’t poor families who have the resource of being highly functional use that to get their child into a school that is a good fit for them too?

On page 333 he writes

It would be dishonest to pretend that Success Academy is not a self-selection engine that allows engaged families who happen to be poor or of modest means to get the best available education for their children.  But it is equally dishonest and close to cruel to deny such families the ability to self-select in the name of “equity.” Indeed, it is nearly perverse to deny low-income families of color — and only those families — the ability to choose schools that allow their children to thrive, advance, and enjoy the full measure of their abilities.

This is not the first time I have heard this argument.  A few years ago Michael Petrilli wrote a piece called “Who Will Speak For The Strivers’ in response to an article that showed that charter schools have much higher expulsion rates than in neighborhood public schools.

My first response to this would be that only 16 out of the inaugural 73 students even endured to graduate Success Academy.  If a higher percentage were actually served by Success Academy, then this argument of ‘shouldn’t they also get to choose a school that is good for them?’ would be more compelling.  Since for the vast majority, they did not choose a school that was good for them, even after going through all those steps, and they did ultimately choose to leave, so what kind of choice did they really get?  For the small number of families and children that turn out to be a good fit after all, there are at least double that number who regretted that choice and surely feel duped by the false promise that Success Academy actually cares about their children.

Maybe an analogy will make this more clear:  On airplanes, only wealthy people have the choice of flying first class while people who can’t afford that must fly in coach.  So now Success Airlines comes along and they have something they give people the choice of flying in something like first class except the seats are outside the plane on the wings and you have to get to the seats on your own and there’s a 2/3 chance that you’re going to be jettisoned from that seat before the flight is over anyway.  Could this really be considered a viable ‘choice’ for poor people?

If Pondiscio is making the case here that Success Academy should have the right to exist, I’ve never said that they shouldn’t exist.  But their existence should not be to just benefit the few that are a good fit at the expense of not only the students at the neighboring schools but also the students who left Success Academy before graduating.  To do this, I think that they need more oversight and regulations and transparency about what goes on inside their schools.  And I’m glad that this book does a nice job about showing the sorts of abuse that occur in the school which I’ll get to next.

One thing that is striking about Success Academy is how young and inexperienced the teaching staff is.  On page 294, the teacher attrition is rationalized this way “Teacher turnover, and lack of experience and continuity, is widely assumed to be a problem, particularly in urban schools.  But it’s never suggested that our military would be better if only soldiers stayed in uniform longer. So far, the relative inexperience of Success Academy teachers hasn’t seemed to compromise their effectiveness.”  I guess the problem with trying to study Success Academy scientifically is that you can’t isolate the variables. So what would a Success Academy look like if they had self-selected parents and experienced teachers? Or a Success Academy with randomly-selected parents and inexperienced teachers?  Success has set up a controlled environment where even inexperienced teachers can seem like they are doing a good job. But the teachers seem to blindly follow the Success Academy protocol even when it is bad teaching practice. An example of this is their overuse of the classroom management strategy called ‘behavior narration.’  The idea is to praise students who are on-task, which of course is good in moderation. But they take it way too far, narrating and narrating which is very annoying and distracting to students and really unnecessary, their students would learn more if the teachers would cut back on that. 

The best example of a bad decision by a teacher, and probably the most frightening pages of the book, happened in Chapter 15, “Come to Jesus” one of the Kindergarten teachers holds a parent meeting in January because many of the Kindergarteners are not on target to reach level C by February.  The teacher hosts a parent meeting and makes a big speech about how everyone needs to work harder so the students will get back on track. She says that some kids have been absent too much and tells the parents “Anytime I’m sick, I come here. I can’t afford to lose that day with your children.  At the same time, your child cannot miss that day with any of us.” She tells them that if their Kindergartners don’t get on track they will not go to college. She tells them that if they don’t get to level D by the end of Kindergarten, they will have to be left back. To me, this is showing a lack of understanding of child development.  This teacher is not a parent and doesn’t really understand that children develop at different rates and that sometimes children who seem to be really behind on some developmental milestone often suddenly catch up. She asks if parents want to ask or say anything and a parent says that her child is on level C but maybe could be a D but when her child is struggling to read at home the child gets upset and the parent has her stop reading for the night.

The teacher responds that it is hard to push kids to do more and she tells them about how earlier that day a student did not do an adequate job on a book review and he lost a privilege.

From page 193:

“That was really rough for him to hear.  At the same time, did I say, ‘It’s OK if you don’t finish your book review?”  Now it’s Syskowski who starts to tear up. “No. It’s not OK. Because why would I let him fail when other kids are surpassing it and they’ll go to first grade?  You would not want me as your child’s classroom teacher. You would not want Mr. Carnaghi or Ms. Skinner to be your child’s teacher if we were like, ‘You know what?  You’re right. It is really hard. Let’s just let them be a B’” How about if they get two out of the four sight words correct? ‘That’s good enough.’ Where are they going to be in thirteen years?  Then we won’t talk about college. And that something that ..” Syskowski lowers her gaze to the floor. “I get chills. That’s something that’s really hard for me to …”

Syskowsky doesn’t finish the thought.  She can’t. She’s crying. “I do get emotional.  Because your children are amazing. They are absolutely amazing.  I try to …” She quickly gathers herself. “We will never lower that bar because it’s too hard.”

Pondiscio follows the description of this outburst with an interesting observation.  He says that some people might see this (as I do) as unprofessional and unnecessary. I was nodding my head at this since she is lying to the parents about how not getting to some arbitrary reading level in four months will mean that their children are on track to get left back and eventually not go to college.  But Pondiscio evidently doesn’t feel this way. He says that the critical view is the “dim view” of this. Then he gives another perspective, obviously, for him, the more accurate one:

“Or you can see Carolyn Syskowski, with her giant heart and Pez-dispenser grin, who calls every student “love bug” and spends hours each day on the floor with other people’s children, wipes their noses, pulls on their coats, sends them home, and then worries into the night about their reading and math scores.  You can see, if you are so inclined, an unusually gifted and competent teacher, with emotional gears you cannot fathom, who can issue a consequence to a five-year old like a bank examiner rejecting a loan, then an hour later bring herself to teacher in front of a hundred strangers when for a single moment she catchers herself weighing the cost of not doing so.”

I don’t buy this.  A teacher who does not care enough to understand basic child development and who lies to parents about how their children are not on track to go to college is someone who doesn’t really care very much.

The principal of the school is in her 30s and is portrayed, as I see her, with a narcissistic cult-leader mentality.  She wins an award at the end of the book and starts weeping about how she doesn’t deserve it and I’m inclined to agree with her on that.

Pondiscio is mainly permitted to observe, not coincidentally, the smoothest running Success Academy school.  He says early in the book that he was given mostly free rein of who he would observe though there were some teachers who he shouldn’t since it would make them uncomfortable.  He was also not given permission to freely go to other schools in the network. There was a chapter where he visited the Harlem 2 Success Academy with Eva Moskowitz and the place was so dysfunctional that Eva said she wanted to ‘slit her wrists.’  Interestingly, that school also gets very high standardized test scores which makes me think that perhaps all the other things the ‘good’ school does by the Success Academy methods really don’t matter, at least when it comes to the test scores.

In a section that even disturbed Pondiscio, [SPOILER ALERT:  SKIP THIS PARAGRAPH IF YOU WANT TO BE DISTURBED BY THIS IN THE CONTEXT OF THE FULL BOOK]  he interviewed the family of a student who was, essentially, kicked out of the school earlier in the book.  The student had behavior problems and the school responded to them by — get this — calling 911 for an ambulance to pick him up and bring him to the hospital for this several times. They also called child services to have them investigate the family.  A teacher who left the school abruptly may have also resigned over being asked to lie to investigators about this incident. I applaud Pondiscio for putting this section into the book. It really shows how heartless the school is.  The school defends themselves on things like this by saying things like “If a kid is throwing a chair and it can hurt another student, we have to do something,” but of course this is just an unnecessary overreaction designed to make the parents ‘voluntarily’ pull their child from the school.

Though Pondiscio is generally a thoughtful and nuanced thinker about education, I was kind of frustrated in the simplicity in the various times he talks about neighborhood public schools.  On page 17 he says “Publicly funded but privately run, charters offer a lifeboat to low-income kids from failing schools while in theory creating choice-driven market pressures that force bad schools — charter and traditional public schools alike — to improve or die.”  Words like “failing”, “bad”, and “lifeboat” are too charged for my taste.

There are places in the book where I would have liked Pondiscio to provide some numerical data.  Like when he writes on page 211 “It is not possible to say with certainty what percentage of the network’s students live in stable homes with both parents, or to make a comparison to students at public schools in the same districts, but dual-income household appear to be overrepresented among the Success parent body.”  Spending a year in a school, it seems like the sort of thing he could have easily come up with a good estimate had he wanted to.

There are some other places, however, where he does use numerical data though that data is often taken out of context and used as a way to bolster Success Academy or to counter some of the most common criticisms of Success Academy.  On page 54 he cites a CREDO study in which it was estimated that “Success Academy students gained the equivalent of 228 days [a year] in math.” I was disappointed to see him introduce a number as outrageous as that (a school year is only 180 days!) without at least mentioning that the ‘days of learning’ statistic is not meant to be taken literally.

On page 296 he uses data that seems to imply that Success Academy has a lower student attrition rate than NYC public schools in the same neighborhood.  This is something I’ve studied a lot so I want to address this here. The first graduating class at Success Academy was 16 students out of 73 that had originally started there as 1st graders.  (In the book Pondiscio mentions the small size of that graduating class a few times but only once mentions that the cohort started with 73 students and even then, didn’t make such a big deal out of it)  I’ve crunched the numbers from 2017-2018 school year to the 2018-2019 school year and found that if the grade to grade attrition rate continues for the next twelve years then the 2,647 Kindergarteners who started at Success Academy last year will be whittled down to only 246 graduating seniors 11 years from now.  (Out of 85,000 Kindergarteners who started in New York City last year, this is about 1/300 of the total population in that cohort, which is where the title of this review comes from.)

Screen Shot 2019-09-22 at 7.06.28 PM

Any study that says that Success Academy has lower attrition than neighborhood public schools is not accounting for the fact that the other schools replace the students who leave, and who often have the most unstable family situations, with other students with equally difficult situations.  They are also not accounting for the fact that students who leave one public school for another school similar to it are not faced with a huge dilemma. But a student leaving Success Academy — The Success Academy — for a public school, well, that’s quite a choice there. I would expect Success Academy’s attrition to be well below the attrition of public NYC schools unless they are forcing students and families out.

One of the more dramatic pieces of evidence of the selective attrition by Success Academy is that their graduating class of 2019 was 20 girls and 6 boys.  Try to find a public school in New York City (besides an all-girls school) that has a gender imbalance anywhere near that. This is the sort of statistic that I think is really compelling since highly functioning families have male babies as often as they have female babies so it indicates that the filtering process beyond the initial parental filtering helps Success Academy cheat even more to improve their test scores.

Seniors2023

There were a few mistakes, I doubt intentional, in the book I should point out since most readers would not would catch these.  On page 159 he writes about the first high school cohort who will graduate in 2017. He calls them “The first class of students to have spent their entire school careers as Success Academy students” when, in fact, the 2018 cohor would be that.  The 2017 cohort started as 1st graders at Success. On page 55 he writes about the famed “Waiting For A School Miracle” New York Times op-ed by Diane Ravitch. “She derided ‘miracle schools,’ citing several examples of schools whose scores skyrocketed one year and crashed the next.”  I helped Ravitch with the research for that op-ed and those were not schools that ever had high test scores. They were schools touted by politicians that had 100% college acceptance rates but actually had high attrition between 9th and 12th grade and always had terrible test scores.

OK, I’m going to bring this to a close.  I don’t want to spoil the book anymore than I did already.  There are many other important scenes and arguments in the book and I jotted things in the margins on nearly all of the 340 pages.  It is well worth reading and I’m thankful to Pondiscio for writing a book that, in my interpretation, will be devastating to the reputation of Success Academy.  He certainly tried to offer some counter arguments (mostly unconvincing to me) about why Success Academy is still a good place, all things considered.

Though Pondiscio is ‘reform friendly,’ he is not of the Kool Aid drinking type of reformer.  I think that in writing this book he is showing his frustration for the false promises of the reform movement.  After twenty years of growing, the reform movement has been a let down. There are some reformers who are in denial about this — most TFA staffers are like this.  There are other reformers who probably know how poor of a job the movement has done and they don’t really care since at least they are in power and that makes them money and feeds their egos.  But Pondiscio does not belong to either of those camps. I see Pondiscio as someone who is trying to use the scientific method to improve schools. He has written about how not enough of education research is based on brain science — especially about how students learn to read.  He believes, and I agree, that a lot of teacher time is spent inefficiently looking for useful curriculum materials. But unlike a lot of reformers who are not willing to evolve and incorporate the meager results of 20 years of reform experimentation into their thinking, Pondiscio writes on page 322, “But there are things we know and do not say in education and education reform.  One of them is that we expect too much of schools.” And the fact that the one apparent piece of evidence of the power of education reform, Success Academy, only gets its results by a form of cheating is particularly disheartening for a reformer who cares enough about the cause to pursue it scientifically. On page 321, “Is Success Academy a proof point that the reform playbook works and that professionally run schools with high standards and even higher expectations can set any child on a path out of poverty?  Or does the rarity of Moskowitz’s accomplishment suggest that however nobly intended it might have been, the reform impulse was doomed from the start?”

Sadly, most reformers are too stupid and overconfident to even know how bad they are at this. Whether or not at least their hearts are in the right place on this is also debatable. You can’t really claim to care so much if you don’t care enough to honestly assess the results of your reform experiment.  I can’t see how reformers can be honestly enthusiastic about the details in this book but if it is true that reformers do really like this book and are not just pretending to then Pondiscio has really accomplished quite a feat.

Four and a half apples out of five.

Posted in Uncategorized | 11 Comments