The Alum-lie

On the heels of the latest call by the NAACP for a charter school moratorium, there has been a media blitz started by The 74 about a report called ‘The Alumni’ in which they claim that charter school graduates go on to graduate college at three to five times the rate of low income students who do not attend charter schools.

Besides being reported in The 74, it has also appeared in The Wall Street Journal and The New York Daily News.

The 74 article is written by Richard Whitmire (as is The Daily News and The WSJ Op-Ed) who is known for his biography about Michelle Rhee (haven’t heard much about her lately) and also one about Rocketship Charters (haven’t heard much about them lately).

The summary of the report says that they have tracked the students at nine charter networks and found that graduates of those charters have between 25% and 50% of those students also graduate college.  Since a commonly quoted statistic is that only 9% of low income students graduate college, these networks seem to be getting between three and five times the rate of college completion.

The major flaw in this report — and they admit this in The 74, but not in The Daily News (The WSJ is behind a paywall, if someone can read it let me know if they address it there) — is that while the 9% statistic is for ALL students who enter schools, these 25% to 50% numbers are only for the students who complete 12th grade at the schools (KIPP is an exception, they use data from students who complete 8th grade — I’ll get to that later.)

In The 74 they actually do an entire other article explaining all the issues with the data that could cause the numbers to be inaccurate, which is something I appreciated, though of course the big takeaway will still be the three times to five times numbers that will be quoted, I’m sure, for the next decade by charter zealots.

Now there is really no way to verify these claims.  We don’t have a list of students who graduated and then another list of which ones graduated college.  But there are, thankfully, a few numbers that can be analyzed in the report.  I’ve done one network so far and hopefully will be able to do more another time.

Here is a quote from The Alumni about the Uncommon Schools network:

 Uncommon Schools: For the New York–based network, the only alumni who have reached the six-year mark graduated from North Star Academy Charter School in Newark. (The alumni from its Brooklyn high school just reached the four-year mark. Of the 142 North Star students who reached the six-year mark, 71 earned four-year degrees: a 50 percent success rate.

I went to the New Jersey data page where they have the databases for the enrollment at their schools throughout the years and downloaded all the data from the 2010-2011 school year down to the 1998-1999 data.

According to their data, there were eight graduating classes that have been out of high school for at least 6 years, the classes of 2011, 2010, 2009, 2008, 2007, 2006, 2005, and 2004.

The sizes of their graduating classes were, respectively, 24, 14, 36, 19, 27, 20, 23, and 19 which is a total of 182 students, not 142.  More importantly, going back more years I found that these 12th grade classes had lost about half their students from when they entered the school as 5th graders.  The 5th grade classes for those cohorts were about 40 each.  So if they really got 71 college graduates out of those 8 cohorts, it is not a rate of 71/142=50% but instead 71/320=22%.

Yes, I know that is still ‘double’ the expected 9%, but there are other factors that might make students who apply to charters have a higher rate.  Really we can’t be sure that there really were 71 graduates, but just taking them at their word for that, it still is dishonest to claim that it is a 50% college graduation rate.

Now the KIPP network claimed a 38% college graduation rate but they claimed that the other schools had inflated numbers because the other eight networks only counted students who completed 12th grade at their schools while KIPP counted students who completed 8th grade at their schools.

Here is a quote from the article about this:

KIPP is a fervent believer that college graduation cohort data should be tracked from ninth grade — not 12th grade, the starting point that the other charter networks included in this study use.

For students who attend KIPP middle schools, KIPP tracks them when they graduate from eighth grade to ensure they are kept track of, regardless of whether they go to a KIPP high school.

For students who go to non-KIPP middle schools and start attending KIPP as high schoolers, they track them when they start ninth grade.

The problem with starting in 12th grade, argues KIPP, is that it could tempt schools to push out weaker students during high school years, thus allowing the stronger students to boost the schools’ college-going and college-completion rates.

KIPP may be right. But in The Alumni, where KIPP is the only network that is currently tracking students from ninth grade, we have decided it is important to share cohort graduation rates that start in 12th grade. What’s key to this series is learning what works in boosting that college graduation rate — lessons that could be passed along to all schools, not just charters. Moving everyone to the gold standard is the next step.

KIPP is correct that schools that only count students who complete 12th grade will have inflated scores compared to KIPP that counts students who complete 8th grade.  But what KIPP doesn’t mention is that the fairest way to make a comparison to the 9% number is to start counting at 5th grade.  KIPP actually has a pretty big attrition between 5th and 8th grade so the true ‘gold standard’ is really not used by anyone.  All the numbers are inflated.  KIPPs might be inflated less than the others, but it still is so they can whine that the others are cheating worse than they are on this statistic, but they should admit that they are doing it too, though to a lesser degree.

There isn’t a lot of detailed data included in the report, just a summary of the main findings.  And I’m not so confident we can trust the data in there anyway, but it still is interesting to see how the numbers are inflated by ignoring attrition.

This entry was posted in Uncategorized. Bookmark the permalink.

9 Responses to The Alum-lie

  1. Patrick J. Sullivan says:

    Here’s the text from the WSJ, no mention of the basis for the comparison: “Roughly half the graduates of Uncommon, YES Prep and the KIPP New York schools—among the biggest and best known charter networks in the country—earn bachelor’s degrees within six years. About a quarter of the graduates of the lower-performing charter networks earn degrees within six years. That may not strike wealthy parents as something to brag about. Eighty percent of children from America’s wealthiest families earn four-year degrees within six years. But charters primarily serve low-income families, where only 9% of students earn such degrees. Charters make a difference for poor families.”

  2. Stephen B Ronan says:


    I think that you and The 74 are correct to be cautiously skeptical of college completion rates of a high school’s graduates in the absence of the H.S.’s attrition data.

    However, kindly correct me if I am wrong, but it appears to me that you are claiming that those who transfer out of a charter school after 5th grade should typically be included in the denominator but not the numerator when calculating the graduation rates and college completion rates for the charter school from which the student departs.

    Is there any state in the USA that follows that procedure for calculating graduation rates?

    What would the 9% figure for students from low-income families look like if, for the purposes of calculating graduation rates, one removed only from the numerator, but not the denominator, any student who transfers out of one school and into another after 5th grade?

    Here is the Massachusetts system for calculating schools’ graduation rates; my sense is that it is not atypical:

    It can create somewhat misleading results if they’re only examined superficially and in isolation from other data; as is the case for every other readily implemented technique for calculating graduation rates that I’ve seen or can imagine.

    In respect to this from The 74 article, “The Data Behind The Alumni,” which you cite:

    “The Academy of the Pacific Rim, for example, was able to track every alumnus, discovering students who earned four-year degrees who were missed by the Clearinghouse. As a result, its success rate rose from 60 percent to 70 percent.”

    I had the privilege of attended their high school graduation ceremony this year… and was awestruck.

    (Nevertheless, with cautious skepticism I investigated, and the summer attrition rate there seems lower, and school-year stability rate higher, than the local traditional district schools).

    • garyrubinstein says:

      Hi Stephen,

      This is an interesting point. In a sense, they are giving what I’d call an upper bound on the college completion rate and I’m giving a lower bound. I believe that ‘true’ number is closer to this lower bound than to the upper bound. For sure, I think that there should be some kind of ‘penalty’ for not holding on to the students. When over half the students leave the school and many of those who leave are the ones who were struggling in the school and would likely be the ones to bring down their college rate, they just can’t get credit for them and I won’t assume that the kids who left had the same college completion rate as those who stayed. Considering that the most motivated families who opt into the lottery probably have a more likely chance of graduating anyway, it is pretty hard to interpret if these rates of 25% to 50% are actually good (if they really ‘added value’). My main point is that i believe these numbers are inflated and I tried to defend that position. The report itself actually admits these issues but then they make these summaries in the Wall Street Journal that don’t mention any of these issues, and I think that is their plan.

  3. esantoine says:

    Here’s the WSJ article.

    Antoine Education (718) 501-5111

    On Fri, Jul 28, 2017 at 10:05 PM, Gary Rubinstein’s Blog wrote:

    > garyrubinstein posted: “On the heels of the latest call by the NAACP for a > charter school moratorium, there has been a media blitz started by The 74 > about a report called ‘The Alumni’ in which they claim that charter school > graduates go on to graduate college at three to five ti” >

  4. Pingback: Gary Rubinstein: Charters and the Big Lie About College Graduation Rates | Diane Ravitch's blog

  5. Laura H. Chapman says:

    Thank you for debunking the PR claims of the charter industry.

  6. Pingback: Ed News, Tuesday, August 1, 2017 Edition | tigersteach

  7. Pingback: UofA + KIPP = Lies: More Rhetoric in the Absence of Evidence | radical eyes for equity

  8. Pingback: The Alum-lie Spreads | Gary Rubinstein's Blog

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s