Two weeks ago, The 74 published the results of The Alumni which claims to show that the graduates of certain charter school networks go on to graduate college at a rate of 3 to 5 times the rate of low-income students on average.
They say that only 9% of low-income students graduate college after six years while among the nine charter networks they studied, their graduates had college completion rates ranging from 25% (approximately 3 times 9%) and 50% (approximately 5 times 9%).
The problem with this calculation is that the charter schools are only counting students who completed 12th grade at that school (or for KIPP, 8th grade). So if a school only has 14 graduates and 7 of them graduate after six years, it is accurate that 50% of their graduates went on to complete college, but if that cohort of 14 students was 40 students three years earlier, then their rate is really 25%. In other words, by just counting the ‘graduates’ they get an inflated college completion rate.
In the original The Alumni article the author, Richard Whitmire, admitted as such. He even put a comment from the KIPP network about how the other schools should use 8th grade as the cutoff so they don’t get unfairly inflated percentages. I argued in my first post about this that 5th grade would be an even more accurate cutoff.
In a follow up article on The 74 called The Data Behind The Alumni the case is made even stronger:
The one network that insists on including students who leave the system is KIPP, which reports its college success data starting in ninth grade for students new to the KIPP system and at the end of eighth grade for existing KIPP students. YES Prep, part of the United for College Success Coalition in Texas, has promised to start calculating its college success data from ninth grade, but no figures are yet available.
All the other networks start their data set in 12th grade — and say they don’t have data that begins in ninth grade. KIPP takes a principled stand on that issue, refusing to release any results that start the tracking in 12th grade, despite the fact that it would boost its college success rate.
Within the charter community, this is turning into a hot-button issue. KIPP feels very strongly that the only honest method for reporting graduation is to start in ninth grade. In theory, a charter network could increase its college success numbers by pushing or counseling out weak students before their senior year. That would apply to any high school, not just charter high schools.
You’ve got to love the part about the other networks “say they don’t have data that begins in ninth grade.”
Also on The 74, and again to their credit, they published something by the chief executive officer of YES prep. He brags first that the class of 2010 had a 54% college completion rate, but then, a few paragraphs later admitted about that same cohort:
When I was principal of YES Prep North Central in 2010, only 34 percent of our founding sixth-grade class went on to graduate from our campus. This unacceptably low persistence rate, a symptom of a “no-excuses” culture, needed to be addressed to align with our mission of increasing the number of students from underserved communities who graduate prepared to lead.
Suddenly the 54% turns into 54% of 35% which is just 18% college completion.
He says that from now on they are going to use the 8th grade cutoff like KIPP, though of course he should be pushing it to 6th grade based on what he just said about their huge attrition numbers.
Another interesting statistic in that follow up article is that if you want to do a more fair comparison you would want to compare not to the 9% number but to the percent of low-income high school graduates that go on to graduate college within 6 years. To their credit, The 74 does say that this is actually 15%, to which they then say:
Even if as many as 15 percent of low-income minority students who make it through high school earn college degrees, that means these top charter networks are still doing three and a half times better.
So suddenly it goes from 3 to 5 times better to 1.6 to 3.3 times better. This is important since surely the 3 to 5 number is the thing that is going to be remembered by people who read just the first article or one of the editorials Whitmire has published in The New York Daily News, The Wall Street Journal, and, most recently, The Hill. There was also a report about The Alumni by another writer in The Houston Chronicle.
So after reading Whitmire’s The Hill piece which has no mention at all the issues with the calculations, I reached out to him on Twitter and we had this amusing exchange.
And that was it. I have no idea what his last tweet means. I think Trump might have written it. Why would I back off if I found flawed data. I just don’t get it. This is why most reformers don’t engage with me. In any kind of debate they have with me on equal terms I absolutely embarrass them.