TFA Celebrates Five Baltimore ‘Turnaround’ Schools. One Is Still Ranked In The Bottom 1%.

As an ashamed TFA alum, I receive their quarterly alumni magazine, ‘One Day.’  In the most recent issue, which I also saw on their Twitter feed, was an article called ‘Undefeated: Inside Five Baltimore Turnaround Schools that Refuse to Fail.’

The article is about five Baltimore schools that are run by TFA alumni and were recipients of some of the Obama/Duncan $3 billion school turnaround grant.  The most aggressive turnaround strategy is to replace the majority of the staff, which is what these five schools did.  The school turnaround grants have generally been considered a failure across the country, even by staunch reformers.

But, at least at a first glance, these turnaround schools were exceptions that prove that firing all the teachers at a school and replacing them, presumably with a lot of TFA teachers, is something that can work as long as TFA leaders are involved.

The five schools are Commodore John Rogers Elementary/Middle, Academy for College and Career Exploration (ACCE), Harford Heights Elementary, James McHenry Elementary/Middle, and Mary E. Rodman Elementary.  They are three years into their turnaround efforts under a plan called ‘The 100% project.’  According to the article, these schools are showing promising improvement though they still have a long way to go.

They provide very little data in the article.  One example is:

 Dave Guzman is the principal of Rodman Elementary, where 23% of students passed the state math assessment in 2018, up from 2% prior to turnaround, and 22% passed the ELA assessment, up from 5%.

Another is:

Based in part on its dismal test scores, Commodore ranked 872 out of 875 schools in Maryland when Martin was hired to lead it in 2010. At that point, the school enrolled 250 students, barely viable in a building built for more than 600.

Today, in a district grappling with steep enrollment declines overall, Commodore squeezes in more than 900 students (including Martin’s three kids), almost all of them from the neighborhood. They crowd into classrooms and spill into meeting rooms. They collaborate on clusters of chairs pulled into alcoves under the stairways. Not a minute too soon, renovations and expansions are planned for completion in 2021.

In October, Commodore was recognized for excellence by the Baltimore City school board. Over four years, 37% of its students who initially tested at the lowest level of proficiency moved up to one of the high levels, compared to 14% of similar students districtwide. In 2019, the school earned Maryland’s three-star rating (out of five), up from two stars in 2018. The improvement reflects factors like Commodore’s high attendance rate and parent satisfaction, as well as student test scores, which consistently meet or exceed Baltimore City averages. (The state no longer ranks schools numerically from best to worst.)

It’s been a while since I’ve done a ‘debunking.’  I remember back in the day, about 8 years ago, stories of miracle turnaround schools were popping up every few days.  After my posts about the miracle schools and usually showing other aspect of their data that were not so miraculous, I would get a lot of trolling from the guys at Education Post or the other lesser known reform propaganda sites about how I don’t believe in the potential of kids.  So let me say, again, for the record:  I do believe in kids and I also believe that almost every school has room for improvement.  I also can imagine a scenario, though it would be rare, where there is a huge percent of low quality teachers and that if that staff were replaced by high quality teachers, the school would improve — that’s just common sense.  But what I consider to be a false miracle is when a school with average teachers replaces their staff with other average teachers and suddenly gets a huge increase in how much their students learn.  I think that schools need more resources for smaller class sizes in order to get an authentic increase in quality.  That’s why I’m accused of not believing in children.

OK, so I took a look at the Maryland school report card site, and here’s what I learned about these five schools that have been ‘turned around.’

Maryland has the star system where schools can get from one to five stars, kind of like the A to F letter grades.  The stars are based on test scores and also on ‘growth’ and other factors.  There are 1,300 schools in Maryland and about 10% of them get either one or two stars.  So 3 stars is like a ‘C’ and over 60% of the schools in the state are either 4 stars or 5 stars.  Of the five schools that have been ‘turned around,’ three are still 2 stars, which is like a ‘D.’  But looking more closely at the data from these five schools, I found some pretty awful numbers.

The Commodore John Rogers Elementary/Middle School  that has the test score increases got two different percentile ranks, one for the elementary and one for the middle school.  While the middle school is the one bright spot of all the schools , or subschools, in the 100% project, having risen to the bottom 28% of schools the elementary school is ranked in the bottom 8%.

One school, The Academy For College And Career Exploration (ACCE) has a middle and a high school.  The middle school is ranked in the bottom 2% while the high school is in the bottom 9%.  In the high school they had 9.3% score proficient in math and 3.6% score proficient in ELA.  In the middle school they had 2.7% score proficient in ELA and, no this isn’t a typo, 0% score proficient in math.

The lowest rated school of the five is James McHenry Elementary/Middle.  While the middle school was ranked in the bottom 15%, the elementary school was only ranked in the bottom 1%.  If not for the middle school, the elementary school would be one of the 35 schools out of 1,300 that would have gotten just one star and be slated for possible closure.

I’m not sure why TFA is clinging to a narrative that went out of style about five years ago, when Arne Duncan stepped down as Secretary of Education.  These five schools, on average, do not prove that firing most of the teachers in a school is likely to cause an incredible turnaround at a school.

Posted in Uncategorized | 1 Comment

The Life And Death Of The Terrible Education Reform Movement

I’ve often thought, over these past eight years of following the politics of education, that one day this saga will make a great book.  What I didn’t expect was that that one day would be today.

slaying

In ‘Slaying Goliath: The Passionate Resistance to Privatization and the Fight to Save America’s Public Schools’ (Knopf 2020, $27.95), education historian Diane Ravitch does the thing that she is best in the country at — taking a complex period of history in education and finding a way to turn it into a story with twists and turns worthy of great literature.

This book is like a sequel not to ‘Reign Of Error’ (2013) or even ‘The Death And Life Of The Great American School System’ (2010), but instead to the first book I had ever read by her, ‘Left Back: A Century Of Failed School Reforms’ (2000).  In that book, as she does in this one, Ravitch methodically weaves her way through the people and events that shaped the conflicts of education policy.  She is a master of brevity and I, as a pretty verbose blogger, marvel at how she can tell the story of such complex issues like the opt-out movement or the Massachusetts charter school ballot issue in just a few pages each.  It’s like watching one of those artists who with a few simple seeming strokes of a pencil captures the essence of her subject.  It looks so easy though of course it isn’t.

Watch as she summarizes twenty years of education reform in three paragraphs (!) on the third page of the book.

For nearly two decades, the “reformers” had promised a dramatic transformation in American education, based on their strategy of high-stakes testing, teacher evaluation by test scores, charter schools, and closing low-scoring public schools.  They confidently claimed that they knew the answers to all the vexing problems in education.  They asserted that they were leading the civil rights movement of our time, funded by billionaires, Wall Street titans, and the federal government, as if the elites would be leading a civil rights movement against the powerful (themselves!).  They insisted that when their remedies were imposed, America’s test scores would soar to the top of international rankings.  No longer would poor children be “trapped in failing schools.”  No more would children’s success be determined by their ZIP code or social status.  They all sang from a common hymnal about the failures of public education and proclaimed their certainty that they knew how to turn failure into high test scores for all.

But despite the investment of billions of federal, state, local, and philanthropic dollars, these malign efforts came up empty.  The leaders of this charade had confidently predicted that success was just beyond the horizon.  But as so often happens with mirages, the horizon kept receding farther away.  None of their promises and claims came true.  Judged by their own chosen metrics — standardized test scores — the fake ‘reformers’ failed

In this book, I will not call these activities and their leaders by the honorable word reform, which they have brazenly appropriated.  The individuals and groups who promote test-best accountability, school closings, and school choice as remedies for low test scores are not reformers.  What to call them?  Others call them “deformers” or the “financial privatization cabal” or the “Destroy Public Education Movement” or “privateers.”  Such groups and individuals often say their goal is to “disrupt” public education, and I think in this instance they have accurately named themselves.  They are Disrupters.  They are masters of chaos, which they inflict on other people’s children, without a twinge of remorse.

And that is just one out of 283 pages!

As someone who has been embroiled in this subject for the past eight years, I figured that I would be able to skim through this book.  I figured that I wasn’t really the audience for this book, it was for readers who can’t name the last three superintendents of the Tennessee Achievement School District off the top of their heads.  Yet I found myself learning something new on every page.  One particularly compelling, and frightening, section was the part about how billionaires funnel money into local school board races.  I had heard about this from time to time, but I never appreciated how much this is happening.   There were fascinating events that I somehow missed altogether when they occurred, like the Douglas County school board saga.  This is why Ravitch is such a master.  She absorbs all these events when they happen and then when it is time for her to don her ‘historian’ cap and to elevate 30,000 feet in a metaphorical hot air balloon and look down and see how the different pieces of this puzzle fit together in a way that us land dwelling mortals could never have the perspective to notice.

If I learned something new on almost every page, the average reader is going to have his mind completely blown by the narrative in this book.  It is a story of underdogs — regular people and most certainly NOT the big bad union who, lets face it, have not played much of a role in this — who stood up against the most powerful and the most wealthy people in the world and who said “No.  You are not going to destroy our schools.”  Against all odds, we see that the tide has turned

Throughout the book, Ravitch introduces the readers to a cast of characters that are only known to those of us who have been embroiled in this fight over the years.  We get to read short histories of people like Karen Lewis, Leonie Haimson, Anthony Cody, and others.  She paints them as heroes and shows how they fit into the big picture where every one of the heroes plays a small but important role.  The only hero that she does not give her due credit is herself — Diane Ravitch.  If someone else were to have written this history — and I don’t think there was anyone else alive who could have done this — Ravitch would have been recognized as THE key figure in taking down the ‘disruptors.’  Her blog with the tens of millions of hits, her Twitter following, and her books — including this new one — are not just recording history, they are making it.  The only mention she makes about her own role in leading the resistance was on page 68 where she wrote “I started my own blog in 2012 and use it as a platform to nationalize the struggle against privatization and high-stakes testing and to showcase the work of other bloggers in order to enable them to reach a larger audience.”

The fact that Ravitch had the energy to write this book after all else that she has done and continues to do is astonishing.  It is as if George Washington not only led the American efforts in the Revolutionary War, but if when it was nearly over he were to also write the definitive history about it.

In summary, this is a must read.  Whether you are just learning about these issues or if you are someone who has studied them for years, you will get so much out of this.  And you really want to get this and read it right away before everyone else starts talking about it and you’re going to feel very left out.

In the early 1500s Michelangelo completed his sculpture ‘David’.  Though it took 500 years for another genius to rival his, Diane Ravitch’s ‘Slaying Goliath’ is a masterpiece that will be studied and celebrated for generations.

Posted in Uncategorized | 4 Comments

The Tennessee ASD: Booted or Re-Booted?

Since 2011 I have been following the biggest, and most predictable, disaster of the education reform movement — the Tennessee Achievement School District (ASD).  It was formed in a perfect storm of reform theory.  First, Tennessee won Race To The Top money.  Then they hired a TFA-alum and the ex-husband of Michelle Rhee, Kevin Huffman to be their state commissioner.  Then he hired TFA-alum and charter school founder Chris Barbic to design and run the ASD.  The initial promise of the ASD was that they would take schools in the bottom 5% and convert them into charter schools in order to ‘catapult’ them into the top 25% in five years.  They started with 6 schools in 2012 and grew to over 30 schools within a few years.

They completely failed at this mission.  Chris Barbic resigned, Kevin Huffman resigned, Barbic’s replacement resigned, Barbic’s replacement’s replacement resigned.  Of the 30 schools they nearly all stayed in the bottom 5% except a few that catapulted into the bottom 10%.

The new education commissioner of Tennessee is also a TFA alum with ideas similar to Huffman.  She promised, however, to get a handle on the ASD and what to do about its failure.  After a listening tour around the state she made, it seemed at first, a decision that was long overdue.

Chalkbeat TN recently had a post with the enticing title ‘All 30 schools in Tennessee’s turnaround district would exit by 2022 in a massive restructuring proposal.’  It would seem like this is good news.  The ASD was such a costly failure, costing about $100 million over the years I think, the only thing to do was to put it out of its misery and dissolve it completely.

But I’ve been studying reformers enough over the years not to get too excited about this.  The headline would make the most optimistic readers think that the 30 schools going back to the district would again become public schools.  The charter schools supposedly traded flexibility for accountability so their failure to deliver on their promises should result in them being sent packing.

But according to the article, it is not clear yet if being returned to the district means that they will become public schools again.  Also they say that there still will be an ASD after this.  Now there can’t be a school district with zero schools, so what’s going on?

I think, and I hope I’m wrong about this, that with the failure of the ASD there was no way that they could justify adding more schools to it.  But by ‘returning’ the 30 schools back to their districts, and probably keeping them as charters, there will now be room to add more schools in the bottom 5% to the re-booted ASD.  If this is what happens, the ASD won’t be disappearing or even shrinking, it will be expanding.  There will be the 30 schools that are still charters, but just operating as part of the district they have been returned to.  And then there will be another 20 schools, maybe, that are in the new ASD.  (They actually call it the ASD 2.0 in the state slide show)

Basically, this is like when a businessman declares bankruptcy yet finds a way to get out of debt that way and actually profits off of it.  Without those original 30 schools that are making their stats look so bad over the years, they will start fresh with other schools.  Then they can spend another 8 years with those schools and say “You can’t expect us to fix these schools overnight, we need more time.”  But this is just a shell game.

That’s what I think is going on.  Maybe I’m wrong, but don’t be surprised if I’m right.

Posted in Uncategorized | 6 Comments

More Reform Funded Research: KIPP Graduates Persist In College At The Same Rate As Their Mothers

Education Reform propaganda at The74 would try to make you believe that while low income students generally graduate from college at a rate of about 9%, charter school graduates complete college at a rate of 3 to 5 times that.

The main flaw in any comparison between the college graduation rates of charter school graduates to low-income students, in general, is that the charter school students do not represent a random sampling of the general population of low-income students.

In The Alumni, Richard Whitmire says that charter schools that have 5 times the expected college completion rate are ones that only counted their students who persisted until 12th grade in their charter schools.  Since for some charter schools, this only represents about 25% of the students who started in that charter school, this even more of a biased sample.  But, Whitmire explains, the one network that has the most valid way of doing a fair comparison is the famed KIPP network.  Since KIPP counts, in their data, any students who enrolled in KIPP, even if they left soon after starting.  And he says that KIPP students, including ones who didn’t persist at KIPP, graduate college 3 times the expected rate.

Reform supporting billionaire John Arnold commissioned Mathematica, a data analysis company, to study the college enrollment and college persistence of KIPP students.  Instead of comparing KIPP students to the general population, they compared KIPP students to students who had applied to the KIPP lottery but did not get into KIPP through the lottery.  This is a much more valid way of measuring the impact of KIPP.  The big takeaway, as I wrote about in my previous post, was that students who applied to KIPP, whether or not they got into KIPP, had a college persistence rate of about 3 times the general low-income population and that students who applied but didn’t get into KIPP had about the same college persistence as students who applied and did get into KIPP.  So students to apply to the KIPP lottery are the ones who, on average, were much more likely to persist in college — something that Whitmire never mentions in The Alumni.

But this Mathematica report includes some other relevant data that I didn’t pick up on when I wrote the last post.  Fortunately there was a discussion among some readers who commented on the last post which pointed this out.

In 2018 the National Center For Education Statistics published a report called ‘First-Generation Students College Access, Persistence, and Postbachelor’s Outcomes.’  In it they say that about 70% of students who have a parent who completed college also complete college compared to about 35% of students who do not have a parent who completed college.  This confirms what most people would expect for so many reasons and this is why we celebrate when students are the first in their family to graduate college.  It means that the descendants of those students will also be more likely to go to college.

In the Mathematica study, they collected statistics about the pool of students who applied to the KIPP lottery.  Among those statistics was the level of education attained by the mother of the KIPP lottery applicant.  Here’s what it says (page 6 of the report):

Screen Shot 2020-01-12 at 11.11.20 PM

Notice that last line.  It says that of the students entering the lottery about 27% of them had mothers who finished college.  This makes the fact that about 30% of the students in the study (which includes students who got into KIPP and also students who did not get into KIPP) have persisted in college through four semesters even less surprising.

Someone like Richard Whitmire suggests in his analysis that has been quoted in USA Today and The Wall Street Journal, that had these students not gone to KIPP they would have only persisted in college at around 9%.  But it can be clearly seen now that even without going to KIPP, these students should be expected to have about a 30% college persistence rate.

Data, like what percent of charter school applicants have parents who are college graduates, are so important but nearly impossible to actually learn.  Thanks to the John Arnold Foundation for commissioning this study and shedding light on a truth that we already knew but didn’t yet have hard data to support.

Posted in Uncategorized | 3 Comments

Reform Funded Research: Winning KIPP Lottery Does Not Significantly Increase Chance Of Persisting In College

The way reformers misuse data follows a very simple and predictable plan:  First they get some skewed data, then pick a ‘researcher’ to interpret the skewed data.  The ‘researcher’ then writes a report which gets touted in The74, EduPost, and eventually even makes it into more mainstream publications like USA Today and The Wall Street Journal.  Since the report is filled with nonsense and half-truths, within a few weeks the truth comes out and the report is discredited, but not before the damage was done and the spin has made it into folklore.  When this happens, the reformers will then ‘move the goalposts’ and get some more skewed data and start the process over again.

An example of this is the July 2017 report by Richard Whitmire called ‘The Alumni‘.  Whitmire has written books about both KIPP and about Michelle Rhee so I think you get the idea of what his point of view is.  In this poorly researched project he concludes that “Data Show Charter School Students Graduating From College at Three to Five Times National Average“.  The national average he is comparing to is the 9% of low-income students that graduate college, according to the Pell institute.

This was probably the easiest report I ever debunked.  The biggest flaw was that for most of the charter schools, they were only counting the percent of graduating seniors who persisted in college and then comparing that percent to the overall percent of all low-income students — an apples to oranges comparison.  Whitmire acknowledges this in another post about the methodology in which he says that only KIPP counts students who leave the school before they graduate and that their numbers are much lower, but still at 38% which is at least triple the expected graduation rate for low income students.

A second flaw, and this one is very difficult to compensate for, is that charter school students are not a random sampling of all students since many families choose not to apply to them.  So you get a biased sampling even if you do count all the students who get into the charter school and not just the ones who make it to graduate from the charter school.  And even though I and others have discredited his report, it is something that still gets quoted in the main stream media.

Just recently, however, I learned of a report generated by Mathematica and funded by the John Arnold Foundation.  I think that Mathematica is a very reputable company and even though reformers often hire them to produce reports, sometimes those reports reach conclusions that reformers were not expecting.

In this case, the report called “Long-Term Impacts of KIPP Middle Schools on College Enrollment and Early College Persistence” , reached a result that completely contradicts Whitmire’s claim that “Charter School Students Graduating From College at Three to Five Times National Average”.

What Mathematica did was follow 1000 students who applied to KIPP schools that required the use of a lottery since they had more applicants that open slots.  500 of them were offered spots and KIPP and 500 of them were not.  Comparing the fates of lottery winners to lottery losers is more accurate than comparing KIPP students to students who never even applied to a lottery since the KIPP students are not a random sampling.  The study compared the college persistence rate (what percent of each group completed four semesters of college) among other comparisons.

Here’s the relevant summary of what they found:

Screen Shot 2020-01-04 at 5.07.25 PM

This needs a lot of unpacking:  So 30.4% of the group who were offered a spot at KIPP were still in college compared to 25.6% of the group who were not offered a spot at KIPP.  Remember that Whitmire claimed a 3x to 5x comparison for charter chains.  Well, even if you just take these two numbers without any other context they provide, this is 30.4 is 1.1875 times as big as 25.6.

The big headline here should be that from this study in which all of the students involved are ones who entered the KIPP lottery (and only half of them actually attended a KIPP), about 28% of them completed four semesters of college.  Where is Whitmire’s post about how this shows that simply applying to KIPP (whether you get in or not) increases your chances of persisting in college by 3 times?  Really this shows what everyone already knows, but which anyone reporting about ‘The Alumni’ report ignores, that students who apply to charter lotteries are not a random sampling and therefore any comparison between college enrollment and college persistence for charter schools vs. all schools is going to be a flawed comparison.

Now since 30.4 is greater than 25.6, it might seem at first that winning a spot in a KIPP lottery causes a slightly higher chance of completing four semesters of college.  But the report also says that this difference “is not significantly different from zero (p-value 0.135).”  What does that mean?

Most people don’t understand what the difference between “large” and “significantly different” when it comes to statistics.  Here’s an analogy that demonstrates this a little.  Suppose there were 100 students altogether and 30 of them completed four semesters of college and 70 did not.  If you took those 100 people and split them into two groups by some sorting rule, like one group is people who have an even social security number and the other group is the people who have an odd social security number and, it turns out, there are 50 people in each group.  You would expect, since there is nothing about having an even or odd social security number that would cause one group to do better than the other group, that each of these groups would have about 15 students who completed four semesters of college, since that would be 30% for each group.  But would you be shocked if it turned out that the group with even social security numbers had 16 students (32%) who completed four semesters of college and the other group had 14 students (28%) who did this?  Would you feel confident in saying that having an even social security number somehow causes people to be more successful in college?  Of course not.  It was still very close to the expected 15 / 15 split.  Well this is what is meant by “not significantly different from zero (p-value 0.135),” in a nutshell.  The larger the p-value, the less significant the difference is with a ‘good’ p-value being very small, less than 0.05.  So this report cannot say with appropriate confidence that winning the KIPP lottery is associated with any increase in college persistence rate.  This is surely not what The Arnold Foundation would have hoped would come from this and I seriously doubt that The74 or EduPost will write about this.

There were three other comparisons that they did for this report and those three look better for KIPP so I want to explain about those also.  Some of the students who won the lottery to get into KIPP did not, for whatever reason, go to KIPP.  Maybe they didn’t want to go, maybe they were discouraged by KIPP not to accept their offer, who knows?  So in addition to the comparison I just mentioned, there is a second comparison where they compared students in the study who did not go to KIPP (lottery losers and lottery winners who did not go for whatever reason) to students in the study who got an offer and also did end up going to KIPP.  For that, the difference in the percent of students completing four semesters of college was 9% (24% vs 33%).  This was still not considered a ‘significant’ difference by the authors of the report and for me, I would consider this statistic biased anyway.  When you remove the lottery winners who did not go to KIPP for whatever reason, you would need to also remove the lottery losers who would not have gone to KIPP had they won the lottery.  Since it is not possible to know who from the lottery losers would have not gone to KIPP, this is a comparison that I consider to be flawed.

Screen Shot 2020-01-04 at 5.09.38 PM

There were two other comparisons where instead of college persistence, they compared just college enrollment.  For this one there was a 7% difference which they considered ‘significant’ for lottery winners vs. lottery losers and a 13% difference for the other way where only the students who got into KIPP and enrolled in KIPP were part of the treatment group.

Screen Shot 2020-01-05 at 9.34.16 AM

Maybe there is a benefit to going to college for a few semesters and then quitting before completing the fourth semester.  I think that since charter schools talk so much about “To and through college,” getting into college and leaving so soon seems like something that they should not be celebrating.

The big takeaway, though, from this recent report is that it is an excellent counter to the 3x to 5x propaganda claim spread by the reform blog sites.  This suggests that the number is “not significantly different from zero” when a more valid comparison is done.

Posted in Uncategorized | 8 Comments

Success Academy Class Of 2020 Sheds 239 Scholars Along The Way

The New York Post recently ran an editorial about the SAT scores of the Success Academy senior class of 2020.  Of all the different numbers they referenced, one that I took note of was 114 — the apparent number of students in the senior class.

The class of 2020 is the third graduating class of Success Academy.  The class of 2018 had 17 seniors out of a cohort of 73 first graders in 2006-2007.  The class of 2019 had 26 seniors out of a cohort of 83 kindergartners in 2006-2007.  Some of the class of 2019 were students who had been held back from the class of 2018 — probably in a comparable number to the number of 2019 students who will graduate this year.  So the 26 out of 83, or 31% persistence rate probably accounts for students who take an extra year to graduate.

For the class of 2020, things get a bit more complicated since in 2008 Success Academy did its first expansion and grew from one school, now called Harlem 1, into four schools now including Harlem 2, Harlem 3, and Harlem 4.  Some of the past records are incomplete for these schools, but when the 2020 cohort was in 2nd grade in 2009-2010, I find that there was a combined 353 students in the cohort.  By 6th grade, they were down to 263 students and by 9th grade it was 191.  In 10th grade they were 161 students and in 11th grade, 146.  And now, according to the New York Post article based on a Success Academy press release, they have 114 seniors.  So only 32% of the students who were there in second grade made it through their program.  And even more startling is that of the 191 9th graders that had been at Success Academy for 10 years, only 59% of them are on track to graduate three years later.

SA2020

[all data gotten from https://data.nysed.gov/ ]

This 32% persistence rate doesn’t even include the students who ‘backfilled’ some of the empty spots for students who have left over the years.  Without access to more granular data, this isn’t something I can study right now.  From Robert Pondiscio’s book about Success Academy we learn that the backfill process is somewhat corrupt.  Families that get off the wait list to backfill vacated spots are sometimes told that if they come to Success Academy their children will have to redo the grade they had just passed in their other school.  Surely many of these families choose to forfeit their place off the waiting list and, in that way, Success Academy makes sure that the backfill students are generally the higher performing students which serve to inflate the school’s test scores.

According to Success Academy, the demographic data for their students are:  74% receive free or reduced-price lunch, 16% have disabilities, and 8% are English language learners.  But the most recent data (From 2017-2018) about the 2020 cohort from the New York State public site is that 66% are economically disadvantaged, 11% are students with disabilities, and 0% English language learners.

Success Academy and The New York Post love to claim that Mayor de Blasio is out to get Success Academy.  I really don’t think so because de Blasio has access to the type of data that could so easily expose the various ways that Success Academy tips the scale in their favor.  Just using publicly available data, I’ve been able to uncover so much about their massive attrition rate.  Imagine how much can be learned from the data that the New York City DOE can access.

Posted in Uncategorized | 13 Comments

Does Louisiana Really Lead The Nation In 8th Grade Math Gains?

The latest National Assessment of Educational Progress (NAEP) scores for the 2019 tests were released on October 30th.  Unlike state tests for which the cutoff scores can be manipulated for political purposes, the NAEP does seem to be somewhat unbiased.  So the NAEP, sometimes called ‘The Nation’s Report Card’ does offer an interesting amount of data that I believe is worthy of analysis.

Often the NAEP results are, intentionally or unintentionally, interpreted to see if it is possible to find some kind of correlation between the education policies a state has enacted and the corresponding NAEP results.  In Obama’s 2014 State Of The Union address, he mentioned that D.C. and Tennessee were improving — as evidenced surely by their NAEP gains from 2011 to 2013 — to show that his Race To The Top recommendations, which were followed closely by those two regions, were working.

So when the 2019 results came out the other day, things looked bad for the reformers.  From 2017 to 2019, the average scale score for 4th grade reading was down 1 point, 8th grade reading was down 3 points, 4th grade math was up 1 point, and 8th grade math was down 1 point.  Though it is not clear to the public whether or not one ‘point’ is a lot or a little, everyone can agree that it is better if the scores go up rather than down.

Only one region had an increased score in 4th grade reading while 17 states decreased.  Only one region increased in 8th grade reading while 31 states decreased.  For 4th grade math, 9 states increased while 3 decreased, and for 8th grade math 3 states increased while 6 states decreased.

Former Secretary of Education, Arne Duncan, wrote an article for The Washington Post called What we can learn from the state of our nation’s education.  He begins by writing:

The latest results from the National Assessment of Educational Progress (NAEP) are prompting some soul-searching about the limited gains over the past decade, but there are outliers worth saluting. More important, we should be analyzing what successful states and school districts are doing differently so that others can learn from them.

He then tells us about some of the ‘bright spots’ which, of course, happen in places like D.C., Tennessee, Louisiana, and Denver — all places that have followed the Race To The Top playbook with charter schools and using value-added to rate teachers.  About Louisiana he says “Louisiana posted nation-leading gains in eighth-grade math” and later credits Chief of Change John White, a former TFAer who has been Louisiana’s State Superintendent of Education since 2012.  He concludes by warning us not to look at the overall lack of improvement as an excuse to rethink the reform agenda he promoted.

The one thing the United States cannot do is use these results as an excuse to go backward to the days when standards and expectations were low. We cannot return to a time when achievement gaps around race and poverty were hidden. We cannot pretend that talent strategies will happen on their own without intentional efforts to recruit, support, retain and hold accountable educators.

I’ve been following Louisiana’s John White for some time and always like to catch him cherry picking data to make it seem like he has helped Louisiana to improve in education.  Debunking this recent falsehood about Louisiana leading the nation in 8th grade NAEP growth was one of the easiest ones to uncover.

Imagine you have a friend who has been on a diet for 6 months.  You ask him how his diet is going and he proudly asserts that he lost 5 pounds in the past month.  But he looks a bit nervous when he is telling you this so you ask the important follow up question:  How much have how lost over the past six months since you started the diet.  He confesses that he hasn’t lost any weight in the six month period and actually gained a pound in that time.  It’s a good thing you knew what sort of follow up question to ask.

So while, yes, Louisiana’s 8th grade math NAEP in 2017 was 267 and their 8th grade math NAEP in 2019 was 272 which was a 5 point gain in that two year period and while that was the highest gain over that two year period for any state, if you go back instead to their scores from 2007, way before their reform effort happened, you will find that in the 12 year period from 2007 to 2019, Louisiana did not lead the nation in 8th grade NAEP gains.  In fact, Louisiana went DOWN from a scale score of 272.39 in 2007 to a scale score of 271.64 in 2019 on that test.  Compared to the rest of the country in that 12 year period.  This means that in that 12 year period, they are 33rd in ‘growth’ (is it even fair to call negative growth ‘growth’?).  The issue was that from 2007 to 2015, Louisiana ranked second to last on ‘growth’ in 8th grade math.  Failing to mention that relevant detail when bragging about your growth from 2017 to 2019 is very sneaky.

Screen Shot 2019-11-03 at 4.55.41 PMScreen Shot 2019-11-03 at 4.56.24 PM

 

Screen Shot 2019-11-03 at 5.15.36 PM

This is just one small concrete example of how reformers will cherry pick data to claim that there are bright spots in this NAEP data that show that we need to continue following the lead of people like Arne Duncan.

I think 2007 is a good benchmark year, in general, to look at how much the country ‘grew’ on NAEP.  For 4th grade reading, the scale score decreased by one point between 2007 and 2019.  For 8th grade reading, the scale score stayed the same from 2007 to 2019.  For 4th grade math, the scale score increased by one point between 2007 and 2019, and for 8th grade math, the scale score increased by one point between 2007 and 2019.

There is a lot more to be said about the NAEP results and how to interpret them.  For example, it is hard to compare growth between two states that had different starting points.  Like when someone is 100 pounds overweight it might be easier to lose 10 pounds while if someone is 10 pounds overweight it might be more difficult.  I’ve seen analysis that higher growth (as measured in points) correlates with lower starting scores, which helps explain the so-called ‘Mississippi Miracle’ reformers are now talking about.

Three of the places that Duncan touts for their ‘gains’: D.C., Louisiana, and Mississippi are three of the lowest scoring NAEP regions.  Why should we be looking to them for things to emulate?

Reformers will always look to cherrypick data that they can twist to make it look like they should continue to have the power to influence education policy.  Usually they have to stretch so far to make their claims that they are fairly easy to uncover.

Posted in Uncategorized | 2 Comments