The Underachievement School District 2015 Edition Part I

Three years ago, the Achievement School District (ASD) in Tennessee began their mission, summarized in the statement on their website under the heading ‘Building the Possible.’

“The Achievement School District was created to catapult the bottom 5% of schools in Tennessee straight to the top 25% in the state.”

The timetable for this goal is just five years from the time the school enters the ASD.  As I wrote to current ASD superintendent, Chris Barbic, who I’ve known for over 20 years from back in the days when we were both TFA teachers in Houston at the same time, this is not a feasible goal.  It isn’t that I don’t think schools or teachers are capable of improvement, I just think that there is a limit to what can be accomplished by only focusing on replacing teachers and giving schools over to charters.

I don’t know if this was an intentional thing or not, but look at the ASD logo.

See that triangle thing?  It is called a Penrose Triangle or sometimes a Penrose Tribar.  It is an optical illusion since something like this cannot be built.  According to Wikipedia, in the 1950’s Penrose, himself, described the object as “impossibility in its purest form.”  In a very candid moment, Barbic admitted during a panel discussion that even if the goal was unattainable, it served a purpose because it created an energy around a lofty goal which attracted funders and talent to the district.

I have been following the progress of the ASD since it started and have been doing a yearly summary, in addition to other posts throughout the year.  The 2013 edition can be found here and the 2014 edition can be found here.  I plan to continue this yearly update until the end of the fifth year in 2017.

At the end of July 2015, Tennessee released the state test score data for all the schools and all the districts.  Every year when the data is released, Tennessee and the ASD are ready with their own spin on the results.

The website Chalkbeat Tennessee titled their article about the data release ‘At critical moment, state run Achievement School District posts big gains at its original schools.’  In the article they had this interesting bar graph supposedly supporting this claim.

So the first thing I noticed is that while math is ‘up’ by about 11% in the past three years, Reading is ‘down’ by about 4%.  And reading had gone up a bit last year, but now its down again.  The Science, I don’t know much about that test so I’m not going to focus on it.  Not that I don’t think science matters, but I’ve never heard a ‘reformer’ talking about anything but math and reading so I’m not going to let science into mix here.

Now reading in the whole state of Tennessee is down 1.5% over the past 3 years, it has actually dropped a little each year since it peaked in 2013.  It is now at the level it was before all the Race To The Top reform happened there, amazingly.  So the ‘big gains’ are the 6 points in math.  But isn’t reading really really important?  What good is a program that increases math scores at the expense of reading scores?  It’s like a drug that gets you to lose weight but also causes Cancer.  The other thing to notice is that these percentages, even the 27% after the ‘big gain’ is way below the state average of 55.6% this year.  And the reading at 13.8% is way way below the state average of 48.4%.

I decided to do a little fact checking on this.  What I found surprised even me.

As the ASD has been around for three years, there are three ‘cohorts’ of schools.  There are the six original schools which have been in the district for all three years.  Then they added eleven more schools the second year, and six more the third year.

For my analysis, I am looking only at the six original ASD schools since I have three years of data for them so they are the most relevant.  The six schools in the original cohort are Brick Church College Prep, Cornerstone Prep — Lester Campus, Corning Achievement Elementary School, Frayser Achievement Elementary School, Humes Preparatory Academy — Upper School, and Westside Achievement Middle School.

Last summer I analyzed the scores from these six schools and determined that two had improved scores, two had about the same scores, and two had worse scores than before the takeover.  It definitely seemed like mixed results to me.  On their ‘growth’ metric the entire district actually got the lowest possible score, a 1 out of 5.

Of the six schools, the one that was doing the best, as of last year, Brick Church, was somewhat different than the other four because they were ‘phase in’ schools meaning that instead of the charter operator taking over the entire school, current students and all, they just took the newest students entering the school.  So these schools were grades 5-8 and the first year they just took the new incoming 5th graders and they would ‘grow’ the school one grade at a time until they eventually took over the whole school.  As these two schools were outperforming the others, it seemed to show that the more disruption the better.  Brick Church had such big gains during the 2013-2014 school year that Barbic declared that it was on target to make it into the top 25% one year ahead of schedule, after just four years!

One year ago Barbic said in an interview that three of the first six schools were on track to make it into the top 25% in five years.

Here’s a quote from that article:

The special statewide district is taking over the lowest-performing schools in the state with a goal of moving them into the top 25 percent in just five years. Now in year three, superintendent Chris Barbic says he’s encouraged.

“You know, when we first talked about this, this was a goal that folks thought was completely crazy. And I think we’re learning is that not only is it not crazy, but we’ve got three of our first six schools that are on track to do it.”

One of those three schools on the right trajectory is Brick Church Pike College Prep in Nashville, which is slowly being converted into a charter school run by LEAD Academy. Barbic says if Brick Church matches this year’s student growth in math and reading, it would leap into the top quartile a year early.

He made this prediction based on the assumption that Brick Church’s 17 point gain in math and 24 point gain in reading was not a statistical outlier but something that could be repeated for the next two years.  Of course Brick Church was the first school I checked their 2014-2015 scores for and they had huge drops (negative gains they call them!) of around 14 points for math and 17 points for reading, taking them way off the trajectory Barbic had predicted.

Looking over the ASD scores for reading and math I noticed that all six of the original schools had gone down in reading while half of them had gone down in math.  The one bright spot in the original cohort was Frayser’s math scores going from 14.6% to 37.6%, a gain of 23 percentage points.  But looking at Frayser’s reading scores they had gone down 1.3% to a minuscule 7.6% proficient, one of the lowest scores in the state.  I don’t know how they got their math scores up so much while having nearly no students pass reading.  Regardless, I’m not sure that Frayser should be throwing any sort of victory party.

The question that nobody seems to be asking is the most obvious one:  At what percentile are each of these six schools now that they have been part of the ASD for three years?  Originally they were in the bottom 5% in order to be eligible for the ASD.  The mission of the ASD is to get them up to the top 25% in 5 years (though Barbic recently said that it could actually be 6 or 7).  We hear about how Frayser’s math scores went up by 23 points this year and that ASD itself ‘outgrew’ the rest of the state in math by increasing it by 6 points.  We also hear that schools that have been in the ASD for two or three years got the highest possible ‘growth’ score this year, which is a 5.  But they never answer the simple question:  At what percentile are the original six schools at now?

Of course there are different ways to assign a single numerical grade to a school in order to rank them and see what percentile each school is at.  I devised the most simple metric possible and which I think the most data-driven ‘reformers’ would approve of.  Just add together the percent proficient in math to the percent proficient in reading to get a score that has a minimum of 0 and a maximum of 200. If a school had 80% passing math and 70% passing reading, they have a total score of 150, for example.  Yes, there’s a lot more to schools than just these two numbers, but I wanted to keep things simple and inline with the sorts of calculations that ‘reformers,’ themselves, like to use.

By this metric the top performing ASD school from the first cohort was Coming with a score of 48.6 followed by Brick Church (47.9), Frayser (45.2), Westside (42.1), Cornerstone (37.6), and Hume (33.1).  To check where these scores ranked compared to all the Tennessee schools, I calculated this metric for all 1358 schools that had 3-8 math and reading and sorted them from high to low.

Here are the results:

School Score Percentile
Hume 33.1 1.5%
Cornerstone 37.6 2.6%
Westside 42.1 3.2%
Frayser 45.2 4.1%
Brick Church 47.9 5.2%
Corning 48.6 5.5%

As you can see, four of the original six schools are still in the bottom 5% while the other two have now ‘catapulted’ to the bottom 6%.  Perhaps this is one reason that Chris Barbic recently announced he is resigning at the end of the year.

Throughout the country, there are states that are considering creating their own ASD based on the supposed success of this one and the Recovery School District in Louisiana, on which this one is based.  Senate Democrats actually tried, and failed, to get an amendment into the reauthorization of the ESEA that would mandate that the bottom 5% of schools in each state become an ASD, essentially.  I hope that my very simple calculations are compelling evidence that the ASD does not live up to the hype.  Getting two out of six schools from the bottom 5% to the bottom 6% has not earned them the right to replicate around the country.

If you want to crunch the numbers yourself to verify my results or to find some of your own, here is my excel file I adapted from the publicly released state data.  And here is the raw data from the state.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

30 Responses to The Underachievement School District 2015 Edition Part I

  1. Staubitz, John says:

    Fantastic post! I’m going to be forwarding this to several people right now. Well done, Gary. Your timing here is perfect. Yesterday on Nashville Public Radio they were referencing Tennessee’s academic gains and highlighted the ASD in particular without referencing any data that made sense.

    Sent from my Verizon Wireless 4G LTE DROID

    Gary Rubinstein’s Blog wrote:

    garyrubinstein posted: “Three years ago, the Achievement School District (ASD) in Tennessee began their mission, summarized in the statement on their website under the heading ‘Building the Possible.’ “The Achievement School District was created to catapult the bottom 5% of s”

  2. Meghank says:

    Thank you so much for your attention to Memphis. Check your information on Corning Achievement Elementary. It’s a whole-school takeover, I know because one of the best, hardest-working teachers I have ever known was let go from Corning in the takeover and came to work with me.

  3. Pingback: Gary Rubinstein Reviews the Failure of the Tennessee Achievement School District | Diane Ravitch's blog

  4. Pingback: ASD Analysis Part 3: Who Is Out of the Bottom 5 Percent? - Bluff City Education

  5. Pingback: Tennessee Education Report | That’s Not That Much, Really

  6. Joe Hartley says:

    Do you know whether the Tennessee test is largely plug-and-chug problems? If so, you could probably get decent mileage at an elementary school and low middle school level with the drill-and-kill tactics that the reformers prefer, without seeing a corresponding rise in the reading skills. Like you, I’m alarmed to see no improvement in reading, as that’s the major problem holding back my urban high-school students. It would be interesting to track the students into high-school math to see how they do there….particularly in algebra 1 and algebra 2.

  7. Pingback: Links 8/1/15 | Mike the Mad Biologist

  8. lbumgarner says:

    The success in New Orleans should be noted when anyone starts talking about Charter Schools.

    And if one needs an example http://www.tindleyschool.org/ is the best. This is a school started by the family of Charles Tindley, (He wrote, We Shall Over Come You). The motto is College Or Die. And the percentage of kids that go to Ivy League Colleges is astounding.

    It is located in Cincinnati, a location where good news is needed the most.

    • The success in New Orleans? Did you know that the NO all-charter school district is in the lowest 6% in Lousiana?

      Here’s an article on NO charters. https://deutsch29.wordpress.com/2015/08/28/louisiana-charters-are-by-far-the-worst-according-to-2011-8th-grade-naep-anaysis/

      Nobody said, all charters are bad.

      • lbumgarner says:

        New Orleans Charter Schools are a great success, have you looked at what the New Orleans system looked like before the Charters came about?

      • I love your strong arguments Very convincing: “You are incorrect, New Orleans is a success”. Teaching is not about raising scores, but if a new system is implemented to raise scores then it’s legitimate to ask, what happened with the promised land? ACT average 15? How has that changed anybody’s life?

        Similarly, ASD was introduced with the promise that in 5 years the schools consumed by ASD will get to the top 25%. So now, tens of millions of $ later, causing suffering and grief to thousands of people, kids, they dare to argue over 1-2% test score increase compared to what we saw before 2011?

        In any case, no civilized country would indiscriminately fire three thousand teachers just because they think they have a better idea how to teach. This just tells you about our laws, and, ultimately, the Constitution that allows them.

      • lbumgarner says:

        So the fact they have not achieved the hopeful goals they stated shows they are not doing their job.

        So if they have only done an average job or even a little better than the system before, then the fact Charter Schools get only 70 cents on the dollar over the regular schools, they do not get free buildings, free buses and a lot of other things, tells us they are a real bargain in this, no real achievement in our education system.

        http://www.cato.org/blog/addressing-critics-purportedly-no-good-very-bad-chart?utm_content=bufferd20a4&utm_medium=social&utm_source=facebook.com&utm_campaign=buffer

      • Now you switched to a different topic. In the article you just linked to I do not see RSD or ASD mentioned.

        But ok, let’s relate what you say back to ASD: According to test results, which is their own assessment criteria, ASD is not doing an average job, they are performing in the lowest 5%. Average performance is 50%.

        What ASD did was, it fired hundreds of teachers, ate up $93 million just last year alone, and improved nothing noticeable even on their own scale.

        As I said, their assessment methods, which agrees with the methods used all over the country by the School Take Over Movement People, have nothing to do with finding out how well teachers are teaching or how well kids are learning.

        On any reasonable human and educational scale, what the ASD and RSD did were criminal.

      • JIM says:

        “…have you looked at what the New Orleans system looked like before the Charters came about?”

        Have you looked at how the new population pre-Katrina changed? That has ramifications on any outcomes.

  9. Pingback: Collaboration not Competition | Dad Gone Wild

  10. Pingback: Plan for charter takeover of worst public schools crafted in secret : NC SPIN Balanced Debate for the Old North State

  11. Pingback: Education turnaround teams successful, yet lawmakers push for takeover : NC SPIN Balanced Debate for the Old North State

  12. Pingback: Andy Spears: Is Tennessee Sick of the (Low) Achievement School District? | Diane Ravitch's blog

  13. Pingback: What does the future hold for Nashville and the Achievement School District? | LWVeducation

  14. Pingback: The Waltons Promote the (Failing) Tennessee ASD | deutsch29

  15. Pingback: The New More ‘Transparent’ ASD | Gary Rubinstein's Blog

  16. Pingback: Tennessee Education Report | So, About the ASD

  17. Pingback: Charlotte, N.C. Editorial: Yes, Let’s Copy a Failed Experiment | Diane Ravitch's blog

  18. Pingback: A Subversive Guide To The Saturday Sessions | tfa25factcheck

  19. Pingback: #TFA25FactCheck: Examining TFA’s Facade Cracks in Real Time | deutsch29

  20. Pingback: Drastic Public School Cuts in Memphis—The New America

  21. Pingback: The Underachievement School District 2015 Edition Part II | Gary Rubinstein's Blog

  22. Pingback: Tennessee Education Report | A Friendly Reminder

  23. Pingback: The Ever Under-Achieving Achievement District « Dad Gone Wild

  24. Pingback: Teaching, a love story: PART 4 (the final chapter) – MY TWO CENTS

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s