If you’re a parent, a student, or a teacher, the last thing you want to hear is that the state you live in has been commended by Secretary of Education, Arne Duncan, as a ‘reform’ leader. This has been occurring recently in the state of Tennessee.
The state education commissioner of Tennessee is former TFA VP of public affairs, and long time casual acquaintance of mine from the time were were in TFA together in Houston in the early 1990s, Kevin Huffman. Huffman is also a member of the ‘Chiefs For Change,’ a group of education ‘leaders’ who blindly support the corporate reform agenda.
One experiment going in in Tennessee is the creation of The Achievement School District (ASD) which has on its website the slogan “We are the ASD: Proving the Possible by moving the bottom 5% of schools in Tennessee to the top 25% within five years.” Notice that the way this is phrased, it implies that they are having success. They are not “trying to prove” but actually “proving” that they have the secret to doing what nobody has ever done, or even come close to doing.
The superintendent of The Achievement District is someone who is more than a long time casual acquaintance from my Houston days, but a bona fide friend from those days. We share an unusual bond as in his first year he taught the sixth graders who had been left back that I had taught the previous year, my first year. Though he wasn’t one of my best friends, I did hang out with him a good amount, from what I remember. I like Chris, and he actually still answers my tweets on twitter, which is more than I can say for some other ‘reformers.’ When I saw he was chosen for this job, it gave me some hope since I have known him for over twenty years and believe him to be a ‘stand up’ guy. Still, I can’t help thinking that in order to be among the group he is associated with, he must have changed a lot in the past twenty years, I don’t know.
So the ASD had a very ambitious goal, though vaguely stated. I think they are implying that the ASD schools will, four years from now, have proficiency levels in the top 25% of the state. Their strategies, though, are not very encouraging to me. Shutting down public schools and turning them over to charter chains and staffing these schools with new TFAers, I don’t really see how that has any chance of working unless they redefine what ‘success’ is in this context.
In November they did some kind of pre-test, which were not showing encouraging results. Of course this was just a few months after the experiment began so this is not surprising. Then, at the end of the school year the students took the state tests and the results came back at the beginning of August. On the ASD’s website, they have a press release titled ‘ASD Schools Make Progress In Y1’. In this we learn that, amazingly, the district earned a score of 5 out of 5 on the states ‘growth rating.’ Holy moly guacamole. That’s good. At least I think it’s good. Definitely sounds good to have the maximum amount of growth, right?
But I’ve seen other districts, most notably the New Orleans ‘Recovery School District’ (RSD) the model on which the ASD is based, invent statistics that make them look like they are making progress when they actually are not. In the ASD press release, there are more details, including this graph comparing last years results to this years.
So they went up by 7.7% in science from 16.5% to 24.2%, up by 3.3% in math from 16.3% to 19.6%, and — wait, this can’t be right, can it? — DOWN by 4.5% in reading from 18.1% to 13.6%. So when we look at the average proficiency in Math and Reading (what reformer really cares about science unless it can be used to make it look like what they’re doing is working) has gotten lower. (‘Dipped’ is the word they used in the press release to describe what happened in reading. Surely they had to get the thesaurus going to get that carefully selected word.)
So here’s the big question: In what world can a school that had their reading proficiency drop by 4.5% and math increase by 3.3% can a school district get the highest possible ‘growth score,’ a five out of a possible five? I’m sorry, Chris, if you are reading this — and don’t take this personally, it’s just like us arguing about why your Atlanta Braves aren’t as good as my New York Yankees –, if the only thing that you can celebrate about your first year’s progress is that you did well on a ‘growth metric’ that is so obviously flawed, it is in need of some serious independent investigation, then you have nothing to celebrate.
What you should have said, and I’m not saying that you want my advice, is that these scores are very disappointing, but you never expected the scores to go up after just one year. Rome wasn’t built in a day, and the turnaround of the schools in Memphis wasn’t built in a year. In fact, you could have said, you would expect the scores to go down at first, just like you might expect your golf score to get worse after your learn a better swing in golf lessons, but before the change gets a chance to sink in and become natural. That’s what you should have said, and that would have bought you, at least, another year. But to pretend that you are happy about your level 5 ‘growth score’ is disingenuous.
As I said in my open letter to Chris a few months ago, which he still hasn’t answered — yes I know my letter was great and it would take a lot of work to write an equally eloquent response, but can’t you get someone on your staff to do it? — the only way that you are going to get the bottom 5% of schools up to the top 25% of schools with a completely in-school strategy (nothing about massive investment in wrap-around services for those pesky out-of-school factors) is if you bend the rules or find a way to bend the truth when reporting the results.
If I seem like I’m being hard on my old friend, it is just that I am hopeful that he is an honest enough guy to be the first Broad trained major ‘reformer’ to ‘break rank’ in true Ravitch style. Me and the others who think that corporate reform is making schools worse and hurting kids could really use a guy like Chris on our side.
Gary Rubinstein's Blog 
He can’t really admit that he expected scores to dip the first year, since teachers at his schools get fired if they make a 1 or a 2 (on a scale from 1 to 5) on their evaluation scores the first year.
By the way, the ASD made a 5 on everything except Literacy, where it made a 1. I don’t see how a 5 and a 1 average out for an overall score of 5, do you?
Also, most of the ASD schools made less than a 5 in Numeracy, so I don’t see how the district got a 5 in Numeracy. 4, 2, 5, 3, 3, 5 are the Numeracy Evaluation Composites for each school in the district.
If they are going to close schools
based on these ridiculous numbers, they shouldn’t be making things up to make themselves look good after they take over.
Report: District-Level Evaluation Composites District: Achievement School District
Year: 2013 Test: TCAP/EOC/SAT 10
2012-2013 Composite Trends
Composite Type One-Year Trend*
Index Level
Overall 4.56 5
Literacy -6.51 1
Numeracy 9.35 5
Literacy and Numeracy 2.55 5
My guess is that they did some kind of percent change thing which is biased toward districts with low benchmark. So 15% to 18% is a 3% increase but that is a 20% gain. That is the only way I could conceive of any district having such a small gain get rewarded that way.
I suggest you look at the complete set of data. It’s the attached xcl file under School-Level data.
http://www.tn.gov/education/tcap/index.shtml
It shows quite a bit. Some schools are doing quite poorly, prompting the question of what is to be done when schools that have been taken over continue to fail. Which I asked Mr. Barbic at an open house, the answers were lacking: “reevaluate,” “resturctures,” etc.
You should also compare the scores to izone, which I am in the middle of. http://www.mcsk12.net/schools/chickasaw.js/site/izone.shtml
The similarities are many: increased money, autonomy, extra insturctional time; but the difference is that it’s managed and operated by veterans of the school system, don’t have to resign their position, and can retain their tenure.
If izone succeeds where ASD fails, then we’re on to something.
The i-Zone story is very interesting. And, frankly, I’ve been surprised that Shelby County Schools has not made more of their scores. They are obviously doing something right.
We’ll find out in four years what kind of scruples your friend still has. But, being a Broad Institute graduate, it is likely that he has few left.
“…it is just that I am hopeful that he is an honest enough guy to be the first Broad trained major ‘reformer’ to ‘break rank’ in true Ravitch style.”
That would be astounding, but not likely to happen, as he’s making big $$$ and his next Broad Institute-pandered job will be lined up before he leaves ASD.
Pingback: The Underachievement School District | Gary Rubinstein's Blog ← NPE News Briefs
Gary, it’s a 5 because Science/Math/English is combined. The average is a 5 in growth. Obviously from test prep since the reading scores went down.
Some insight into how data in TN works…
(1) TVAAS is the value-added system used (e.g., the growth measure referred to). You can access the public site here: https://tvaas.sas.com/public.html?as=t
(2) Level 5 on TVAAS is defined as an index of <+2.0. The ASD scored a 5 overall because the numeracy scores (+9.35) were high enough to counter the low literacy scores (-6.51).
(3) The reason you can have high levels of growth without much movement, and even negative movement, on achievement scores is that kids who enter extremely low will likely take multiple years to get to proficiency. For example, if you enter at the low end of Below Basic, you can grow a lot just by getting to the high end of BB or by jumping up to Basic. Since the ASD is serving only the lowest performing schools in the state, one should not expect proficiency scores to soar in year one – although one should not expect scores to decrease either.
(4) Year two is key for turnarounds. Oftentimes year one is a culture-setting year. But all the data say that, if a turnaround is going to be successful, substantial growth will at least start to be experienced by year two.
(5) It should be noted that the state average for reading scores for grades 4-8 was just a +0.4 point gain. Both Memphis and Nashville had negative gains, as did about half the state. I'm not sure what's going on there, as hardly anyone had negative gains in Math. I heard they might have tinkered with the test.
Love it!!!!!
I’ve thought a lot about this blog post over the past few days and waffled back and forth on whether or not to respond and engage. I have never been the kind of person who avoided having an important and necessary conversation about what WE need to do to support our students and prepare them for a life that they choose.
I’ll begin with some background for full disclosure sake. I was a Teach For America corps member placed in NOLA prior to Katrina. After the storm, I joined the team at YES Prep in Houston, first teaching and working as a Grade Level Chair and then taking charge as the recruitment and selection person, trying to attract the best teachers for our students. After a short time at Rocketship in San Jose as a school leader, I was compelled by the opportunities in Tennessee to get things right and joined the team there. I currently work to support our middle and high school principals, working directly with our teachers, students and school leaders every day.
Having worked with Barbic for nearly a decade, I know that he is deeply committed to the success of our students. He cares tremendously about this work and grapples every day with solving our most important problem in this country. I admire his leadership and appreciate him taking responsibility for the hard work that needs to be done to help our students, families and communities. They deserve the best we can give them.
1 – our slogan. If you talk to anyone in the organization, they’ll be completely honest with you. We haven’t proven anything yet. We have our theories about change and we are working incredibly hard to prove that this is possible. Our slogan is aspirational and hopeful, and if our theories about change are right, then it will have positive outcomes for our students in Memphis. We have been humble from the start and get humbled every day in our work. This is tough. No one will pretend otherwise. Our students bring serious issues with them to school, but our teachers are committed to pushing our students despite those circumstances.
2 – fact checking. blogs are tough huh? they aren’t places for credible journalist driven by ethical guidelines. Reporter have to be diligent and careful in what they say. First, we are moving our school into the top 25% in the state, not the top 5%. Secondly, Teach For America teachers are a pretty small percentage of our teaching force. These are important distinctions to make.
3 – state ranking. We don’t choose our ranking, Gary. We’re glad that we are among the Level 5 schools in the state which is a testament to the hard work of our students and teachers last year, but for you to suggest that we are high fiving in the hallways is inaccurate. We took the necessary time to be frustrated and disappointed in the results we had last year with reading. We applauded those teachers who made gains with their students. We figured out the changes we needed to make and got back to work serving our students. If you talk to anyone in the organization, I don’t think you will hear one person claim that they are “happy” about our results. We are fully aware that we have a lot of work to do.
It may be helpful to actually explore your ideas with the folks living this work in the future prior to posting blogs. I think it will enlighten your opinions and help the credibility of your words. Today alone, we have 200 teachers and staff members in the Frayser neighborhood of North Memphis busting their rear ends so that students like Maliko, Charles and Diamond have a chance to make the kinds of decisions for their futures like we had, instead of living a future that was decided for them. Frayser is a neighborhood that leads the country in negative economic factors. Eleven of the 14 schools in the neighborhood are performing among the bottom 5% in the state. Average ACT sores of the high school in the neighborhood is a 15.7. Sitting around is not an option. We simply have to do something.
I might also provide some additional advice: when thinking through your ideas, it is always helpful to provide some concrete solutions. Unless I missed them, I didn’t see anything in your post that suggested what we could do better. This year, in our Frayser schools, increasing our proficiency in literacy is a key strategic goal. We have pumped up our staff to support this and continue to support our teachers in their work to boost reading proficiency among our students. We are open to your advice and suggestions on how we might continue getting better.
Am happy to be a resource for you next time you write and assure you that we are working our tails off for our students. I am honored to work for our communities and humbled by the task we have ahead of us.
My suggestion is to improve the libraries. You responded to a comment of mine saying it had been decided that the libraries were going to be improved, but I’d love to see evidence of that or hear something more about it.
I don’t doubt that you and your colleagues are working hard for your students; I just doubt that you are working harder than the teachers you displaced did. And I have done some fact-checking of my own and know that you have large percentages of TFA teachers. So I KNOW that while they may be working as hard as the teachers these students had, they do not have the experience to teach difficult subjects like reading as well as those more experienced teachers could.
I also believe that the instability you caused in these children’s lives by removing their teachers is yet another obstacle these children have to overcome.
Your solution to everything is to improve the libraries? Thank you Meghank, your thoughtful consideration to these problems is appreciated. You’ve truly hit the nail on the head and found the crux of the problem.
Typical bloggers, long on critiques, completely lacking in any substantive solutions.
Yes, greater access to books is a major step in solving the problems these children have with reading.
Why is that so hard for you to understand?
Hi Gary,
I have a quick question (since you frequently ask them of people, some of whom are working 12-15 hours a day at those ‘bad’ schools that you offered several excuses for leaving long ago). Why don’t you take all this insight and energy, to expose the ‘corporate reform’ apparently taking over our public schools, and devote it to teaching in a high-need school (30% frl isn’t that, and you know it) or helming one, since you’re an award winning teacher? We have the edushysters and ravitches of the world (a strange hero for you, since she’s never taught and conveniently became critical of the very establishment she helped build once she began to lose prominence in it), and really any disgruntled teacher–credentials aren’t really prized in the blog world you inhabit, as much as ad hominem attacks, straw men, and selective data citing to weave conspiracy theories. For those of us working in schools that really need help–urban schools around the nation have huge shortages of math and science teachers, and I hate to break it to you, but several states don’t have the traditional candidates to fill them, which is where imperfect programs like tfa come in.
You seem like you do care, even if you have a complete double standard for yourself and those you consider on the right side of issues (actual experience in the urban school system where upwards of 70% of students are frl). Is this a productive use of your energy, which can’t be summoned to teach in a ‘bad’ school where your brilliance would be invaluable?
Jeremy and Cary-
You’re coming into a conversation that Gary has been having for several years and not seeing the context of his post. Please go back and read things from the start.
It’s not fun to feel that someone is casting (seemingly out of left field) aspersions at one’s efforts, so go back and read Gary’s work for your own sake.
I haven’t investigated the details of this particular post, and I don’t know Gary and can’t speak for him. But I know a lot of caring people (including those criticizing Gary in these most recent comments) want to improve things.
The problem that I think Gary is trying to bring light to are the apparent shifts in truth that are happening with many education reformers. Remember, a lot of education reformers are arguing that they need to disrupt the slow and terrible education status quo. And a lot of the reformers need funding to do this, so this creates a certain incentive for them to try and prove they’re doing better. This makes sense, businesses must prove that they are doing better in order to get more customers. Their product must show this. The thought, is that this will put pressure on schools and systems to improve.
So the question to me is, are the disruptions better? I think innovation can be useful, and a change can be good. But stories like this one, which seem to be published more and more:
http://www.nydailynews.com/new-york/education/success-academy-fire-parents-fight-disciplinary-policy-article-1.1438753
scare me. I ask myself, are the reformers now doing more harm than good? And why aren’t they more honest about what they are / are not doing? Sometimes I wonder if it’s cause it’ll hurt their funding.
http://thereisabookforthat.com/tag/self-esteem/
Hi Gary, I regularly read this blog. I found this, noodling around, and was happy to find something about your other work. I’m a retired Elementary school librarian, spending way too much time reading the depressing news about Ed deform.
It is excessively naïve to think that a TFA alum who became a Broad-trained School Leader is ever going to be on the–morally, academically–correct side. Sorry. This game is getting old, and children are suffering severely. Parents are suffering hugely. Teachers are losing meaningful jobs to TFA and Turnaround Schools are bleeding staff, administrators; and any families that can, take the leap. Actually, I fought having TFA forced on our school district–your research helped me, but the stars of reform were aligned for the State of Connecticut to take over. Now one of my children actually has a TFA, and after five days, it is worse than I imagined. I would like to contact you, Mr.Rubinstein, to share some information–I would think it might shock even you.
Would people stop comparing well-executed business strategy to teaching children. This false analogy is nauseating. We make the assumption that businesses are better run than government agencies. This isn’t always the case. There are horribly run companies, inept gov’t agencies, terribly run schools, churches, and other institutions. Just because one uses business jargon like portfolios and data-driven results does not mean they are good business people or school leaders or even know what they’re doing. From my experience, anyone who subscribes to this type of thinking in education does not have a clue about how to educate children.
Stop using buzz words, ridiculous acronyms, ranking schools, manipulating data like Enron, and start teaching kids. Public schools should not be competing with each other in the first place. They are public goods and community institutions. There should be no winners or losers.
I appreciate you Gary- and I understand completely why you use your energy blogging-it is well spent. Thanks!
Pingback: Underachievement School District 2014 Edition | Gary Rubinstein's Blog
Pingback: Frayser 9GA, the miracle school of the Achievement School District | Gary Rubinstein's Blog
Pingback: The Underachievement School District 2015 Edition Part I | Gary Rubinstein's Blog