Success Academy Class Of 2020 Sheds 239 Scholars Along The Way

The New York Post recently ran an editorial about the SAT scores of the Success Academy senior class of 2020.  Of all the different numbers they referenced, one that I took note of was 114 — the apparent number of students in the senior class.

The class of 2020 is the third graduating class of Success Academy.  The class of 2018 had 17 seniors out of a cohort of 73 first graders in 2006-2007.  The class of 2019 had 26 seniors out of a cohort of 83 kindergartners in 2006-2007.  Some of the class of 2019 were students who had been held back from the class of 2018 — probably in a comparable number to the number of 2019 students who will graduate this year.  So the 26 out of 83, or 31% persistence rate probably accounts for students who take an extra year to graduate.

For the class of 2020, things get a bit more complicated since in 2008 Success Academy did its first expansion and grew from one school, now called Harlem 1, into four schools now including Harlem 2, Harlem 3, and Harlem 4.  Some of the past records are incomplete for these schools, but when the 2020 cohort was in 2nd grade in 2009-2010, I find that there was a combined 353 students in the cohort.  By 6th grade, they were down to 263 students and by 9th grade it was 191.  In 10th grade they were 161 students and in 11th grade, 146.  And now, according to the New York Post article based on a Success Academy press release, they have 114 seniors.  So only 32% of the students who were there in second grade made it through their program.  And even more startling is that of the 191 9th graders that had been at Success Academy for 10 years, only 59% of them are on track to graduate three years later.

SA2020

[all data gotten from https://data.nysed.gov/ ]

This 32% persistence rate doesn’t even include the students who ‘backfilled’ some of the empty spots for students who have left over the years.  Without access to more granular data, this isn’t something I can study right now.  From Robert Pondiscio’s book about Success Academy we learn that the backfill process is somewhat corrupt.  Families that get off the wait list to backfill vacated spots are sometimes told that if they come to Success Academy their children will have to redo the grade they had just passed in their other school.  Surely many of these families choose to forfeit their place off the waiting list and, in that way, Success Academy makes sure that the backfill students are generally the higher performing students which serve to inflate the school’s test scores.

According to Success Academy, the demographic data for their students are:  74% receive free or reduced-price lunch, 16% have disabilities, and 8% are English language learners.  But the most recent data (From 2017-2018) about the 2020 cohort from the New York State public site is that 66% are economically disadvantaged, 11% are students with disabilities, and 0% English language learners.

Success Academy and The New York Post love to claim that Mayor de Blasio is out to get Success Academy.  I really don’t think so because de Blasio has access to the type of data that could so easily expose the various ways that Success Academy tips the scale in their favor.  Just using publicly available data, I’ve been able to uncover so much about their massive attrition rate.  Imagine how much can be learned from the data that the New York City DOE can access.

Posted in Uncategorized | 7 Comments

Does Louisiana Really Lead The Nation In 8th Grade Math Gains?

The latest National Assessment of Educational Progress (NAEP) scores for the 2019 tests were released on October 30th.  Unlike state tests for which the cutoff scores can be manipulated for political purposes, the NAEP does seem to be somewhat unbiased.  So the NAEP, sometimes called ‘The Nation’s Report Card’ does offer an interesting amount of data that I believe is worthy of analysis.

Often the NAEP results are, intentionally or unintentionally, interpreted to see if it is possible to find some kind of correlation between the education policies a state has enacted and the corresponding NAEP results.  In Obama’s 2014 State Of The Union address, he mentioned that D.C. and Tennessee were improving — as evidenced surely by their NAEP gains from 2011 to 2013 — to show that his Race To The Top recommendations, which were followed closely by those two regions, were working.

So when the 2019 results came out the other day, things looked bad for the reformers.  From 2017 to 2019, the average scale score for 4th grade reading was down 1 point, 8th grade reading was down 3 points, 4th grade math was up 1 point, and 8th grade math was down 1 point.  Though it is not clear to the public whether or not one ‘point’ is a lot or a little, everyone can agree that it is better if the scores go up rather than down.

Only one region had an increased score in 4th grade reading while 17 states decreased.  Only one region increased in 8th grade reading while 31 states decreased.  For 4th grade math, 9 states increased while 3 decreased, and for 8th grade math 3 states increased while 6 states decreased.

Former Secretary of Education, Arne Duncan, wrote an article for The Washington Post called What we can learn from the state of our nation’s education.  He begins by writing:

The latest results from the National Assessment of Educational Progress (NAEP) are prompting some soul-searching about the limited gains over the past decade, but there are outliers worth saluting. More important, we should be analyzing what successful states and school districts are doing differently so that others can learn from them.

He then tells us about some of the ‘bright spots’ which, of course, happen in places like D.C., Tennessee, Louisiana, and Denver — all places that have followed the Race To The Top playbook with charter schools and using value-added to rate teachers.  About Louisiana he says “Louisiana posted nation-leading gains in eighth-grade math” and later credits Chief of Change John White, a former TFAer who has been Louisiana’s State Superintendent of Education since 2012.  He concludes by warning us not to look at the overall lack of improvement as an excuse to rethink the reform agenda he promoted.

The one thing the United States cannot do is use these results as an excuse to go backward to the days when standards and expectations were low. We cannot return to a time when achievement gaps around race and poverty were hidden. We cannot pretend that talent strategies will happen on their own without intentional efforts to recruit, support, retain and hold accountable educators.

I’ve been following Louisiana’s John White for some time and always like to catch him cherry picking data to make it seem like he has helped Louisiana to improve in education.  Debunking this recent falsehood about Louisiana leading the nation in 8th grade NAEP growth was one of the easiest ones to uncover.

Imagine you have a friend who has been on a diet for 6 months.  You ask him how his diet is going and he proudly asserts that he lost 5 pounds in the past month.  But he looks a bit nervous when he is telling you this so you ask the important follow up question:  How much have how lost over the past six months since you started the diet.  He confesses that he hasn’t lost any weight in the six month period and actually gained a pound in that time.  It’s a good thing you knew what sort of follow up question to ask.

So while, yes, Louisiana’s 8th grade math NAEP in 2017 was 267 and their 8th grade math NAEP in 2019 was 272 which was a 5 point gain in that two year period and while that was the highest gain over that two year period for any state, if you go back instead to their scores from 2007, way before their reform effort happened, you will find that in the 12 year period from 2007 to 2019, Louisiana did not lead the nation in 8th grade NAEP gains.  In fact, Louisiana went DOWN from a scale score of 272.39 in 2007 to a scale score of 271.64 in 2019 on that test.  Compared to the rest of the country in that 12 year period.  This means that in that 12 year period, they are 33rd in ‘growth’ (is it even fair to call negative growth ‘growth’?).  The issue was that from 2007 to 2015, Louisiana ranked second to last on ‘growth’ in 8th grade math.  Failing to mention that relevant detail when bragging about your growth from 2017 to 2019 is very sneaky.

Screen Shot 2019-11-03 at 4.55.41 PMScreen Shot 2019-11-03 at 4.56.24 PM

 

Screen Shot 2019-11-03 at 5.15.36 PM

This is just one small concrete example of how reformers will cherry pick data to claim that there are bright spots in this NAEP data that show that we need to continue following the lead of people like Arne Duncan.

I think 2007 is a good benchmark year, in general, to look at how much the country ‘grew’ on NAEP.  For 4th grade reading, the scale score decreased by one point between 2007 and 2019.  For 8th grade reading, the scale score stayed the same from 2007 to 2019.  For 4th grade math, the scale score increased by one point between 2007 and 2019, and for 8th grade math, the scale score increased by one point between 2007 and 2019.

There is a lot more to be said about the NAEP results and how to interpret them.  For example, it is hard to compare growth between two states that had different starting points.  Like when someone is 100 pounds overweight it might be easier to lose 10 pounds while if someone is 10 pounds overweight it might be more difficult.  I’ve seen analysis that higher growth (as measured in points) correlates with lower starting scores, which helps explain the so-called ‘Mississippi Miracle’ reformers are now talking about.

Three of the places that Duncan touts for their ‘gains’: D.C., Louisiana, and Mississippi are three of the lowest scoring NAEP regions.  Why should we be looking to them for things to emulate?

Reformers will always look to cherrypick data that they can twist to make it look like they should continue to have the power to influence education policy.  Usually they have to stretch so far to make their claims that they are fairly easy to uncover.

Posted in Uncategorized | 2 Comments

Nevada ‘Abolishes’ Its Achievement School District

The ‘portfolio’ model of education reform was pioneered in New Orleans after Hurricane Katrina and the creation of the Recovery School District (RSD) in 2005.  Then Tennessee, fueled by Race To The Top money, brought in TFA alum and Michelle Rhee ex-husband Kevin Huffman who hired TFA alum and YES prep founder Chris Barbic to create a similar thing called the Achievement School District (ASD) in 2012.

Even though the Tennessee ASD has been a complete failure by every metric possible (unless you count Huffman and Barbic getting high paying consultant jobs after resigning in shame as one of your metrics), other states around the country have followed their ‘lead’ and proposed or created ASDs of their own.  There was one created in Michigan, North Carolina, and Nevada and others proposed in Georgia and Pennsylvania.

Not surprisingly, the ASDs have all been utter failures.  And since it is hard to keep such failure a secret, states are starting to wake up to this fraud.  So it is somewhat good news that another ASD has bitten the dust, this time in Nevada.

On June 3, 2019 Nevada passed SB321 abolishing their ASD.  The official description is that it is “An act relating to education; abolishing the achievement school district; requiring an existing achievement charter school to convert to a charter school under the sponsorship of the State Public Charter School Authority or cease operations; and providing other matters properly relating thereto.”  So even though their ASD is officially ‘abolished,’ it is not like those schools are converted back to public schools.  Instead they are now under the State Public Charter School Authority which, I’d guess, has even less accountability than there was under the ASD.

But my sense is the fewer ASDs the better.  The fewer there are, the lower the chance that some of them will lie about their success and that those lies convince other states to replicate the model.

As far as the Tennessee ASD, it has been the biggest trainwreck in education reform history.  But now even the Tennessee media which has been really slow to realize the scam that was going on  for so many years is finally writing about this.  Here is a recent article from The Tennessean exposing the failures of the ASD.

The new commissioner of education of Tennessee is a TFA alum, much like Kevin Huffman was.  She has recently announced a seven city ‘listening tour’ where they will get community input on what should be done next.  When the Tennessee ASD started, the plan was to ramp it up to about 100 schools.  They started with six schools and grew to over 30 schools in the first four years.  But they have stalled at that number and are not taking on any new schools this year.

On the latest available Tennessee state data, the ASD had not just the lowest test scores , an 11.6% success rate, but also the lowest growth score of a 1 out of 5 on their Tennessee growth measure invented in Tennessee by William Sanders on which most other growth measures are based.

Screen Shot 2019-10-05 at 11.03.07 PMScreen Shot 2019-10-05 at 11.04.22 PM

Things have changed a lot in the past four years.  It was just a little over four years ago that Michael Petrilli moderated a panel called ‘Turnaround Districts:  Lessons from Louisiana, Tennessee, and Michigan.”  Since that time the Louisiana RSD was mixed with the other New Orleans schools, most likely so they can make it more difficult for the public to analyze the data from the original RSD schools.  The Michigan ‘EAA’ was dissolved a few years ago, and the Tennessee ASD is fighting for its life.  For sure this panel discussion and the fanfare around these supposedly visionary leaders has not aged well.

 

 

Posted in Uncategorized | 6 Comments

How The Other 1/300th Learns

Robert Pondiscio’s new book “How The Other Half Learns” (Avery September 2019) answers the age old question:  Can a bunch of twenty-something teachers who know nothing about education, nothing about child development, and nothing about what it is like to be a parent, get a non-random sampling of students of color to pass standardized tests?

In addition to answering that question in the affirmative, Pondiscio skillfully paints the most thorough picture of what exactly goes on behind the closed doors of Success Academy charter schools.  We get to meet the teachers, the administrators, the ‘scholars’, and the parents. I found the book very illuminating and recommend it highly.

There is a pretty short list of writers who Eva Moskowitz might trust enough to allow them to observe in a school for a year.  I’m glad that it was Pondiscio. He is one of the few reformers I have any respect for. We disagree on some things and we agree on some other things.  I even once went out to lunch with him and had a very pleasant time.

Before the book came out he wrote a blog post on The74 called “I Just Wrote a Book About Success Academy Charter Schools. It Does Not Support Your Preferred Narrative. I Hope You Hate It.”  Not to dash his hopes, but I, for one, did not hate it. Overall I thought it was great which means that only reformers who are willing to be honest with themselves will actually hate this book.

One premise of the book is that the fundamental secret to Success Academy’s amazing standardized test scores, mentioned throughout the work is the filtering of the right families.  On page 266 he writes “The common criticism leveled at Moskowitz and her schools is that they cherry pick students, attracting bright children and shedding the poorly behaved and hardest to teach  This misses the mark entirely. Success Academy is cherry-picking parents.” Parents must go through a series of tests and hoops to jump through for their children to get into and to stay in a Success Academy school.  First there is, of course, the lottery. But winning the lottery is just the first step. Described in great — and frightening — detail in chapter 20 “The Lottery”, lottery winners have to attend a mandatory informational session where they are told how much work it is to be a parent of a child at the school — how lateness is not tolerated and there is a 7:30 AM start time.  How there is no transportation provided. How every Wednesday is a half day and there is no after school program. How absences require a doctor’s note. Many prospective lottery winners give up after that meeting. Then there are several other steps like extensive paperwork and uniform fittings and a dress rehearsal. Even Pondiscio is shocked to watch how a student who is deep on the waitlist eventually get admitted to the school.  But having families who are this willing and able to comply with the demands made by Success Academy leads, predictably, to high standardized test scores. He doesn’t say this so bluntly, but let’s face it — this is a kind of cheating.

But if you look at the back of the book, you see that it was well reviewed by various reformers including former NYC schools Chancellor Joel Klein.  How can this be? Well even though Pondiscio says the test scores need to be seen in the context of the family selection process, he also argues, several times throughout the book, that it is OK that they do this.  The argument is that wealthy families use their resources to get their child into a school that is a good fit for them so why shouldn’t poor families who have the resource of being highly functional use that to get their child into a school that is a good fit for them too?

On page 333 he writes

It would be dishonest to pretend that Success Academy is not a self-selection engine that allows engaged families who happen to be poor or of modest means to get the best available education for their children.  But it is equally dishonest and close to cruel to deny such families the ability to self-select in the name of “equity.” Indeed, it is nearly perverse to deny low-income families of color — and only those families — the ability to choose schools that allow their children to thrive, advance, and enjoy the full measure of their abilities.

This is not the first time I have heard this argument.  A few years ago Michael Petrilli wrote a piece called “Who Will Speak For The Strivers’ in response to an article that showed that charter schools have much higher expulsion rates than in neighborhood public schools.

My first response to this would be that only 16 out of the inaugural 73 students even endured to graduate Success Academy.  If a higher percentage were actually served by Success Academy, then this argument of ‘shouldn’t they also get to choose a school that is good for them?’ would be more compelling.  Since for the vast majority, they did not choose a school that was good for them, even after going through all those steps, and they did ultimately choose to leave, so what kind of choice did they really get?  For the small number of families and children that turn out to be a good fit after all, there are at least double that number who regretted that choice and surely feel duped by the false promise that Success Academy actually cares about their children.

Maybe an analogy will make this more clear:  On airplanes, only wealthy people have the choice of flying first class while people who can’t afford that must fly in coach.  So now Success Airlines comes along and they have something they give people the choice of flying in something like first class except the seats are outside the plane on the wings and you have to get to the seats on your own and there’s a 2/3 chance that you’re going to be jettisoned from that seat before the flight is over anyway.  Could this really be considered a viable ‘choice’ for poor people?

If Pondiscio is making the case here that Success Academy should have the right to exist, I’ve never said that they shouldn’t exist.  But their existence should not be to just benefit the few that are a good fit at the expense of not only the students at the neighboring schools but also the students who left Success Academy before graduating.  To do this, I think that they need more oversight and regulations and transparency about what goes on inside their schools.  And I’m glad that this book does a nice job about showing the sorts of abuse that occur in the school which I’ll get to next.

One thing that is striking about Success Academy is how young and inexperienced the teaching staff is.  On page 294, the teacher attrition is rationalized this way “Teacher turnover, and lack of experience and continuity, is widely assumed to be a problem, particularly in urban schools.  But it’s never suggested that our military would be better if only soldiers stayed in uniform longer. So far, the relative inexperience of Success Academy teachers hasn’t seemed to compromise their effectiveness.”  I guess the problem with trying to study Success Academy scientifically is that you can’t isolate the variables. So what would a Success Academy look like if they had self-selected parents and experienced teachers? Or a Success Academy with randomly-selected parents and inexperienced teachers?  Success has set up a controlled environment where even inexperienced teachers can seem like they are doing a good job. But the teachers seem to blindly follow the Success Academy protocol even when it is bad teaching practice. An example of this is their overuse of the classroom management strategy called ‘behavior narration.’  The idea is to praise students who are on-task, which of course is good in moderation. But they take it way too far, narrating and narrating which is very annoying and distracting to students and really unnecessary, their students would learn more if the teachers would cut back on that. 

The best example of a bad decision by a teacher, and probably the most frightening pages of the book, happened in Chapter 15, “Come to Jesus” one of the Kindergarten teachers holds a parent meeting in January because many of the Kindergarteners are not on target to reach level C by February.  The teacher hosts a parent meeting and makes a big speech about how everyone needs to work harder so the students will get back on track. She says that some kids have been absent too much and tells the parents “Anytime I’m sick, I come here. I can’t afford to lose that day with your children.  At the same time, your child cannot miss that day with any of us.” She tells them that if their Kindergartners don’t get on track they will not go to college. She tells them that if they don’t get to level D by the end of Kindergarten, they will have to be left back. To me, this is showing a lack of understanding of child development.  This teacher is not a parent and doesn’t really understand that children develop at different rates and that sometimes children who seem to be really behind on some developmental milestone often suddenly catch up. She asks if parents want to ask or say anything and a parent says that her child is on level C but maybe could be a D but when her child is struggling to read at home the child gets upset and the parent has her stop reading for the night.

The teacher responds that it is hard to push kids to do more and she tells them about how earlier that day a student did not do an adequate job on a book review and he lost a privilege.

From page 193:

“That was really rough for him to hear.  At the same time, did I say, ‘It’s OK if you don’t finish your book review?”  Now it’s Syskowski who starts to tear up. “No. It’s not OK. Because why would I let him fail when other kids are surpassing it and they’ll go to first grade?  You would not want me as your child’s classroom teacher. You would not want Mr. Carnaghi or Ms. Skinner to be your child’s teacher if we were like, ‘You know what?  You’re right. It is really hard. Let’s just let them be a B’” How about if they get two out of the four sight words correct? ‘That’s good enough.’ Where are they going to be in thirteen years?  Then we won’t talk about college. And that something that ..” Syskowski lowers her gaze to the floor. “I get chills. That’s something that’s really hard for me to …”

Syskowsky doesn’t finish the thought.  She can’t. She’s crying. “I do get emotional.  Because your children are amazing. They are absolutely amazing.  I try to …” She quickly gathers herself. “We will never lower that bar because it’s too hard.”

Pondiscio follows the description of this outburst with an interesting observation.  He says that some people might see this (as I do) as unprofessional and unnecessary. I was nodding my head at this since she is lying to the parents about how not getting to some arbitrary reading level in four months will mean that their children are on track to get left back and eventually not go to college.  But Pondiscio evidently doesn’t feel this way. He says that the critical view is the “dim view” of this. Then he gives another perspective, obviously, for him, the more accurate one:

“Or you can see Carolyn Syskowski, with her giant heart and Pez-dispenser grin, who calls every student “love bug” and spends hours each day on the floor with other people’s children, wipes their noses, pulls on their coats, sends them home, and then worries into the night about their reading and math scores.  You can see, if you are so inclined, an unusually gifted and competent teacher, with emotional gears you cannot fathom, who can issue a consequence to a five-year old like a bank examiner rejecting a loan, then an hour later bring herself to teacher in front of a hundred strangers when for a single moment she catchers herself weighing the cost of not doing so.”

I don’t buy this.  A teacher who does not care enough to understand basic child development and who lies to parents about how their children are not on track to go to college is someone who doesn’t really care very much.

The principal of the school is in her 30s and is portrayed, as I see her, with a narcissistic cult-leader mentality.  She wins an award at the end of the book and starts weeping about how she doesn’t deserve it and I’m inclined to agree with her on that.

Pondiscio is mainly permitted to observe, not coincidentally, the smoothest running Success Academy school.  He says early in the book that he was given mostly free rein of who he would observe though there were some teachers who he shouldn’t since it would make them uncomfortable.  He was also not given permission to freely go to other schools in the network. There was a chapter where he visited the Harlem 2 Success Academy with Eva Moskowitz and the place was so dysfunctional that Eva said she wanted to ‘slit her wrists.’  Interestingly, that school also gets very high standardized test scores which makes me think that perhaps all the other things the ‘good’ school does by the Success Academy methods really don’t matter, at least when it comes to the test scores.

In a section that even disturbed Pondiscio, [SPOILER ALERT:  SKIP THIS PARAGRAPH IF YOU WANT TO BE DISTURBED BY THIS IN THE CONTEXT OF THE FULL BOOK]  he interviewed the family of a student who was, essentially, kicked out of the school earlier in the book.  The student had behavior problems and the school responded to them by — get this — calling 911 for an ambulance to pick him up and bring him to the hospital for this several times. They also called child services to have them investigate the family.  A teacher who left the school abruptly may have also resigned over being asked to lie to investigators about this incident. I applaud Pondiscio for putting this section into the book. It really shows how heartless the school is.  The school defends themselves on things like this by saying things like “If a kid is throwing a chair and it can hurt another student, we have to do something,” but of course this is just an unnecessary overreaction designed to make the parents ‘voluntarily’ pull their child from the school.

Though Pondiscio is generally a thoughtful and nuanced thinker about education, I was kind of frustrated in the simplicity in the various times he talks about neighborhood public schools.  On page 17 he says “Publicly funded but privately run, charters offer a lifeboat to low-income kids from failing schools while in theory creating choice-driven market pressures that force bad schools — charter and traditional public schools alike — to improve or die.”  Words like “failing”, “bad”, and “lifeboat” are too charged for my taste.

There are places in the book where I would have liked Pondiscio to provide some numerical data.  Like when he writes on page 211 “It is not possible to say with certainty what percentage of the network’s students live in stable homes with both parents, or to make a comparison to students at public schools in the same districts, but dual-income household appear to be overrepresented among the Success parent body.”  Spending a year in a school, it seems like the sort of thing he could have easily come up with a good estimate had he wanted to.

There are some other places, however, where he does use numerical data though that data is often taken out of context and used as a way to bolster Success Academy or to counter some of the most common criticisms of Success Academy.  On page 54 he cites a CREDO study in which it was estimated that “Success Academy students gained the equivalent of 228 days [a year] in math.” I was disappointed to see him introduce a number as outrageous as that (a school year is only 180 days!) without at least mentioning that the ‘days of learning’ statistic is not meant to be taken literally.

On page 296 he uses data that seems to imply that Success Academy has a lower student attrition rate than NYC public schools in the same neighborhood.  This is something I’ve studied a lot so I want to address this here. The first graduating class at Success Academy was 16 students out of 73 that had originally started there as 1st graders.  (In the book Pondiscio mentions the small size of that graduating class a few times but only once mentions that the cohort started with 73 students and even then, didn’t make such a big deal out of it)  I’ve crunched the numbers from 2017-2018 school year to the 2018-2019 school year and found that if the grade to grade attrition rate continues for the next twelve years then the 2,647 Kindergarteners who started at Success Academy last year will be whittled down to only 246 graduating seniors 11 years from now.  (Out of 85,000 Kindergarteners who started in New York City last year, this is about 1/300 of the total population in that cohort, which is where the title of this review comes from.)

Screen Shot 2019-09-22 at 7.06.28 PM

Any study that says that Success Academy has lower attrition than neighborhood public schools is not accounting for the fact that the other schools replace the students who leave, and who often have the most unstable family situations, with other students with equally difficult situations.  They are also not accounting for the fact that students who leave one public school for another school similar to it are not faced with a huge dilemma. But a student leaving Success Academy — The Success Academy — for a public school, well, that’s quite a choice there. I would expect Success Academy’s attrition to be well below the attrition of public NYC schools unless they are forcing students and families out.

One of the more dramatic pieces of evidence of the selective attrition by Success Academy is that their graduating class of 2019 was 20 girls and 6 boys.  Try to find a public school in New York City (besides an all-girls school) that has a gender imbalance anywhere near that. This is the sort of statistic that I think is really compelling since highly functioning families have male babies as often as they have female babies so it indicates that the filtering process beyond the initial parental filtering helps Success Academy cheat even more to improve their test scores.

Seniors2023

There were a few mistakes, I doubt intentional, in the book I should point out since most readers would not would catch these.  On page 159 he writes about the first high school cohort who will graduate in 2017. He calls them “The first class of students to have spent their entire school careers as Success Academy students” when, in fact, the 2018 cohor would be that.  The 2017 cohort started as 1st graders at Success. On page 55 he writes about the famed “Waiting For A School Miracle” New York Times op-ed by Diane Ravitch. “She derided ‘miracle schools,’ citing several examples of schools whose scores skyrocketed one year and crashed the next.”  I helped Ravitch with the research for that op-ed and those were not schools that ever had high test scores. They were schools touted by politicians that had 100% college acceptance rates but actually had high attrition between 9th and 12th grade and always had terrible test scores.

OK, I’m going to bring this to a close.  I don’t want to spoil the book anymore than I did already.  There are many other important scenes and arguments in the book and I jotted things in the margins on nearly all of the 340 pages.  It is well worth reading and I’m thankful to Pondiscio for writing a book that, in my interpretation, will be devastating to the reputation of Success Academy.  He certainly tried to offer some counter arguments (mostly unconvincing to me) about why Success Academy is still a good place, all things considered.

Though Pondiscio is ‘reform friendly,’ he is not of the Kool Aid drinking type of reformer.  I think that in writing this book he is showing his frustration for the false promises of the reform movement.  After twenty years of growing, the reform movement has been a let down. There are some reformers who are in denial about this — most TFA staffers are like this.  There are other reformers who probably know how poor of a job the movement has done and they don’t really care since at least they are in power and that makes them money and feeds their egos.  But Pondiscio does not belong to either of those camps. I see Pondiscio as someone who is trying to use the scientific method to improve schools. He has written about how not enough of education research is based on brain science — especially about how students learn to read.  He believes, and I agree, that a lot of teacher time is spent inefficiently looking for useful curriculum materials. But unlike a lot of reformers who are not willing to evolve and incorporate the meager results of 20 years of reform experimentation into their thinking, Pondiscio writes on page 322, “But there are things we know and do not say in education and education reform.  One of them is that we expect too much of schools.” And the fact that the one apparent piece of evidence of the power of education reform, Success Academy, only gets its results by a form of cheating is particularly disheartening for a reformer who cares enough about the cause to pursue it scientifically. On page 321, “Is Success Academy a proof point that the reform playbook works and that professionally run schools with high standards and even higher expectations can set any child on a path out of poverty?  Or does the rarity of Moskowitz’s accomplishment suggest that however nobly intended it might have been, the reform impulse was doomed from the start?”

Sadly, most reformers are too stupid and overconfident to even know how bad they are at this. Whether or not at least their hearts are in the right place on this is also debatable. You can’t really claim to care so much if you don’t care enough to honestly assess the results of your reform experiment.  I can’t see how reformers can be honestly enthusiastic about the details in this book but if it is true that reformers do really like this book and are not just pretending to then Pondiscio has really accomplished quite a feat.

Four and a half apples out of five.

Posted in Uncategorized | 9 Comments

My Personal Top 10 Blog Posts

I started this blog about 11 years ago, and in that time I have written over 500 posts.  Some of these posts are particularly meaningful to me so I thought I would collect my personal ‘top 10’.  I know that I don’t often go through the archives of blogs I like so this is a way for people who read this blog to either re-live the ‘greatest hits’ or for newer readers to get caught up on it:

#10 I Taught At The XQ Houston Super School

When Steve Jobs’ widow got involved in education reform and aired a prime time infomercial on all three major networks to promote her plan to reinvent high school, it was a major coincidence that the main school she featured was the high school that I taught at in Houston.  with some investigative reporting and some contacts I still have from my Houston days, I revealed a scandal which may have contributed to the rock star principal there being fired.

#9 Same Kids, Same Building, Same Lies

At the Teach For America 20th anniversary alumni summit, I heard Obama’s secretary of education, Arne Duncan, tell a story about an amawing school turnaround that sounded, to me, to be too good to be true.  This led to my first ever school ‘debunking’,  This got me discovered by Diane Ravitch and I became a key ‘fact checker’ for the corporate reform skeptics from that day on.

#8 The Death of math

I haven’t written much about my main passion in education — teaching math.  This post summarized what I think about how the math curriculum evolved in this country and what I think of it.  This one was widely read because it was picked up by a popular site called ‘the dish’.

#7 Teacher quality at KIPP

I had the rare opportunity to visit one of the most high profile charter chains in the country, the New York City KIPP school.  As I blended into the woodwork, I had the opportunity to reveal that what is going on there is not particularly impressive.  I never got invited there again.

#6 How I teach 2.6 months more of math in a year than the rest of you slackers

I’ve written a lot about research in education and how it is often presented in a misleading way.  For this post I showed how the ‘months of learning’ statistic is completely unreliable.

#5 4th Best High School In New York Is A KIPP School That Doesn’t Exist

This post revealed that KIPP had cheated on their US News & Zorld Report entry.  A few weeks later their rating was disaualified and I am confident that this post was the cause of it.

#4 Open Letters To Reformers I Know. Part 8: Wendy Kopp

Over the years I wrote about 20 ‘open letters’ to various people in the education reform community.  Some were to people I knew and others were to people I never met.  This one got a lot of attention, including a response and there were articles about this exchange in both the New York Times and The Washington Post.

#3 Open Letters to Reformers I DON’T Know. Part IV: Arne Duncan

This is a post I’m really proud of.  In it I explain to Arne Duncan about why the methods he pushes to evaluate schools and teachers is inaccurate by applying them to his basketball career at Harvard.  This reauired me becoming an expert in late 1980’s Ivy League basketball.

#2 Analyzing Released NYC Value-Added Data Part 2

When the New York media released the teacher ratings to the public, I used the data to show how flawed the metrics were.  This post was the most compelling where I showed that a teacher can be evaluated ineffective and highly effective in the same year.

#1 Why I did TFA, and why you shouldn’t

This was the ‘viral’ post that got me on NPR and propelled the popularity of this blog.  I’m not sure that TFA ever fully recovered from it.

 

Posted in Uncategorized | 2 Comments

Are 85% of TFA alumni really “working in education or careers serving low-income communities”?

In a recent podcast on ‘Getting Smart,’ they interviewed TFA CEO Elisa Villanueva-Beard.  At the 0:56 mark the host gave this series of numbers, straight from the TFA PR department, “Of the 53,000 alumni, 85% work in education or in careers serving low-income communities. That includes 1,260 school leaders, 471 school system leaders, 500 policy and advocacy leaders and 200 social entrepreneurs.”

A major critique of TFA is that the teachers use TFA as a way to pad their resumes — that they teach for two or maybe three years and then go on to law school, medical school, or business school.  If this 85% number is accurate, it would serve as a great counter to any critique of TFA that the corps members do not commit long enough.

Six years ago in a HuffPost editorial, Elisa Villanueva-Beard said that it was 80%.  Over the last few years this has grown, at least in theory, to 85% and it is something that is now quoted on the Teach For America website in the section about their impact.

Teach For America has a 28-year track record of advancing educational excellence and equity in the United States through our network of remarkable and diverse leaders working to expand opportunity and access for all children. With nearly 60,000 alumni and corps members in 51 regions around the country, our network now includes 14,000 teachers; 3,700 school principals, assistant principals, and deans; more than 300 school system leaders; 500 policy and advocacy leaders; nearly 200 elected leaders; and almost 200 social entrepreneurs. And while only one in five Teach For America corps members had plans to teach before applying to TFA, 85% of alumni are now working in education or careers serving low-income communities.

This 85% statistic, if to be taken literally, would mean that 43,350 out of 51,000 TFA alumni have a career in education or serving low income communities.

The first question to ask is:  How was this data collected?  Did TFA track down all 51,000 alumni?  Did they do some kind of random sampling?  Or is this based on their alumni survey?  I know it is based on their alumni survey since when I fill mine out there is this question:

tfasurvey2

So what they should say is that out of the people who self-selected to take the alumni survey, 85% of the responders answered yes to one or both of these questions.  There are two types of bias at work here.

The first is selection bias since this is not a random sampling — it is the people who choose to answer which likely has a higher percent of people likely to answer ‘yes’ to these questions.  We don’t really know what percent responded.  Since there is so much selection bias it probably doesn’t matter if the response rate is 60% or 70%.  But if the response rate is something like 10%, that would make the statistic even less reliable.

The other type of bias comes from the wording of these questions.  What qualifies as “relates to improving the quality of life in low-income communities”?  Since it is up to the responder to decide, we really don’t know.

The way these questions are worded, something I’m really wondering is:  What percent of college graduates, in general, would answer yes to one or both of these questions.  Since we don’t have this control group to compare to the TFA group, it is hard to know if 85% is actually impressive.

Another thing kind of ironic about the 85% number is that, in general, 85% is the percent of corps members who don’t quit during their first two years of teaching.  Since this statistic is just about ‘alumni’, those people who quit are not included and that further skews the numbers.

In the past six years TFA has found a way to say that this number has grown from 80% to 85%.  Who knows what they will be saying it is six years from now.  Whatever it is, people should know, as TFA absolutely does, that this number is complete nonsense, something that it would not even be fair to call a half-truth.  If TFA continues to use this 85% number and they didn’t realize it was bogus before, they know now and they will be purposely using something they know is misleading at best.

There are two ways to get more accurate data.  One is for TFA to try to fully account for all 51,000 alumni.  This is a difficult thing to do and something that TFA is not going to invest the time and money into since it can only make that number more accurate and make them look worse for it.  The other way is to do a random sampling where they pick about 10,000 people who started with TFA out of the about 65,000 alumni plus quitters.  Then they would have to track down all 10,000 of those and that would be a pretty good random sample, I think.  They won’t be willing to to this either.

So I’ve decided to do a little crowd sourced experiment.  In the early days of TFA, they sent me an alumni directory with the name of every corps member from 1990 to about 2000.  So here is my experiment.  There were 522 corps members in the first cohort of 1990.  I had WolframAlpha picke 100 numbers between 1 and 522.

Screen Shot 2019-08-05 at 9.25.29 PM

Then I alphabetized the 522 corps members from 1990 and assigned each a number and then picked the 100 people who corresponded with the 100 numbers that were randomly chosen.  This is a true random sampling and it is about 20% of the total population which is a pretty good size sample actually.

215 Holifield, Erin
278 Lienhard, Bill
111 Darby, John
308 McGlone, Thomas
19 Arsuaga, Maritere
461 Tan, Chin
425 Simes, Jeffrey
115 Davis, Geoff
253 Koo, Chiray
171 Gomez, Carlos
184 Groom, Ileetha
392 Rivera, Richard
370 Polen, Michael
277 Lewis, Kimberly
299 Martinez, Jane
271 Lerntouni, Tank
113 Davis, Andrew
60 Brown, Daryl
320 Miller, John
401 Sabin, Caroline
3 Abell, Jennifer
273 Levine Grimaldi,
72 Can, Kristen
95 Cobb, Kendall
265 Lay, Corey
384 Rehl, Michael
281
Livingston, Therese
485 Wade, Andrea
122 Dennis, Terrence
379 Ramsey, Lukman
512 Winiecki, Marc
236 Jones , Stephanie
397 Roth, Sharon
429 Skolaslci, Renee
499 Wickliff, Derek
364 Phoa, Cynthia
193 Hamilton, Donna
413 Seligman, Miklci
95 Cobb, Kendall
125 Dineen, John
467 Thompson, Julia
56 Brooks , Daniel
361 Peterson, Lisa
99 Collins, Philip
502 Willey, Kristin
136 Ebby, Rachel
514 Wright, Ernest
41 Boatright, Laura
194 Harrigan, Lisa
505 Williams, Brandi
501 Wilkinson, Wendy
173 Gonzales, Emilio
373 Price, Wendy
20 Aumou, Elizabeth
367 Plaman, Kathryn
180 Grado, Danielle
399
Ruvoli-Gruba, JoAnne
444 Steensland, Lara
478 Utley, Stephen
57 Brooks, Hoff
27 Beck , John
517 Yudell, Michael
384 Rehl, Michael
185 Guerrero, Scott
116 Davis, Lorna
350
Palazzolo, Rayann
513 Wolf, Matthew
467 Thompson, Julia
203 Heitmann, Noel
205
Hendricks Richman, Susan
68 Bushnaq, Faith
109 Crean, William
129 Donoho, Lori
186 Gulling, Egypt
339 Nicholas, Robert
210 Heyl, Densie
257 Kruse, Jennifer
137 Edge, Kecia
204 Held, Robert
521
Zimmerman, Andrea
313 McPherson, Maria
224 Israel, Todd
63 Brown, Michael
106 Cox , David
251 Klender, Kimberly
336 Newkirk, Jennifer
64 Buckley, Michael
333 Nagler, Mary
25 Basich, David
181 Graham, Elliott
299 Martinez, Jane
146 Eppolito, Veronica
201 Haynes, Michael
216 Holmes, Tiffany
464 Taylor, Olu
437 Snyder, Christina
460 Tabb, Kathleen
226 Jacobs, Sandi
297 Marie!, Kecia
237 Jones, Brian

OK, now I did not then go through and start tracking down each of these 100 people.  There was a time a few years back where I may have had the energy for such a project.  But, with six degrees of separation and all that perhaps some readers will know some of these people who are all about 51 years old right now and graduated college in 1990.  Or maybe readers can pick someone off the list at random and write a comment, something like “297 is a banker at Wells Fargo” with some kind of link to prove this.  I haven’t really thought this through fully, but if these 100 people can be researched, it would be interesting to see if approximately 85 of them are “working in education or careers serving low-income communities.”

 

Posted in Uncategorized | 16 Comments

The Missed Opportunity Myth

Before Michelle Rhee was a board member for Miracle-Gro she was the founder and CEO of StudentsFirst.  Before that, she was Chancellor of Washington D.C. schools from 2007 to 2010.  Before that, she was the CEO of The New Teacher Project.

And even though Rhee is not a public figure anymore in education, she continues to influence education policy through The New Teacher Project which has since changed its name to TNTP.  TNTP puts out slick papers that it calls research but is really propaganda disguised as research.  Their first one was called ‘The Widget Effect’ which laid out the case for replacing salary schedules with a system based on merit pay based on statistically inappropriate analysis of standardized test scores.

And over the years they have put out other papers with clever titles like ‘The Irreplaceables’, ‘Rebalancing Teacher Tenure’, and ‘Teacher Evaluation 2.0.’  These papers are often quoted by ed reform propaganda sites like The74 and Education Post.

One of their most recent papers is called ‘The Opportunity Myth.’  Its central thesis is something that reformers love to use in their teacher bashing arguments, which is that too many teachers shortchange their students by having low expectations for them.  The work they assign is not challenging enough and since students always rise to the challenge of whatever you assign to them, these teachers are negligent in their duties.

So TNTP observed 1,000 lessons in five school districts and analyzed 5,000 assignments and 20,000 student work samples.  They concluded that the work that the students were assigned was usually inappropriate for their grade level.  Here is one of their graphics from the paper.

Screen Shot 2019-07-30 at 3.05.41 PM

So 75% of the time students are working on things “that were not grade appropriate.”

My first question about this study is:  If a teacher is teaching a 5th grade class where the students are behind and only at a 3rd grade reading level, but you give them material at a 4th grade level, are you someone with low expectations or high expectations?  I would say you are someone with high expectations.  But this research paper would conclude that this teacher was someone with low expectations because the 4th grade reading level is lower than the grade the students are in.  This is the main flaw with the misleading conclusion so happily quoted by the reform propaganda sites.

TNTP actually put up samples of the assignments they rated on their website so I took a look at how they rated some math assignments to see if I agreed with their ratings.  I was not surprised to see that when it came to 8th grade math, the TNTP raters had no idea what they were talking about.

This part is a bit ‘mathy’ but I will do my best to explain the context of this assignment so if you are not a math teacher you can still get the idea of how misguided TNTP is in their rating system.

One of the most important concepts is math is something called ‘slope.’  It is first taught in 7th or 8th grade and it is weaved into all the following courses and actually comprises a large part of Calculus.  Informally speaking, the slope of a line is a measure of how steep it is.

In the picture below, the segment AB has a slope of 1, the segment CD has a slope of 2, and the segment EF has a slope of 1/2.

Screen Shot 2019-07-30 at 3.20.40 PM

A slope of 2 means that for every unit you move to the right, you have to move up 2 units.  For segment CD, to get from C to D you would move two units right and four units up.  So if you only moved one unit to the right from C, you would move two units up to get to the point on the line.

One of the main ways of testing to see if students know slope is to give a diagram with a line segment on it and ask them to compute the slope.

Screen Shot 2019-07-30 at 3.27.55 PM

The slope of this line segment is:

A) 1/3    B) 3    C) -1/3    D) -3    E) Slope is not a ‘thing’

If you said ‘B’, you are correct.  To get from (2,1) to (5,10) you have to go three units to the right and nine units up.  So if you only went one unit to the right, you would only go three units up.

There is also a ‘formula’ for slope:

Screen Shot 2019-07-30 at 3.33.54 PM

There are a lot of way to test to see if a student understands the concept of slope.  If you just have the students calculate slope over and over there is a chance they they don’t really ‘understand’ slope but that they just memorized a formula.  Still, if you have a student who cannot calculate the slope of a line or line segment when the coordinates of two points are known, then that student surely does not understand the concept of slope.

Now (and I know I’m losing readers here by the minute — but you have to have some background so you can understand why I contend that TNTP is not qualified to judge the quality of a math assignment based on a lesson about slope) you may not be surprised that if there are three points on the same line segment and if you calculated the slope by picking any pair of those points, you would get the same answer.  In this case, you will get a slope of three no matter which two points you choose.

Screen Shot 2019-07-30 at 4.06.36 PM

Informally speaking, if the slope of the segment from (2,1) to (4,7) was different from the slope of the segment from (4,7) to (5,10) then it wouldn’t be a straight line.

When you go to the TNTP page where they show examples of activities and how they rated them, they have this example for a lesson about slope.

8th_Grade-math-weak_640_1093_s

Math teachers I’m sure will agree that this is a good set of questions.  The first two questions are based on a graph and the student knew to draw a vertical and horizontal line which shows a conceptual understanding beyond just plugging numbers into a formula.  In the third question, the data is presented as a chart instead of on a graph, giving an opportunity for students to get comfortable with multiple representations and this question has six different ways to calculate the slope so there is opportunity for students to discuss that.  In the fourth question the student creates a chart from the data and then does it like the third question.  One of the answers is a positive integer, one is zero, one is a negative fraction, and one is  a positive fraction so all different sorts of answers are covered.  This is a fine example of practicing the skill of calculating the slope as data is presented in different ways.

But according to the ‘experts’ at TNTP, the teacher who created this assignment was guilty of low expectations.

Screen Shot 2019-07-30 at 4.46.20 PM

TNTP explains why they made this judgement, but to understand why their rationale is nonsense, I’m going to have to take you through some more background about math and about the common core standards.

There are math standards from K to 8th grade and then standards for six different aspects of high school math.  The math standards for a grade, like 8th grade, are pretty short, taking up maybe 10 pages if you print them out.  States will take those standards and turn them into lesson maps where there will be 150 or so subtopics based on the standards.  The standards are not very thorough, actually.  There are big gaps in them that anyone who is a practicing math teacher would know need to be filled in.

So for 8th grade math, the concept of slope is only mentioned twice.  Here is the text from the two mentions:

Screen Shot 2019-07-30 at 4.55.26 PM

So there is no mention in the standards at all of mastering the skill of actually using the slope formula to calculate the slope of a line defined by data in various representations.  Any real teacher would know that this is an extremely important skill that you would spend several days on as it will come up for the next 4 years in many important math units including several months of Calculus.  And the teacher who made this assignment that was deemed ‘weakly’ aligned to the standards was aware that it would be negligent not so spend some time practicing with the slope formula.

So the TNTP people gave this explanation for why this was ‘weakly aligned’:

Screen Shot 2019-07-30 at 5.09.19 PM

OK, so you likely have not been teaching math for 27 years like me so this explanation does look like something that they put a lot of thought into.  So I’m going to say a few things about their rationale for bashing this assignment.

First of all, this is not the standard that the assignment was trying to address.  This assignment is addressing a standard that is not explicitly stated because it is so obvious to everyone (except the TNTP raters) who needs to understand what sorts of things are needed in teaching the concept of slope.

Secondly, this standard 8.EE.B.6 is perhaps the most unnecessary and, sorry to use this language, stupidest standard I can imagine for this topic.  Remember this picture I put up before?

Screen Shot 2019-07-30 at 4.06.36 PM

They are saying that a better way to justify that if you have a straight line then the slope of any segment on that line will have the same slope is to apply a much more difficult concept to master from Geometry, the idea of similar triangles.

Similar triangles are two triangles where one is like a ‘zoomed in’ version of the other.  So they are not necessarily congruent triangles because there is a little one and a big one.  What makes them similar is that they have the same angle measurements.

So these two triangles are similar since they both have angles of about 72 degrees, 90 degrees, and about 18 degrees.

Screen Shot 2019-07-30 at 5.29.07 PM.png

Notice that the sides of the big triangle are double the sides of the smaller triangle.  We say that the corresponding sides are in proportion and this will always happen with similar triangles.  It also works the other way around — if we know the corresponding sides are in proportion, then the triangles are similar so all their angles will be equal.

Using this concept of similar triangles, we can look at that straight line problem with three points on it in a different way (though this would not be advised since it is unnecessarily confusing and really won’t offer any truly useful insight into this topic at this point in 8th grade).

Screen Shot 2019-07-30 at 5.37.20 PM

Adding some lines to make two triangles, you’ve got the big one with a horizontal side of 2 and a vertical side of 4 and you’ve got the small one with a horizontal side of 1 and a vertical side of 2.  You also have that right angle in the bottom right corner of the triangle.  If two sides are in proportion and the angle between those two sides is the same then these are similar triangles.  And if they are similar triangles, the angles BAC and ECF would need to be the same too and if those angles are the same then the line ACF would have to be a straight line.

This is not something I would ever teach to 8th graders unless my principal said that I’ll get fired for not following the common core standards like they are the ten commandments.  It is maybe an interesting curiosity that I could mention or not mention, depending on the situation.  It certainly doesn’t belong on a short list of standards, like the common core standards.  And for TNTP to pan this assignment because they don’t understand that a good teacher knows it would be negligent to try to push this ill advised standard if they can avoid it, and that a good teacher knows that even though it doesn’t say that the slope formula is something that should be practiced, it certainly is something that should.

I can probably do a similar analysis for every assignment that TNTP said is weakly aligned to the standards, but I won’t torture myself or the people who like to read this blog with any more of this.

TNTP is the legacy of Michelle Rhee.  She, a bit like Voldemort in the early Harry Potter books, is out of sight right now but she continues to influence education policy through her intermediates and it is important to show that making a fancy paper that looks like real research and is quoted by The74, Education Post, and the TFA blog does not mean that the researchers are qualified to analyze the data they collect.

 

 

Posted in Uncategorized | 16 Comments