Underachievement School District Superintendent Resigns In Disgrace

One of the most high profile experiments in education is Tennessee’s Achievement School District (ASD).  As part of their Race To The Top funding, the former commissioner of education, TFAer Kevin Huffman, hired another TFAer Chris Barbic to make a plan for the lowest performing schools in the state.

Funded with millions of dollars, the ASD launched in 2011 with a very specific goal spelled out very clearly on their website (until about six months ago when they changed it)

Screen Shot 2017-09-09 at 12.08.47 AM

Any school that was in the bottom 5% of schools based on test scores could be either taken over and run by the ASD or could be handed over to a charter organization with the plan to ‘catapult’ them into the top 25% within 5 years.  The ASD started with a cohort of six schools in 2011 and has about 30 schools today.

In 2014, three years into the experiment, Barbic claimed in an interview that of the first 6 schools, three were on track to meet that top 25% goal including one school that was on track to meet the goal one year ahead of schedule.  Around this time, Barbic was inducted into Jeb Bush’s ‘Chiefs For Change’ organization, a who’s who of ed reform, many of whom are now working as high paid consultants rather than as school or district leaders.

In 2015, four years into the experiment, things were not looking good for the ASD.  Of those original six schools, four of them continued to be in the bottom 5% while the other two had merely catapulted into the bottom 6%.  Barbic resigned at the end of 2015, about 8 months before we could learn if he was able to accomplish the goal of moving those schools from the bottom 5% to the top 25%.  Fortunately for Barbic, computerized testing glitches caused the Tennessee state tests in the spring of 2016 to be invalidated so we will never know if the ASD got any of those schools where they promised in the five year window.  Also fortunately for Barbic, he got a job for the billionaire John Arnold working on their education initiatives.

Barbic was succeeded by a member of his leadership team, Malika Anderson.  Some time during her term, the ASD changed their mission on their website.  No longer are they talking about catapulting the bottom 5% to the top 25% in 5 years.  Some time between March 2016 and April 2017, the website now states that their mission is ‘By 2025, we will close the opportunity gaps long persistent in Tennessee’s public education.’  So they want 8 more years and they learned not to make any specific promises, like how much they want to reduce those gaps even.

Screen Shot 2017-09-09 at 12.09.38 AM

Anderson was the superintendent during the latest state tests in the spring of 2017.  Though the scores for elementary and middle schools have not yet been released, the high school scores were and they were so bad that even the normally forgiving Chalkbeat Tennessee wrote an article entitled Tennessee’s turnaround district scores worse in nearly all high school subjects.  In this article it was revealed that the students in the ASD high schools, despite having been in ASD schools for six years had 8 percent passing English and less than 1 percent passing math.

Chalkbeat Tennessee actually tried to make a lame excuse for this poor performance

Four of the ASD’s six Memphis high schools are turnaround schools: Fairley High, GRAD Academy, Hillcrest High and Martin Luther King Jr. College Preparatory. Two are alternative schools that aim to help off-track or disconnected students attain their diplomas. Comprising a third of the scores across the ASD’s high schools, those two alternative schools likely skew the district’s test results down.

This shows a complete lack of numeracy since even if those other two schools got 0% passing, that would mean that the other four schools would have at most 2% passing in order to have an average of 1% passing.  So, no, those two schools would not ‘likely skew the district’s test results down’ as they claim.

Anderson herself takes the opportunity to defy the ‘no excuses’ philosophy of ed reform by responding to the scores with “[Our] students have often spent most of their educational careers in underperforming elementary and middle schools and have very little time before they will have to transition to postsecondary life …,” even though those students spent much of their educational careers in ASD schools that she helped run.

With the elementary and middle school scores coming soon — and really the final nail in the coffin of the ASD as even with an extra year on that five year plan they likely did not move the bottom 5% of the schools even out of the bottom 5% — I was not surprised to learn four days ago that Malika Anderson (also a recent inductee into ‘Chiefs For Change’) has ‘decided’ to resign and become a consultant.  It turns out that Anderson was actually the final member who remained of the original team that started in 2010 with Barbic.  So not one person will be around to be held accountable for the failure of the ASD we have already seen in the high school and very likely to see when the other scores are soon released (which surely Anderson has already gotten an early preview of).

Chalkbeat Tennessee misses the opportunity to write about this as the scandal that it is.  In describing the failure of the ASD to improve the test scores of the schools it took over they wrote

While scores have been lackluster for most ASD schools (scores released last week for high schools were disappointing), even its critics acknowledge that the district has nudged Memphis school leaders out of complacency and created a sense of urgency to address longstanding deficiencies in neighborhood schools.

Words like ‘lackluster’ and ‘disappointing’ do not accurately convey what a disaster this overhyped experiment has resulted in.  And to give the ASD any credit for ‘nudging’ other school leaders into improving is somewhat deluded, I think.

Chalkbeat Tennessee had another chance to cover this story appropriately, though instead the same reporter did an exit interview with Anderson with the absurd title ‘Outgoing ASD chief reflects on Tennessee’s turnaround journey’ .  She clearly resigned before she got fired so ‘outgoing ASD chief’ is too gentle.  And ‘Tennessee’s turnaround journey’?  It wasn’t a ‘journey’ it was more of a ‘fiasco.’

In this interview, the reporter — I can’t believe that this is actually someone’s full time job to report about education in Tennessee and they can’t even identify a newsworthy story — avoids any tough questions about the test scores.  Anderson answers the last question about what she would tell her successor and she says

Also, to maintain exceptionally high expectations for what our kids can do. When we hit challenges, or don’t see the outside gains that we all want for our kids in a very short time, some people could start to lower expectations for what’s possible for our kids, that’s the wrong move. We have to keep expectations high and adjust our own perceptions and resources to help our kids, who we know get there.

This is ironic since she is supposedly resigning on her own will so if that’s true, she’s giving up on the kids.  And as far as keeping the expectations high, why is it that under her leadership the ASD changed their mission from the ambitious and clearly measurable goal of getting the bottom 5% to the top 25% in 5 years into the nebulous thing about closing the opportunity gaps by 2025, 14 years after the ASD started?

Incidentally, the ASD is being replicated around the country and even in the Every Child Succeeds Act there is a nod to it as states must come up with an intervention for their bottom 5% of schools.

The main thing for Anderson is that, like Barbic, she got her induction into Chiefs For Change which generally gains you a pretty good job as a consultant.  So nobody will be held accountable who was responsible for the ASD failure and the only people who get punished are the children and the parents who had to endure the instability that this program caused.


Posted in Uncategorized | 3 Comments

TFA’s ‘Political Arm’ LEE Reveals Their Support Of DeVos’ Main Priority

Teach For America has become a lot less outspoken, politically, since Trump became President and DeVos the Secretary of Education.  During the Obama/Duncan years, TFA would often have blog posts and tweets supporting some ed reform policy about charter schools or about standardized test based accountability.

Wendy Kopp, before she stepped down as CEO to be replaced by Matt Kramer and Elisa Villanueva-Beard, was a bit better at, at least, saying the right things to make it seem like she was not a complete ed reform zealot.  Her second book, for example, had an entire chapter about how there are no silver bullets in education.  Wendy also wrote an op-ed opposing the release of teacher evaluation scores in the newspaper, even though Arne Duncan praised them.  But Wendy often showed her extreme pro-reform side, most recently, just two months ago, in a revisionist history of the impact TFA has had in Los Angeles while failing to mention that two of the most prominent reform critics (Former board president Steve Zimmer and current union leader Alex Caputo-Pearl) are TFA alumni.  In this piece, Wendy even goes so far to quote the first judge in the Vergara case about how teacher tenure there causes inequity that ‘boggles the mind’ even though that case was overturned and that judge was completely duped by misleading statistics presented by the prosecution.

Matt Kramer and Elisa Villanueva-Beard took over as co-CEOs in 2013.  They quickly proved to be very one-sided in their understanding of ed reform.  Villanueva-Beard, in particular, would give speeches where she was quick to use phrases borrowed from prominent reformers and reports from Michelle Rhee’s TNTP.  In 2015, Villanueva-Beard went ‘full Rhee’ in a speech that evoked the title of the failed anti-union Walton funded propaganda film “Won’t Back Down.’

Eventually Kramer resigned leaving Villanueva-Beard as the sole CEO.  As sole CEO, Villanueva-Beard has not tried so hard to conceal her allegiance to the Duncan style of reform.  One example is a panel that she moderated at the TFA 25 event on the influence of Joel Klein and the TFA alumni he mentored.  And most recently, in a speech, Villanueva-Beard actually took a quote from a DeVos speech when she said that education has not changed much in the past 100 years.

Most of the time, however, TFA does try to at least pretend that they are somewhat neutral when it comes to the education wars.  They are happy to point out that 60% of their teachers are not at charter schools.  They mention that some of their members are union representatives.  But I recently came upon something that leaves no doubt where TFA stands politically, and much of it neatly aligns with the Trump / DeVos which isn’t all that different from Obama / Duncan when it comes to education.

Over the years, TFA has struggled with its public messaging.  They have tried to do various blogs and podcasts.  For a few years they had something called On The Record where they would respond to criticisms of TFA in the media.  These responses were so defensive, it actually made them look worse so they stopped adding to that site about a year ago.  It looks like they recently scrubbed the page, actually.

A few years ago they dabbled in podcasting.  A staffer named Aaron French did a podcast called ‘Education on Tap’ for 21 episodes.  Though he was reform leaning, he actually did a pretty nice job on this podcast and even had as one of the guests Jennifer Berkshire (AKA EduShyster).  When he left TFA, French actually teamed up with Berkshire to work on a podcast called Have You Heard before he left that podcast too.  I’m not sure what he’s up to now.

The ‘political arm’ of TFA is called The Leadership For Educational Equity or LEE.  The main purpose of this group, it seems, is to aid TFA alumni who want to run for public office.  LEE produces a podcast called The Leaders’ Table which I find to be very revealing.

Each episode begins with the producer Mollie Stevens who formerly worked for the failed Edison project setting up the episode.  The guests on the podcast include people from the ‘who’s who’ list of reformers like John Deasy, Kaya Henderson, and Chris Cerf and also some from the ‘who’s that?’ list, like Marc Holley of the Walton Foundation.  One thing for sure, you will not see any people critical of ed reform on the list of the 21 guests.

After the introduction, the host gives an introduction about who he is about to speak to.  I haven’t listened to every episode, maybe a third of them, and the name of this host is never mentioned as far as I can tell.  [Update: The host has been identified by a reader as Jason LLorentz, whoever that is] We know a little about him from some things he says in almost every podcast, like that he ‘survived New York City schools’, but we never get his name.  He asks the same five questions in each episode, things like “What would you tell your younger self” and “What piece of technology do you find indispensable?”

The focus of most of the podcasts I listened to had to to with charters and choice.  In the one with Nina Rees of The National Alliance for Public Charter Schools she said that all studies show that students with special needs do better at charter schools, which I found surprising since it is pretty well known that charters serve students with special needs at lower rates than their district counterparts and some charters get out of the responsibility of accepting special needs students by saying that the school is too small and doesn’t have the staffing to serve those students.

In the one with Chris Cerf, who is now the Newark superintendent and used to be the commissioner of education for New Jersey, Cerf claims that they closed down a significant number of charter schools, something that was news to me.

The host rarely challenges the guests.  He is just there to ask the questions on his sheet.  In one interesting part of the interview with Kaya Henderson, she actually said something somewhat controversial for this podcast.  She said that charter schools were not much different than district schools in the way they were educating students.  I really though the host might ‘push back’ on that or at least ask a follow up to that bombshell but instead he just kind of said ‘uh huh’ and moved on to his next question.

If you were to ask Betsy DeVos to summarize her ideas about education into one word, that word would certainly be ‘choice’ and many of the guests of this podcast would completely agree.  There was even a podcast with someone from an organization called Friends Of Choice in Urban Schools or FOCUS.  And when the Walton guy is interviewed, he also says that ‘choice’ is the number one thing that school systems need.

In my younger days I would have listened to all 21 podcasts and summarized them and transcribed key quotes.  I tried to listen to some but they were so vapid and these ‘leaders’ clearly had nothing interesting to say and knew nothing about education, I just didn’t have the stomach for it.  But I thought maybe this could be ‘crowd sourced.’  If you are a reader of this blog and you have an hour of your life to waste and never get back, maybe you could pick one of the 21 podcasts and listen to it and leave a comment on this blog post.



Posted in Uncategorized | 2 Comments

Teach To None

Teach To One is a math program sold by New Classrooms.  Currently used by about 16 schools in the country, the Teach To One program incorporates a mix of video tutorials, small group instruction, and full class instruction.  A computer creates a ‘playlist’ for each student each day which, in theory, will help the students learn more efficiently as the computer can keep track of the individual needs of each student more easily than a human teacher could.

The CEO of New Classrooms is Joel Rose who created the program, then called ‘School Of One’ when he worked for the New York City DOE under Joel Klein.  After leaving the DOE he began selling the technology around the country including, and controversially, in New York City.  I think for the first few years the city paid either nothing, or a very small amount to use Teach To One.  At one point, there were 11 schools in New York City using the program.  I actually got a chance to witness this program 5 years ago and was horrified by it.  Most of the schools using Teach To One are unhappy with it which is why currently only 5 schools in New York City use it.  Recently a school in California was so unhappy with this program that they abandoned it mid-year.

In about a week, the New York City Panel For Education Policy (or PEP), will vote on whether or not to award Teach To One a contract worth $669,000.  The New York Daily News wrote a nice summary of the issues involved in this contract.

Knowing, first hand, how awful this program is and what a waste of money this is, I delved into the test results of the schools in New York City that currently use this program.  The program is mainly for middle school math, grades 6, 7, and 8.  Since this is a program obviously geared toward getting math state test scores up, I thought the most useful numbers to look at were the 8th grade math scores at the schools involved.

The 2017 test scores have not been released yet, so I downloaded the 2016 test scores from the city’s public data site.  According to New Classroom’s 2015 annual report , there were 16 schools in the country using the program.  The five from New York city were:

Screen Shot 2017-08-08 at 4.36.41 PM

So I checked for the 8th grade test scores for these 5 schools.  To put these numbers into perspective, the citywide average for 8th graders on this test was 25% getting either a 3 or a 4.

SCHOOL % getting 3 or 4
IS 228 24.3%
JHS 88 11.1%
I.S. 381 14.4%
I.S. 49 1.8%
 I.S. M286 0.0%

The last school is no longer called Renaissance Leadership Academy, but now it is The Urban Assembly Academy For Future Leaders. and still has the M286 code number so I am pretty sure it is the same school with a new name.

So there’s a school with 0% and another with 1.8%.  And they are asking for three quarters of a million dollars to expand?  With numbers like these, surely this program has not proved itself to be effective.  But in reform marketing is much more important than results which is why this program continues to grow and to get these large contracts.

Of course this program is funded by the usual list of reform cheerleader, with over $1 million from The Gates Foundation and from the Bezos Family Foundation.  You can also see Reed Hastings on there and Joel Klein and all the other reform types.  Here is their list of funders from their 2015 report.

Screen Shot 2017-08-12 at 4.51.22 PM

One final irony is revealed in the list of the Board of Advisors:

Screen Shot 2017-08-12 at 4.51.57 PM.png

The two founders of KIPP, Mike Feinberg and Dave Levin (as well as TFA founder Wendy Kopp, who is married to another leader of KIPP, Richard Barth) are all on the board yet this Teach To One program is not in use by any of the 200 or so KIPP schools across the country.  That is quite a non-endorsement of their own product!

I actually know Joel Rose from way back.  We met at college when we lived in the same dorm in 1989.  I went to TFA in Houston in 1991 and he did TFA in Houston in 1992.  Even now, twenty five years later, I run into him time to time as we live less than a block from each other.  He was always a nice guy, pleasant, funny.  He was actually the recipient of one of my famous ‘open letters’ that I wrote years ago.  I don’t know what happens that makes some of the TFA types get involved with the likes of Joel Klein and lose their minds.  Maybe it is the opportunity to make money, I really can’t say.  But for sure if this post can be seen by the members of the PEP voting on whether or not to award $669,000 to this failure of a program, it would be great to not reward an ineffective program with any money at all.

Posted in Uncategorized | 2 Comments

The Alum-lie Spreads

Two weeks ago, The 74 published the results of The Alumni which claims to show that the graduates of certain charter school networks go on to graduate college at a rate of 3 to 5 times the rate of low-income students on average.

They say that only 9% of low-income students graduate college after six years while among the nine charter networks they studied, their graduates had college completion rates ranging from 25% (approximately 3 times 9%) and 50% (approximately 5 times 9%).

The problem with this calculation is that the charter schools are only counting students who completed 12th grade at that school (or for KIPP, 8th grade).  So if a school only has 14 graduates and 7 of them graduate after six years, it is accurate that 50% of their graduates went on to complete college, but if that cohort of 14 students was 40 students three years earlier, then their rate is really 25%.  In other words, by just counting the ‘graduates’ they get an inflated college completion rate.

In the original The Alumni article the author, Richard Whitmire, admitted as such.  He even put a comment from the KIPP network about how the other schools should use 8th grade as the cutoff so they don’t get unfairly inflated percentages.  I argued in my first post about this that 5th grade would be an even more accurate cutoff.

In a follow up article on The 74 called The Data Behind The Alumni the case is made even stronger:

The one network that insists on including students who leave the system is KIPP, which reports its college success data starting in ninth grade for students new to the KIPP system and at the end of eighth grade for existing KIPP students. YES Prep, part of the United for College Success Coalition in Texas, has promised to start calculating its college success data from ninth grade, but no figures are yet available.

All the other networks start their data set in 12th grade — and say they don’t have data that begins in ninth grade. KIPP takes a principled stand on that issue, refusing to release any results that start the tracking in 12th grade, despite the fact that it would boost its college success rate.

Within the charter community, this is turning into a hot-button issue. KIPP feels very strongly that the only honest method for reporting graduation is to start in ninth grade. In theory, a charter network could increase its college success numbers by pushing or counseling out weak students before their senior year. That would apply to any high school, not just charter high schools.

You’ve got to love the part about the other networks “say they don’t have data that begins in ninth grade.”

Also on The 74, and again to their credit, they published something by the chief executive officer of YES prep.  He brags first that the class of 2010 had a 54% college completion rate, but then, a few paragraphs later admitted about that same cohort:

When I was principal of YES Prep North Central in 2010, only 34 percent of our founding sixth-grade class went on to graduate from our campus. This unacceptably low persistence rate, a symptom of a “no-excuses” culture, needed to be addressed to align with our mission of increasing the number of students from underserved communities who graduate prepared to lead.

Suddenly the 54% turns into 54% of 35% which is just 18% college completion.

He says that from now on they are going to use the 8th grade cutoff like KIPP, though of course he should be pushing it to 6th grade based on what he just said about their huge attrition numbers.

Another interesting statistic in that follow up article is that if you want to do a more fair comparison you would want to compare not to the 9% number but to the percent of low-income high school graduates that go on to graduate college within 6 years.  To their credit, The 74 does say that this is actually 15%, to which they then say:

Even if as many as 15 percent of low-income minority students who make it through high school earn college degrees, that means these top charter networks are still doing three and a half times better.

So suddenly it goes from 3 to 5 times better to 1.6 to 3.3 times better.  This is important since surely the 3 to 5 number is the thing that is going to be remembered by people who read just the first article or one of the editorials Whitmire has published in The New York Daily News, The Wall Street Journal, and, most recently, The Hill.  There was also a report about The Alumni by another writer in The Houston Chronicle.

So after reading Whitmire’s The Hill piece which has no mention at all the issues with the calculations, I reached out to him on Twitter and we had this amusing exchange.

Screen Shot 2017-08-08 at 4.09.59 PMScreen Shot 2017-08-08 at 4.10.13 PMScreen Shot 2017-08-08 at 4.10.46 PMScreen Shot 2017-08-08 at 4.11.05 PMScreen Shot 2017-08-08 at 4.11.23 PMScreen Shot 2017-08-08 at 4.11.40 PMScreen Shot 2017-08-08 at 4.11.52 PM

And that was it.  I have no idea what his last tweet means.  I think Trump might have written it.  Why would I back off if I found flawed data.  I just don’t get it.  This is why most reformers don’t engage with me.  In any kind of debate they have with me on equal terms I absolutely embarrass them.

Posted in Uncategorized | 21 Comments

What Happened To The Math Regents? Part IV

Surely not every question on a New York State math Regents should be completely straight forward.  You want a test with a mix of some challenging questions so that students prepare for a test that is going to require a lot of thinking.  But a challenging question should not be challenging just because it is poorly worded or ambiguous.  In examining the recent Algebra II Regents, I have found way too many examples of ‘bad’ questions.

What is the appropriate number of ‘bad’ questions that should be permissible?  If by ‘bad’ you mean poorly worded, confusing, ambiguous, or mathematically inaccurate, then there should be none of those.  But I suppose it is a matter of taste how many ‘challenging’ questions there should be.  For me these extra ‘challenging’ questions belong at the end of a unit test about a topic and not as 2 point multiple choice question on the Regents, but I suppose I should expect one or two of those that, in my opinion, were too difficult for the test, if the goal is to accurately assess how well students in New York State understand Algebra II.

I’m absolutely sure that when a curve is made EVEN BEFORE THE TEST IS ADMINISTERED saying that a 30% is going to be curved to a 65%, that is a serious issue.  Making a test too hard, whether it is the good kind of hard because it required a lot of good mathematical thinking or the bad kind of hard where the question was worded poorly or ambiguous or just otherwise ‘bad’ is a mistake for a Regents.  The Regents should be reasonable and should have no curve whatsoever.  That should be one of the first goals in the creation of the test.

This is my last post about this particular exam, though I could do a similar thing for the Algebra I test and others, particularly Patrick Honner, have done a great job on the Geometry test.

Screen Shot 2017-08-07 at 10.29.28 AM

One problem with multiple choice questions is that there is no partial credit for getting the ‘second best’ answer.  So especially for a Regents test, you want to make the questions pretty clear otherwise you can’t distinguish a student who knows most of the material from a student who knows little of the material.  For this question the part about “If she plans to run the unit for 3 months out of the year” is an unnecessary twist.  I know that sometimes test makers like to put in extraneous information to check if a student can distinguish the relevant information from the irrelevant, but in this case I could see a student thinking this is relevant information, using it, and creating an equation based on this, which would already have an extra twist since it asking for cost per year instead of the typical total cost question, and still get no credit.

For a unit test on just this topic I wouldn’t have a problem with this being one out of 15 questions or something, just to make it tricky to get a 100 on the test, but for a Regents exam, it is not good test making to throw in this extra piece of irrelevant information with no opportunity for partial credit for following through on the question correctly after not realizing this.

Screen Shot 2017-08-07 at 10.29.41 AM

This is an example of how a question can be simultaneously ‘interesting’ and also a ‘bad’ Regents question.  This is a great application of the finite geometric series problem, and it is something that I do with my classes.  But it is something that works well as a twenty minute activity, not as something that you try to do during a Regents in the multiple choice section.  If students have not been exposed to this exact question, this would be a big challenge to decipher in the moment during the test.

And even if they really want to do this question, because it is ‘rigorous’ for them to apply the formula to an unfamiliar situation like this, the wording of this question is terrible.  Earlier in the test they have a probability question that takes an entire page and is still confusing.  In this one, they make this explanation way too short.  It is also ambiguous since the answer depends on when she puts the money in and when she eventually takes it out.  It seems to me that she puts the money in at the beginning of each month, but must take it out instantly after she makes her last deposit since if she waits until the end of the year, there will be more money in the account.  If I needed to give this question, I would say

“Jasmine decides to open a savings account.  On January 1st 2017 she puts $100 into the account.  The account earns 3% a month compounded monthly.  On February 1st 2017 she puts another $100 into the account.  She continues this until December 1st 2017.  Right after her final deposit, she withdraws all the money in the account.  Which expression is equivalent to the amount of money she withdraws?”

Again, this would still not be a great question since it has way too much going on in it.  It could be made more reasonable by instead of doing 12 months, have her make a deposit once a year for 12 years, still compounding, but not having to deal with the converting the annual interest rate to a monthly interest rate.

Ironically, if students were to just type the four answers into the calculator, only choices (3) and (4) would be over $1200 so they are the only reasonable answers here anyway.

So it’s an interesting problem that makes for a good classroom activity, but not good for testing the sum of a finite geometric series formula on the Regents.

Screen Shot 2017-08-07 at 10.30.20 AM

This is one of the open ended questions which was already basically tested in the multiple choice question number 16.  I think I’d rather have seen question 24 with the compound interest made into an open ended question and have some better multiple choice question in its place.

Screen Shot 2017-08-07 at 10.31.43 AM

The 4 point questions are an opportunity to make a question that is more interesting and that students can demonstrate their thinking.  This question is just an exercise in typing very carefully into a calculator.  The bigger problem is with the second part since the thing they are asking for, the down payment, is not part of the formula at all so students would have to be familiar with the idea that when you take a mortgage out you sometimes make a down payment first to cut money off the principal and to lower the mortgage payment.  This is a pretty sophisticated idea and one that is not explained very clearly in the statement of the problem.  The idea of ‘down payment’ is just slipped in there and very easy to overlook.  Personally, I read this question too fast the first time and didn’t even notice the ‘down payment’ thing and answered the question that I thought they were asking, which was what principal would lead to an $1100 payment, so I would have lost at least a point there.

I don’t know who the team is that made this test.  I’d hope that whoever assembled the team and gave them direction is held ‘accountable’ and not permitted to do this for future exams.

Making a good Regents exam is not a task to be taken lightly, so many people are depending on you to do it right.  I don’t know if New York State really cares that much about how good of a test it is.  The curve makes it that roughly the same percent of people pass this ‘more rigorous’ test, so in that way it makes politicians happier even if it is really frustrating for students and teachers who worked hard all year expecting to get a fair test.

I know a lot of people, including myself, who could single-handedly make better math Regents exams than we are currently seeing.  If whoever was responsible for putting together the team that created this round of Regents exams ever wants my advice on defining a better philosophy of what the tests should be like, or about the selection process for being on the team that writes the question, or if they want to let me look at a draft of the test to do this kind of critique before the test is finalized, I’d be willing to participate in that.


Posted in Uncategorized | 1 Comment

What Happened To The Math Regents? Part III

The New York Regents Exams are the standardized final exams for high school students in New York State.  For many years, the math Regents were very well made tests that teachers (most that I know) seemed to like.  But lately the math Regents have poorly constructed tests.  As much as I’d like to blame this on The Common Core, it actually has nothing to do with the Common Core and everything to do with whoever has been tasked to write these tests.  In two recent posts I’ve analyzed some questions on the June 2017 Algebra II Regents, and will continue with more examples here.

Screen Shot 2017-08-01 at 4.27.10 PM

Modeling real-world scenarios with exponential functions is a very interesting topic in the course which lasts for several weeks.  There are so many good ways to assess if students understand what the different numbers in the exponential equation are.  This Roman numeral thing with two Roman numerals and the four choices (I), (II), Both, Neither, is very lazy test making.  Basically they have turned this into two True/False questions that you get either get full credit for getting both, or no credit even if you know one of the two.

Roman numeral I is unnecessarily confusing.  When there is a number like a 110 in front of the e, it is generally the ‘starting point.’  So in this case it means that when t=0, or 0 years after 2010, or just 2010, the population was 110 million.  But look at Roman numeral I.  Rather than say the population in 2017 is 110 million, they say ‘The current population’ which can throw people off since ‘the current’ could mean 2010 when it was 2010.  It is a poor wording for this question which still wouldn’t be a great question even if this was worded correctly.

For Roman numeral II, I think that the ‘approximately’ can lead to extra confusion as well.  Had they made the exponent 0.039t, then they wouldn’t need to put the ‘approximately’ in at all.  But if they want to use that, they should tell what it would need to be rounded to otherwise it becomes a matter of opinion if 3.922% is approximately 3.9%.

A better question would be to have the students select from different functions to say which makes the best model after being given the starting population and the growth rate.  Or they could make four choices and ask something like “What does the 0.03922 represent?” or “What does the 110 represent?”.

Screen Shot 2017-08-01 at 4.42.36 PM

I know that many people reading this are not math teachers, but I can tell you from a math teacher’s point of view that this is an awful way to test this topic, of rational equations.  This is the only question on this entire test about rational equations, a topic that must take about two weeks or more when you include the various word problems that go along with it (like the ones where two people paint a house together).

The equation, itself, is fine for this.  But then instead of just asking for the solution set to the equation, which requires understanding that one of the two apparent solutions needs to be rejected, they do something extremely odd, which is telling the students what the first step in the solution is.  There are actually two ways to solve this problem (you could combine the fractions on the left and ‘cross multiply’), and it shouldn’t matter which way the student chooses since either way will lead to one actual solution and one extraneous solution.

Choice (3) is what’s known as a ‘distractor’ since it is something that might look tempting to someone who has a partial understanding of the topic.  They might assume that just because 0 does make one of the terms undefined, it must be an extraneous solution even though it isn’t a solution to the equation that pops up in the process.

I think in all my years of teaching this topic and in making and seeing tests on this topic, I have never seen this tested in just this way.  Perhaps if you have a test on just this topic and you have 10 questions and most of them are pretty straight forward and you want to put one like this on so that getting a 100 is tough, then I don’t have a problem with it.  But if on a statewide Regents exam, this is your only question about this unit and it is an all-or-nothing multiple choice question, this is bad test making.

Screen Shot 2017-08-02 at 9.03.50 AM

There is no reason to give the f(9)=-2 unless you are trying to force the students to calculate the d value by doing -2 = -8 + d(9-1), but why would anyone do that when it is clear that d=0.75 by just looking at the sequence itself.

Using the formula, you would get f(n)=-8.75 + 0.75(n-1) which looks a lot like choice (2) though choice (2) is not correct since there is a minus sign instead of a +.  Personally, I would have made choice (3) in the f(n)=-8.75 + 0.75(n-1) form instead of the equivalent form that they have.

For a Regents, and maybe some people will say that this is too straightforward but I think it is the right way to test this topic, what is the 1000th term of the sequence -8, -7.25, -6.5, -5.75, … or if you want to make it more difficult, in the sequence -8, -7.25, -6.5, … what term number will be 315.25, or something like that.  Since this is pretty much the only question on this topic which is a four or five day topic, you’ve got to keep it straight forward on the Regents.  Being able to answer the two questions that I propose is a true indication that you understand this formula and how to apply it.  The way this question is, it is quite possible for a student who would get something like an 80% on a full test on this topic to get zero out of two points on this one, the same as a student who would have gotten a 0% on a full test on this topic.

Screen Shot 2017-08-02 at 9.12.38 AM

They should change the function to something like f(x)=x^2+2x, or something of that complexity.  The point here is to see if the students know how to calculate the average rate of change, which I think is something they should know how to do.  But by making this unnecessarily complicated function that requires putting the calculator into radian mode even (even though I could argue that this is ambiguous, you could have degrees with a pi in them).

This turns into a question about how well the students can manipulate the calculator and a student who understands the concept of average rate of change but presses one button out of the 40 or so buttons that need to be pushed for this, will get the same amount of credit, none, as the student who doesn’t know what average rate of change is.

I think this question, more than any so far, reveals how this test making team just did not ‘get’ the idea of what a good test question is and why.

So these four questions, were actually four consecutive questions on this test.  Each could have been improved or edited to be a fine question.  If they had someone in charge who is an expert on making good test questions, maybe they wouldn’t have had to curve this test so that 30% got curved up to a 65.

Posted in Uncategorized | 2 Comments

The Alum-lie

On the heels of the latest call by the NAACP for a charter school moratorium, there has been a media blitz started by The 74 about a report called ‘The Alumni’ in which they claim that charter school graduates go on to graduate college at three to five times the rate of low income students who do not attend charter schools.

Besides being reported in The 74, it has also appeared in The Wall Street Journal and The New York Daily News.

The 74 article is written by Richard Whitmire (as is The Daily News and The WSJ Op-Ed) who is known for his biography about Michelle Rhee (haven’t heard much about her lately) and also one about Rocketship Charters (haven’t heard much about them lately).

The summary of the report says that they have tracked the students at nine charter networks and found that graduates of those charters have between 25% and 50% of those students also graduate college.  Since a commonly quoted statistic is that only 9% of low income students graduate college, these networks seem to be getting between three and five times the rate of college completion.

The major flaw in this report — and they admit this in The 74, but not in The Daily News (The WSJ is behind a paywall, if someone can read it let me know if they address it there) — is that while the 9% statistic is for ALL students who enter schools, these 25% to 50% numbers are only for the students who complete 12th grade at the schools (KIPP is an exception, they use data from students who complete 8th grade — I’ll get to that later.)

In The 74 they actually do an entire other article explaining all the issues with the data that could cause the numbers to be inaccurate, which is something I appreciated, though of course the big takeaway will still be the three times to five times numbers that will be quoted, I’m sure, for the next decade by charter zealots.

Now there is really no way to verify these claims.  We don’t have a list of students who graduated and then another list of which ones graduated college.  But there are, thankfully, a few numbers that can be analyzed in the report.  I’ve done one network so far and hopefully will be able to do more another time.

Here is a quote from The Alumni about the Uncommon Schools network:

 Uncommon Schools: For the New York–based network, the only alumni who have reached the six-year mark graduated from North Star Academy Charter School in Newark. (The alumni from its Brooklyn high school just reached the four-year mark. Of the 142 North Star students who reached the six-year mark, 71 earned four-year degrees: a 50 percent success rate.

I went to the New Jersey data page where they have the databases for the enrollment at their schools throughout the years and downloaded all the data from the 2010-2011 school year down to the 1998-1999 data.

According to their data, there were eight graduating classes that have been out of high school for at least 6 years, the classes of 2011, 2010, 2009, 2008, 2007, 2006, 2005, and 2004.

The sizes of their graduating classes were, respectively, 24, 14, 36, 19, 27, 20, 23, and 19 which is a total of 182 students, not 142.  More importantly, going back more years I found that these 12th grade classes had lost about half their students from when they entered the school as 5th graders.  The 5th grade classes for those cohorts were about 40 each.  So if they really got 71 college graduates out of those 8 cohorts, it is not a rate of 71/142=50% but instead 71/320=22%.

Yes, I know that is still ‘double’ the expected 9%, but there are other factors that might make students who apply to charters have a higher rate.  Really we can’t be sure that there really were 71 graduates, but just taking them at their word for that, it still is dishonest to claim that it is a 50% college graduation rate.

Now the KIPP network claimed a 38% college graduation rate but they claimed that the other schools had inflated numbers because the other eight networks only counted students who completed 12th grade at their schools while KIPP counted students who completed 8th grade at their schools.

Here is a quote from the article about this:

KIPP is a fervent believer that college graduation cohort data should be tracked from ninth grade — not 12th grade, the starting point that the other charter networks included in this study use.

For students who attend KIPP middle schools, KIPP tracks them when they graduate from eighth grade to ensure they are kept track of, regardless of whether they go to a KIPP high school.

For students who go to non-KIPP middle schools and start attending KIPP as high schoolers, they track them when they start ninth grade.

The problem with starting in 12th grade, argues KIPP, is that it could tempt schools to push out weaker students during high school years, thus allowing the stronger students to boost the schools’ college-going and college-completion rates.

KIPP may be right. But in The Alumni, where KIPP is the only network that is currently tracking students from ninth grade, we have decided it is important to share cohort graduation rates that start in 12th grade. What’s key to this series is learning what works in boosting that college graduation rate — lessons that could be passed along to all schools, not just charters. Moving everyone to the gold standard is the next step.

KIPP is correct that schools that only count students who complete 12th grade will have inflated scores compared to KIPP that counts students who complete 8th grade.  But what KIPP doesn’t mention is that the fairest way to make a comparison to the 9% number is to start counting at 5th grade.  KIPP actually has a pretty big attrition between 5th and 8th grade so the true ‘gold standard’ is really not used by anyone.  All the numbers are inflated.  KIPPs might be inflated less than the others, but it still is so they can whine that the others are cheating worse than they are on this statistic, but they should admit that they are doing it too, though to a lesser degree.

There isn’t a lot of detailed data included in the report, just a summary of the main findings.  And I’m not so confident we can trust the data in there anyway, but it still is interesting to see how the numbers are inflated by ignoring attrition.

Posted in Uncategorized | 9 Comments