Fat (Lie) Tuesday

Yesterday, there was some exciting news out of New Orleans.  I learned about it from a tweet by TFA co-CEO Elisa Villanueva-Beard.

Now I’ve written before about The New Orleans ‘Miracle.’  They generally have a way of twisting whatever numbers they get into something that they can celebrate.  In this case, I read the article from the Times-Picayune with the impressive headline ‘Louisiana students earn most Advanced Placement credits in state’s history’  This was also reported on New Orleans channel 8 website KNOE.

In that KNOE report it begins:

Governor Bobby Jindal and State Superintendent John White announced that the number of college credits earned on Advanced Placement (AP) exams has increased by more than 1,000 credits or 25 percent over the past year—the greatest individual increase in state history. Louisiana high school students scored high enough to earn college credit on 5,144 AP exams in 2013.

The credits earned by students who scored a 3, 4 or 5 on the AP exams are transferable to nearly any college in the nation, and Louisiana leads the country in growth of number of exams taken. 10,529 students took the AP exam in 2013, which is 4,000 more students than last year.

Oh boy.  Instantly I was able to see through the nonsense.

John White, who I once had a nice chat with while he was still in New York, chimed in.

We went back and forth a bit on this.  Though this is something that sounds good, I disagree with it.  Having kids take AP tests and getting ones and twos on them is not a very good use of time and money.  Yes, teachers should give their students ‘rigorous’ tasks — assuming that just means something that is not rote and mindless but something that really requires students to think about things.  But that can be done with material that students are ready for too.  So I don’t think that making kids take APs really accomplishes much.

In the Times-Picayune article they indicated “the percentage of students passing the exam dropped from 44 percent to 33 percent: 3,501 of the 10,529 test-takers.”  So in 2012, 41% (the article had this number wrong) of students who took at least one AP test passed at least one, while in 2013 this number dropped to 33%.  So they are celebrating, basically, that 4,000 new students TOOK the AP.  Of those 4,000 students, only 19% passed an AP.

The Times-Picayune included links to excel files where the number of test takers and test passers from each school and district are given.  From these I learned that in the Recovery School District (RSD) — the crown jewel of the ‘reformers’ which is being franchised in places like Memphis, out of 405 students who took an AP, only 24, which is 5.9%, passed one.  Now the RSD has about 40,000 students which means there should be approximately 3,000 12th graders, so these 24 students would represent eight tenths of one percent of the seniors.

Finally, I looked at the breakdown of the RSD by school.  If the RSD is the crown jewel of the ‘reformers’ then Sci Academy is the crown jewel of the RSD.  This is the ‘miracle school’ that supposedly gets students to advance three grades in one semester.  I found that only 11% of the students who took an AP at that school earned a passing grade.

Notice that the best percentage was KIPP Renaissance who evidently had 9 students passing out of 47 who took a test for a passing percentage of 19.1%.

Making kids take AP tests who are not ready for them is not a good thing to do.  There are plenty of other ways to give kids “access to rigor.”  The 6% passing rate in the RSD, I suppose, is useful information, though, for people who are looking for the truth behind the New Orleans Miracle so, in that sense, I’m glad to have this information.

They can only twist numbers so much until they eventually break.  They will have to admit, one day, that their destructive reforms don’t have the impact on achievement that they expected.



This entry was posted in Debunking, Research. Bookmark the permalink.

20 Responses to Fat (Lie) Tuesday

  1. Jessica says:

    In NOLA, many of us went to private hs. In private hs, you aren’t even allowed to take AP classes unless you are making good grades in honor’s classes. They do not just let anyone take these classes, bc they aren’t for everyone.

  2. Pingback: Fat (Lie) Tuesday | Gary Rubinstein's Blog ← NPE News Briefs

  3. Educator says:

    I know I read somewhere some study where students who took AP courses in high school, even if they didn’t get a 3, 4, or 5, did better in college, as the exposure to the rigor in high school helped prepare them more for college. I don’t have the source, but I remember reading it somewhere! I’ll try to find it.

    So in other words, increasing the number of students taking AP courses in high school (within reason), may on aggregate benefit more students. A high school’s % pass rate decreases since there’s greater access. So this typically stirs debate – by increasing access do we decrease/dilute rigor.

    If true, I think I’m OK with increasing access while sacrificing % pass rates. The question remains – exactly how much to increase access. I think every high school in the nation faces this question when figuring out how many AP classes to open.

    • Agreed. LA has clearly not seen a major success, here. But it’s not obvious they’ve seen a failure, either. It’s an empirical question whether those kids they pushed into AP classes but failed the test are worse off. I could see it going either way.

    • Well, shucks, then: let’s have high school kids take graduate courses in mathematics. None of them will actually be able to understand a word of what’s taught, none will be able to pass the classes, but “exposure to the rigor in high school” will undoubtedly help prepare SOME of them in some way for something or other.

      As long as we’re willing to accept “exposure” as equivalent to learning.

  4. Lee Barrios says:

    You read it in a study provided by the College Boards. College Boards creates and SELLs the AP tests. Get it? Read the paper carefully.

  5. Cosmic Tinkerer says:

    What a sad joke White and Jindal continue to play on LA constituents. So glad you’re onto this, Gary! Great catch!!

    Conflating testing with learning is typical for non-educator politicians and corporate “reformers.” Increased access to rigor is important in courses, not in testing. In this situation, as is often the case, the high failure rate does not prove that more rigor was best for most students involved. No doubt, KIPPsters are told, ‘never mind, just grit and bare it”. I am unfamiliar with a body of research indicating that taking and failing ANY rigorous tests improves college success rates. Education is about teaching and learning, NOT testing.

    White, Jindal and the media should be taken to task for reporting such nonsense. Thanks so much for bringing this to light, Gary!

  6. Jack says:

    At private high schools, students aren’t permitted to even take A.P. exams unless they’ve already taken and passed A.P. courses in the subject area being tested.

    Makes sense, right?

    Well, John White and the TFA-affiliated folks had the bright idea of making masses of totally unprepared Louisiana students take A.P. tests in subjects in which they’ve never even taken courses, let alone achieved a passing grade. The result: almost all of them fail miserably—probably just randomly bubbling in, and leaving the constructed response essay portion blank.

    A idiotic exercise, right?

    No, because a miniscule few did pass while the rest (96% or more, depending on the school) failed, John White, TFA CEO Ms. Villanueva-Beard, and others tout this a “major success.”

    Why would they claim such nonsense?

    Well, this is because that even though that cohort of students failed the test miserably, they were “exposed to more rigor” in theprocess of taking the test, and thus benefited from this “exposure”.

    Welcome to Louisiana!

    How about first “exposing” them to the actual courses in the subject matter that the A.P. is testing, and then see if they can achieve proficiency… and then let them take the test?

    Throwing paraplegics and quadroplegics—or even people capable of swimming, but who have never been in the water—into the deep end of a swimming pool, certainly “exposes” them to more “rigor”, but is it a good idea?

    According the John White and TFA folks like Ms. Beard, the answer is “YES.”

  7. EduShyster says:

    I’m so glad you wrote about this. We have several “high-flying” charters in Boston that make this claim regularly yet their students post dismal scores on the AP tests, particularly in science. Since AP tests are widely regarded as the best predictor of college performance wouldn’t this seem to indicate these “college prep” schools aren’t actually preparing kids for college? Our state education department seems to get this and has raised the issue with charter operators. Of course it wasn’t enough to derail their continued expansion… Journalists fall for he boosterish association of AP tests w/ “rigor” again and again though.

  8. AF says:

    Interesting. In Chicago Public Schools, schools are rated on a system in which they get a certain number of points in different categories: attendance, freshmen on track, dropout rate, etc. This determines whether a school is Level 1 (the best), 2, or 3 (the worst.) One of the categories is AP Enrollment; not the percentage of students passing AP Exams, but percent of students enrolled in AP classes. In a push by district networks to increase schools’ ratings, schools are being encouraged to open more AP courses and enroll all students in them, whether or not the students are actually ready for an AP class. As a result, many schools now are just replacing regular level classes with AP classes. Specifically, I know of a charter school a few blocks away from the HS I teach in the has ALL freshmen take AP Human Geography– that is the only 9th grade history course offered. I know my school has also felt pressure to enroll more students in AP just to get the few points we’d get by doing so. Luckily, I work at a school with a strong staff that pushes back on the network. I wish the public could attend the meetings and see just how out of touch and ratings-centered the network leaders are; the conversation doesn’t center around learning or student growth but rather asks how can we manipulate the school community and integrity to get just a few more points in category x?

    I am conflicted about the idea of AP access vs. success, though. By allowing more students access, we are giving them exposure to more rigor as well as giving them confidence and beefing up their college transcripts (assuming they do well enough) when they have to compete to get into college with students who had more opportunities than they did. However, AP classes aren’t the only way (and shouldn’t be the only way) to push rigor– it’s just more visible to others when it says AP in front of the course name. The point of AP classes was originally to get college credit, no? If you won’t succeed at getting that college credit, then why else take the course? It’s become much more about status now.

  9. Gary, I agree with a lot that you write about – but from my British perspective, I find it bonkers that students are ever *not* entered for these exams. Surely it should simply be a RIGHT for every single student that they can sit this exam, whether they fail at it or not? Limiting access to an exam that can have an impact on the next step in your life seems a very clear way of ensuring that students from certain backgrounds will never get the chance to sit it.

    In this blog you say that 4000 MORE students sat it but of that group only 19% passed. On calculation, that’s 760 students who under the previous system wouldn’t have sat the exam yet would have passed it. That can’t be right. There should not be 760 students who could have had an important qualification but were denied it for the sake of….what? The fact that the % who don’t pass it looks bad politically?

    There is a very valid argument here about the quality of the tests and whether or not they are worth students studying for. That I will give you. In England we are lucky that we have a pretty rigorous system which goes beyond multiple choice tests and includes long answers and independent coursework. But every student here has the absolute right to sit their exams, and from next year there is no choice about what ‘level’ they sit – students sit an ‘all levels’ paper to ensure that everyone has a fair chance. To me, that absolutely seems like the right thing.

    • Alison says:

      I’m not sure this is an adequate description of the exams in England for an American audience.

      First, you are talking about GCSEs, correct? These are exams for students approximately the same age as American sophomores/juniors.

      Do they know coursework, assignments and multiple exams throughout the year (“modules”) have been replaced by a single, high-stakes exam, coming after 2 or even 3 years of study?

      Do they understand what an “all-levels paper” means”? Namely, that “foundation” GCSE students used to sit exams, but only received grades as high as C, no matter how well they performed, because the exam questions differed from the higher exam? And teachers chose which exam pupils took? And this was sometimes done for the benefits of league tables, and not students?

      It would also be good to explain to American educators facing various “performance metrics” all about UK school performance (“league”) tables, how GCSEs feature, and how some students are now forced to sit exams twice to enhance schools’ positions in those tables.

      Finally, it is important to explain that A-levels (sixth-form) are closer in function to APs, insofar as fewer students take them, usually around age 16+, and UK universities look at those marks more closely than GCSEs.

      A better comparison to GCSEs might be New York State Regent exams. Only 7% of UK students fail to obtain 5 GCSEs. In NYS, 17% fail to gain a Regents diploma (also 5 exams)*.

      A smaller cohort take APs. In NYS, over the last 5 years, between 20-28% of high school graduates passed at least one AP (score:3). About 40% took an exam.

      my two cents, erm, pence

      *outside NYC, this failure rate is closer to the UK one

  10. jokefest says:

    John White’s Twitter reaction to Gary’s tweet is the standard line to justify forcing AP courses on students: it is good thing that we are “exposing them to more rigor” –as if merely labeling a class AP means it is a more challenging and worthwhile academic experience for students. It is just another of the many false premises and lazy assumptions that drive “reform” thinking.

    Jay Matthews and his “Challenge Index” have helped to foster this perception and school boards and administrators across the country have bought in to this line of thinking, too.

    Gary is right to point out that this is more about selling a certain brand of education “reform” than it is about genuinely celebrating impressive gains in the academic achievement of students.

  11. Pingback: Gary Rubinstein Deconstructs the Latest “Miracle” in Louisiana | Diane Ravitch's blog

  12. Kenobi says:

    Gary, I’m not sure where to post this, and this is sort of on topic…ok, it’s a stretch. Earlier this summer the Louisiana DOE approved $1.2m to TFA for recruitment of 25 teachers and retention of 23 others. If the local boards are paying salaries for the current TFA members, why do we need to pay TFA for retention. Then, just recently, Walmart gave $3m to bring 500+ TFAers to New Orleans. How can, on one hand, $1.2m buy 47 then on the other $3m buy 500+? I don’t need a calculator to see that those numbers don’t quite add up. Can you shed some light for me on how this actually works? I feel like we’re getting taken advantage of here.

  13. It’s not only private schools that don’t let kids take AP classes at will. My son just graduated from a public high school in Ann Arbor where, though he actually was more than qualified, he was advised (wrongly) by the guidance dept. that he couldn’t take AP Biology because he’d missed some summer assignment (he was transferring from a neighboring district that also limited who could take AP classes). While the department chair (whom he had for a couple of chemistry classes in the fall) told me that he would have been welcome in AP Biology based on his previous track record, the point is that they DO look at students’ grades in related classes before admitting them to AP classes.

    That said, I’m not all that enamored of AP in general, and I have long decried its apotheosis by people like WaPo’s Jay Mathews. I think the IB courses and tests are more impressive from what little I’ve seen of IB schools in this area, but still, I don’t think they walk on water.

    What White is doing just shows how easy it is to game the system (which is one major flaw in Mathews’ ranking of high schools based on these exams). And of course how utterly corrupt he and Jindahl are. But that’s hardly a surprise, is it?

  14. laMissy says:

    I saw this “miracle” of the AP performed here in Boston. The Mass Math and Science Initiative set up shop in my school (89% of our students were minorities). We already had an outstanding track record of well-prepared kids diligently working their way toward scores of 4 and 5 in a host of AP classes. But the goal was not to have kids do well, the goal was simply to get more kids to take AP classes. Why?

    Well, although teachers had long taught AP courses successfully, no outsider consultants were involved. Suddenly, we were inundated with “verticle alignment” workshops, AP workbooks, CD’s, mandatory extra time for teacher AP training (including Saturdays) and cash payments to students taking the tests, as well as “merit pay” to AP teachers for high scores. In other words, what had been an in-house effort to take our most talented students a step forward toward distinguishing their academic records was co-opted to make bank for test fees, materials and consultants.

    In the same time period, the College Board began to require that AP teachers write up and submit an AP curriculum to them for approval (un-reimbursed, of course), and AP training courses began to be required of teachers so that they would be “qualified” to teach those “endorsed” classes. More “ca-ching” at the cash register.

    Remember that our faculty and students had a long track record of success in this arena. Under pressure from the school department, our numbers of students taking AP classes expanded exponentially, until nearly every student was enrolled in some AP class or another. So we met the goal of more kids, but of course our percentage of high scores fell off precipitously.

    It so happened that my own kids were applying for college during this time period. I noticed that though AP had been on the lips of admissions officers of “elite” schools four years earlier for my older child, now there was little interest. Every admissions person I asked about this at competitive liberal arts colleges had the same answer – that credential has been devalued.

    Follow the money.
    See: http://www.nms.org/AboutNMSI/BoardofDirectors.aspx
    And also: http://www.hesselbeininstitute.org/knowledgecenter/journal.aspx?ArticleID=95

    Tom Luce served in Bush’s cabinet as an under secretrary of education. Failing to win the governor’s race in Texas, he was inspired to form “two nonprofit ventures that led public schools across the United States to measure performance based on standardized tests.” An early innovator (read NCLB) – all good ideas come from Texas!

  15. Pingback: Louisiana Makes Big Gains* On AP Tests | Gary Rubinstein's Blog

  16. Pingback: Louisiana Moves From 4th Worst To 3rd Worst On AP Performance | Gary Rubinstein's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s