The Missed Opportunity Myth

Before Michelle Rhee was a board member for Miracle-Gro she was the founder and CEO of StudentsFirst.  Before that, she was Chancellor of Washington D.C. schools from 2007 to 2010.  Before that, she was the CEO of The New Teacher Project.

And even though Rhee is not a public figure anymore in education, she continues to influence education policy through The New Teacher Project which has since changed its name to TNTP.  TNTP puts out slick papers that it calls research but is really propaganda disguised as research.  Their first one was called ‘The Widget Effect’ which laid out the case for replacing salary schedules with a system based on merit pay based on statistically inappropriate analysis of standardized test scores.

And over the years they have put out other papers with clever titles like ‘The Irreplaceables’, ‘Rebalancing Teacher Tenure’, and ‘Teacher Evaluation 2.0.’  These papers are often quoted by ed reform propaganda sites like The74 and Education Post.

One of their most recent papers is called ‘The Opportunity Myth.’  Its central thesis is something that reformers love to use in their teacher bashing arguments, which is that too many teachers shortchange their students by having low expectations for them.  The work they assign is not challenging enough and since students always rise to the challenge of whatever you assign to them, these teachers are negligent in their duties.

So TNTP observed 1,000 lessons in five school districts and analyzed 5,000 assignments and 20,000 student work samples.  They concluded that the work that the students were assigned was usually inappropriate for their grade level.  Here is one of their graphics from the paper.

Screen Shot 2019-07-30 at 3.05.41 PM

So 75% of the time students are working on things “that were not grade appropriate.”

My first question about this study is:  If a teacher is teaching a 5th grade class where the students are behind and only at a 3rd grade reading level, but you give them material at a 4th grade level, are you someone with low expectations or high expectations?  I would say you are someone with high expectations.  But this research paper would conclude that this teacher was someone with low expectations because the 4th grade reading level is lower than the grade the students are in.  This is the main flaw with the misleading conclusion so happily quoted by the reform propaganda sites.

TNTP actually put up samples of the assignments they rated on their website so I took a look at how they rated some math assignments to see if I agreed with their ratings.  I was not surprised to see that when it came to 8th grade math, the TNTP raters had no idea what they were talking about.

This part is a bit ‘mathy’ but I will do my best to explain the context of this assignment so if you are not a math teacher you can still get the idea of how misguided TNTP is in their rating system.

One of the most important concepts is math is something called ‘slope.’  It is first taught in 7th or 8th grade and it is weaved into all the following courses and actually comprises a large part of Calculus.  Informally speaking, the slope of a line is a measure of how steep it is.

In the picture below, the segment AB has a slope of 1, the segment CD has a slope of 2, and the segment EF has a slope of 1/2.

Screen Shot 2019-07-30 at 3.20.40 PM

A slope of 2 means that for every unit you move to the right, you have to move up 2 units.  For segment CD, to get from C to D you would move two units right and four units up.  So if you only moved one unit to the right from C, you would move two units up to get to the point on the line.

One of the main ways of testing to see if students know slope is to give a diagram with a line segment on it and ask them to compute the slope.

Screen Shot 2019-07-30 at 3.27.55 PM

The slope of this line segment is:

A) 1/3    B) 3    C) -1/3    D) -3    E) Slope is not a ‘thing’

If you said ‘B’, you are correct.  To get from (2,1) to (5,10) you have to go three units to the right and nine units up.  So if you only went one unit to the right, you would only go three units up.

There is also a ‘formula’ for slope:

Screen Shot 2019-07-30 at 3.33.54 PM

There are a lot of way to test to see if a student understands the concept of slope.  If you just have the students calculate slope over and over there is a chance they they don’t really ‘understand’ slope but that they just memorized a formula.  Still, if you have a student who cannot calculate the slope of a line or line segment when the coordinates of two points are known, then that student surely does not understand the concept of slope.

Now (and I know I’m losing readers here by the minute — but you have to have some background so you can understand why I contend that TNTP is not qualified to judge the quality of a math assignment based on a lesson about slope) you may not be surprised that if there are three points on the same line segment and if you calculated the slope by picking any pair of those points, you would get the same answer.  In this case, you will get a slope of three no matter which two points you choose.

Screen Shot 2019-07-30 at 4.06.36 PM

Informally speaking, if the slope of the segment from (2,1) to (4,7) was different from the slope of the segment from (4,7) to (5,10) then it wouldn’t be a straight line.

When you go to the TNTP page where they show examples of activities and how they rated them, they have this example for a lesson about slope.


Math teachers I’m sure will agree that this is a good set of questions.  The first two questions are based on a graph and the student knew to draw a vertical and horizontal line which shows a conceptual understanding beyond just plugging numbers into a formula.  In the third question, the data is presented as a chart instead of on a graph, giving an opportunity for students to get comfortable with multiple representations and this question has six different ways to calculate the slope so there is opportunity for students to discuss that.  In the fourth question the student creates a chart from the data and then does it like the third question.  One of the answers is a positive integer, one is zero, one is a negative fraction, and one is  a positive fraction so all different sorts of answers are covered.  This is a fine example of practicing the skill of calculating the slope as data is presented in different ways.

But according to the ‘experts’ at TNTP, the teacher who created this assignment was guilty of low expectations.

Screen Shot 2019-07-30 at 4.46.20 PM

TNTP explains why they made this judgement, but to understand why their rationale is nonsense, I’m going to have to take you through some more background about math and about the common core standards.

There are math standards from K to 8th grade and then standards for six different aspects of high school math.  The math standards for a grade, like 8th grade, are pretty short, taking up maybe 10 pages if you print them out.  States will take those standards and turn them into lesson maps where there will be 150 or so subtopics based on the standards.  The standards are not very thorough, actually.  There are big gaps in them that anyone who is a practicing math teacher would know need to be filled in.

So for 8th grade math, the concept of slope is only mentioned twice.  Here is the text from the two mentions:

Screen Shot 2019-07-30 at 4.55.26 PM

So there is no mention in the standards at all of mastering the skill of actually using the slope formula to calculate the slope of a line defined by data in various representations.  Any real teacher would know that this is an extremely important skill that you would spend several days on as it will come up for the next 4 years in many important math units including several months of Calculus.  And the teacher who made this assignment that was deemed ‘weakly’ aligned to the standards was aware that it would be negligent not so spend some time practicing with the slope formula.

So the TNTP people gave this explanation for why this was ‘weakly aligned’:

Screen Shot 2019-07-30 at 5.09.19 PM

OK, so you likely have not been teaching math for 27 years like me so this explanation does look like something that they put a lot of thought into.  So I’m going to say a few things about their rationale for bashing this assignment.

First of all, this is not the standard that the assignment was trying to address.  This assignment is addressing a standard that is not explicitly stated because it is so obvious to everyone (except the TNTP raters) who needs to understand what sorts of things are needed in teaching the concept of slope.

Secondly, this standard 8.EE.B.6 is perhaps the most unnecessary and, sorry to use this language, stupidest standard I can imagine for this topic.  Remember this picture I put up before?

Screen Shot 2019-07-30 at 4.06.36 PM

They are saying that a better way to justify that if you have a straight line then the slope of any segment on that line will have the same slope is to apply a much more difficult concept to master from Geometry, the idea of similar triangles.

Similar triangles are two triangles where one is like a ‘zoomed in’ version of the other.  So they are not necessarily congruent triangles because there is a little one and a big one.  What makes them similar is that they have the same angle measurements.

So these two triangles are similar since they both have angles of about 72 degrees, 90 degrees, and about 18 degrees.

Screen Shot 2019-07-30 at 5.29.07 PM.png

Notice that the sides of the big triangle are double the sides of the smaller triangle.  We say that the corresponding sides are in proportion and this will always happen with similar triangles.  It also works the other way around — if we know the corresponding sides are in proportion, then the triangles are similar so all their angles will be equal.

Using this concept of similar triangles, we can look at that straight line problem with three points on it in a different way (though this would not be advised since it is unnecessarily confusing and really won’t offer any truly useful insight into this topic at this point in 8th grade).

Screen Shot 2019-07-30 at 5.37.20 PM

Adding some lines to make two triangles, you’ve got the big one with a horizontal side of 2 and a vertical side of 4 and you’ve got the small one with a horizontal side of 1 and a vertical side of 2.  You also have that right angle in the bottom right corner of the triangle.  If two sides are in proportion and the angle between those two sides is the same then these are similar triangles.  And if they are similar triangles, the angles BAC and ECF would need to be the same too and if those angles are the same then the line ACF would have to be a straight line.

This is not something I would ever teach to 8th graders unless my principal said that I’ll get fired for not following the common core standards like they are the ten commandments.  It is maybe an interesting curiosity that I could mention or not mention, depending on the situation.  It certainly doesn’t belong on a short list of standards, like the common core standards.  And for TNTP to pan this assignment because they don’t understand that a good teacher knows it would be negligent to try to push this ill advised standard if they can avoid it, and that a good teacher knows that even though it doesn’t say that the slope formula is something that should be practiced, it certainly is something that should.

I can probably do a similar analysis for every assignment that TNTP said is weakly aligned to the standards, but I won’t torture myself or the people who like to read this blog with any more of this.

TNTP is the legacy of Michelle Rhee.  She, a bit like Voldemort in the early Harry Potter books, is out of sight right now but she continues to influence education policy through her intermediates and it is important to show that making a fancy paper that looks like real research and is quoted by The74, Education Post, and the TFA blog does not mean that the researchers are qualified to analyze the data they collect.



This entry was posted in Uncategorized. Bookmark the permalink.

16 Responses to The Missed Opportunity Myth

  1. Pingback: TFA and TNTP don’t understand how to teach math | GFBrandenburg's Blog

  2. anonymous says:

    Gary I generally disagree with 99% of what you write, but your point here about high/low expectations and “grade level” expectations cannot be overstated and is absolutely correct. Someone who is teaching 5th grade to students who are reading at a 3rd grade level would – as you already pointed out – be holding their students to high expectations by having them read at, say, a 4th grade level. The people who go on and on about what is “grade level appropriate” have zero concept of how any good teacher approaches curriculum design – the curriculum should fit the students. Well-stated and I’m glad you called this B.S. out.

  3. eanelson2014 says:

    Gary –
    Thank you for this post.
    My work involves preparing students for the math required in the physical sciences and engineering. That’s a subset of math, so I would not be one to be confident in writing the grade 8 CCMS standard is stupid from a mathematical perspective, though it does look that way to me. You have credentials on that I don’t, and your explanation of what is and is not important to me rings true.
    And that matters because discussion of whether the CCMS can be improved upon has largely been suppressed. Either the wealthy foundations who backed the un-piloted CCMS have bought the silence of the groups who are supposed to ask hard questions about education, or the staff at those groups lack the knowledge needed to ask hard questions about math instruction.
    An area where as an educator I do a lot of reading is the research by cognitive scientists on what the student brain can and cannot do. Comparing the science to the CC math standards, it is clear that on topic after topic, the CCMS ask students to solve problems in ways their brains simply cannot do.
    I have a paper on those issues posted at (check your PDF downloads).
    The CCMS must work both mathematically and cognitively. On the cognitive side, they often don’t. I don’t blame the 3 drafters of the CCMS for that – in 2009 the science of how the brain works was less clear than became by 2012. But no review of the CCMS in light of new science has been done by groups with credentials since 2009. That’s neglect of young people. And perhaps abuse.
    The problem is not just TNTP. There’s a whole infrastructure that’s been developed (including EdReports) that promote curriculum alignment with a CCMS. The problem is, science says the CCMS badly in need of change to align with how science says math needs to be taught to work cognitively.
    By failing to review the 2009 CCMS in light of subsequent uncontested science, the powers that decide math standards and curriculum are abandoning their duty to serve the interest of children and the nation.
    Keep asking hard questions, Gary. For the kid’s sake.
    — rick nelson

  4. Dee says:

    Moreover, you’ll be hard pressed to find a state end of year test that addresses that standard in the exact way couched by TNTP or the CCSS for that matter. The furthest that it will go is is diagram with three points on a line and more than one slope triangle. They won’t actual ask a student in eighth grade to prove that the geometrical interpretation fits the algebra and all the other representation forms.

    Rather, you can make an argument that the teacher is preparing students for the HS standards on multiple representations. I will freely admit that without a clear set of investigatory/discussion questions, it is hard to see if this teacher set the students up for that success or whether they were building procedural fluency only. Fluency may still have been the right teacher play in the ways your post described even in the absence of those questions which were not on the printed assignment. TNTP probably won’t tell us what they actually observed the teacher DO, so we can only guess at that.

    Finally, I’d be remiss is I didn’t mention that technically the CCSS allows fluency as a part of its mathematical practice in MS. (Oddly, this is technically not “allowed” in CCSS in HS…. don’t get me started.)

  5. Pingback: Gary Rubinstein: The Missed Opportunity Myth | Diane Ravitch's blog

  6. There is a deeper back story to this concept, that goes back to an organization called EdTrust. This group was involved in the crafting of NCLB. When I worked in Oakland public schools, back around 2001, they hired EdTrust to bring in their model of systemic reform, which they called Standards in Practice. The concept was very similar to this TNTP analysis. The idea was that the reason many Oakland students were working below grade level was that teachers were delivering lessons below their grade level. The method to remedy this was to place every teacher in the District into groups that met every month. In these groups, teachers were asked to bring in their lesson plans. Then those plans would be analyzed to identify how they were below grade level, and the state content standards would be used to revise or create new lessons that would be properly aligned to the grade level, as specified by the state standards.

    Can you guess what happened? For some reason, teachers lacked enthusiasm for this process. Few brought forth their lessons to be dissected in this way, and after a few months attendance at these mandatory sessions fell off, and the program was abandoned. The concept was not successful in “raising expectations” for students.

    This is what EdTrust now has on their site on the concept:

    • jlsteach says:

      Anthony here’s my question for you: do you think that there are some teachers out there who have lessons that don’t meet the grade level content? If so how many do you think fall into this category? 10%? 20%?

      I’m sure you going to come back and say that teachers may have a lesson it’s out of nowhere standard because the students are at that level of learning. But here’s what I’ve seen in my numerous years of experience in education: Particularly in mathematics, I think teachers time and again lowering the expectations of students. In an algebra one course a student may struggle with solving an equation, And instead of doing a mini lesson for review the teacher will give out multiple worksheet and spend multiple days on the one concept. Kids fall further behind in they’re learning.

      Look, you made it interesting point about and trust and their work. And yes no teacher wants to openly amid that they’re having lessons that are below grade level standards. And as Gary pointed out I don’t fully agree with the TNTP report or its methodology. But that doesn’t mean that the problem doesn’t exist. So here’s a challenge to you and others want to point out the negatives. First, admit that this does happen. And second, suggest potential solutions. Here’s a small one that I recommended to some of the teacher leaders I’ve worked with. I created a simple document that ask teachers to justify any changes the van we’re making to a given lesson as part of a curriculum. Justifications can’t just be a simple as “my kids don’t get it” Rather they would have to specifically discuss what the students and you get and how they were addressing the lack of prior knowledge.

      • eanelson2014 says:

        You claim teachers should be made to explain in writing what they will do to vary the standard curriculum to fill in missing student background knowledge. That’s a paperwork nightmare for teachers in most classrooms, but especially in high poverty areas.

        In a suburban 6th grade classroom, most of the children have likely been together in school since first grade, but different teachers may have covered different topics. Should not teachers be free to assess those difference in background knowledge and adjust the “standard curriculum” accordingly, without jumping through the hoops you call for.

        For teachers in low income neighborhoods, the situation is more challenging. Studies have found what teachers quickly note: Low income kids have much higher than average transiency. They have likely been in many different textbooks, curricula, and districts. The fragmentation of their instruction will have left holes in their knowledge that are far larger than those of the average suburban student. A responsible teacher in these classes has no choice but to slow down to try to fill in missing topics, due to circumstances beyond teacher and student control.

        This “simple document” you recommend to “teacher leaders” that teachers must fill out to change a school’s “one size fits all” curriculum, when students come from many different math backgrounds — based on my years of experience, I suspect at at least some schools, teachers will hear about this as a requirement at their first week August faculty meetings.

        What do you think will be the impact on teacher attitudes at the start of the school year?

      • jlsteach says:

        So a few thoughts…first of all I’ve taught in inner city schools. I’ll never forget my first year at a school that just opened. A class of 10th graders in Alg 2 (a year ahead)…as I soon learned these students were essentially given grades in Alg 1 and Geometry.

        You say the responsible thing to do it slow down…so then what happens the next year, or the year after that? Or even worse when they go to college and have to pay for remedial math courses worth no credit because their Alg 2 teacher slowed down so much. Then what?

        As for my document – I’ll use this analogy. I worked with preservice teachers. We expected their lessons plans to be very detailed. Way more than normal plans. Why? So they could examine their thinking, etc. the hope is that when the became full time teachers some of the items within their lesson plans would be ingrained already. For example If a plan had a space for prior knowledge they would have already thought about that..

        As for the document – the hope is similar. Teachers could come to a point where they are internally thinking through reasons for changes

        As for how teachers feel – two thoughts. One I think some teachers would welcome the chance to explain their changes post observation instead of a principal or AP only focusing on why they weren’t doing the exact curriculum. And two, I’d be very leery of a teacher who was unwilling to justify changes to a curriculum. What is the teacher trying to hide? Or did the teacher really not think through things?

      • eanelson2014 says:

        So when you got your class of Algebra 2 that hadn’t really had Algebra 1, didn’t you need to teach them Algebra 1 before they could learn Algebra 2?

      • jlsteach says:

        So yes, there were some concepts from Algebra 1 that I had to incorporate into my Algebra 2 lessons because they may not have seen things in their earlier class. But that is VERY different from teaching Alg 1 the entire year but calling it Algebra 2. I am not saying that teacher may not need to incorporate prior knowledge into their lessons. What I am saying is that many teachers (particularly in math courses) feel that they have to do LOTS of review/entirely review – this hurting students in the long run

  7. jlsteach says:

    Gary, I responded to some of this on Diane Ravitch’s blog but thought I would respond to you here. I believe that you are only partially correct in that pointing out the misalignment between the problem in the standard you’re also missing a viable point in discussing the types of questions around slope. I too am a mouth educator with over 20 years of experience. And in looking at these problems about slope I think they are the types of problems I would’ve used in my first or second year of teaching. They have students understand rise over run. They also have a memorize the formula and use it.

    But I would disagree with you that these problems are good problems. For example on the large line which students have to find the slope, I think a better question would have been to ask them to identify a point in between the two end points on the segment. You rightfully point out that on a line segment any two points on the segment will have the same slope. So why not test that idea as well which is a bit more related to the conceptual understanding of slope and not just plugging into a formula.
    You rightfully point out that on a line segment any two points on the segment will have the same slope. So why not test that idea as well which is a bit more related to the conceptual understanding of slope and not just plugging into a formula.

    Taking a step even further, I asked students to find two other points that we create a segment with the same slope as the original segment (the one with a slope of 1/3)

    As I stated in the other blog post, I think both the opportunity math and you in these cases focus more on what’s wrong and what possibly could be right with the message. In the case of the report, you rightfully point out how the problem is not really a line to the standards it could address. But in your case, do you want evening knowledge have some teachers do you have lower expectations of students as think the problems on slope are good

    • garyrubinstein says:

      Those are some great thought provoking questions. They would serve well as a motivation to slope or as a higher level assessment. Still, TNTP would say that since they don’t have the words ‘similar triangles’ they would not be aligned with that standard.

      • jlsteach says:

        Maybe so. But you have a chance to show how you can have more rigor AND show TNTP is wrong in its analysis. You only do one of them which limits the emphasis of your argument

        Also i would also see if there are Alg 1 standards that may be better applicable.

  8. eanelson2014 says:

    Gary –

    May I ask two questions on which I would appreciate hearing the views of you and your readers?

    Q1: The common core standards ask first graders to solve multi-step addition and subtraction problems, such as 13 – 4 = 13 – 3 – 1 = 10 – 1 = 9 (CCMS 1.OA.6). but do not ask students to “know from memory” any addition facts until 2nd grade. How likely is it they will be able to reliably do such calculations?

    Q2: The common core standards do not at any point ask students “know from memory” any subtraction or division facts. The CC authors have said the intent was that subtraction and division be mentally calculated instead of automatically recalled. Should students be asked to recall with automaticity the basic subtraction and division facts?

    — rick nelson

    • garyrubinstein says:

      I’m not an expert in early childhood education, but I do have an 8 year old and an 11 year old and I definitely want them both to just ‘know’ something like 13-4=9. I think the idea of 13-4=13-3-1=10-1=9 is a nice thing for kids to think about but that should eventually turn to what seems like an intuition. It’s not just a fully memorized thing but something that does have a logic to it. So when an adult is asked 13-4 and they say 9 immediately, different things go through different people’s minds so there is not just one way to know this. If these ‘number sense tricks’ lead to a deeper understanding that helps them eventually know the facts quickly, then I am all for them. But if they are done at the expense of quickly being able to calculate, then they have not served their full purpose I think.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s