In a recent podcast on ‘Getting Smart,’ they interviewed TFA CEO Elisa Villanueva-Beard. At the 0:56 mark the host gave this series of numbers, straight from the TFA PR department, “Of the 53,000 alumni, 85% work in education or in careers serving low-income communities. That includes 1,260 school leaders, 471 school system leaders, 500 policy and advocacy leaders and 200 social entrepreneurs.”
A major critique of TFA is that the teachers use TFA as a way to pad their resumes — that they teach for two or maybe three years and then go on to law school, medical school, or business school. If this 85% number is accurate, it would serve as a great counter to any critique of TFA that the corps members do not commit long enough.
Six years ago in a HuffPost editorial, Elisa Villanueva-Beard said that it was 80%. Over the last few years this has grown, at least in theory, to 85% and it is something that is now quoted on the Teach For America website in the section about their impact.
Teach For America has a 28-year track record of advancing educational excellence and equity in the United States through our network of remarkable and diverse leaders working to expand opportunity and access for all children. With nearly 60,000 alumni and corps members in 51 regions around the country, our network now includes 14,000 teachers; 3,700 school principals, assistant principals, and deans; more than 300 school system leaders; 500 policy and advocacy leaders; nearly 200 elected leaders; and almost 200 social entrepreneurs. And while only one in five Teach For America corps members had plans to teach before applying to TFA, 85% of alumni are now working in education or careers serving low-income communities.
This 85% statistic, if to be taken literally, would mean that 43,350 out of 51,000 TFA alumni have a career in education or serving low income communities.
The first question to ask is: How was this data collected? Did TFA track down all 51,000 alumni? Did they do some kind of random sampling? Or is this based on their alumni survey? I know it is based on their alumni survey since when I fill mine out there is this question:
So what they should say is that out of the people who self-selected to take the alumni survey, 85% of the responders answered yes to one or both of these questions. There are two types of bias at work here.
The first is selection bias since this is not a random sampling — it is the people who choose to answer which likely has a higher percent of people likely to answer ‘yes’ to these questions. We don’t really know what percent responded. Since there is so much selection bias it probably doesn’t matter if the response rate is 60% or 70%. But if the response rate is something like 10%, that would make the statistic even less reliable.
The other type of bias comes from the wording of these questions. What qualifies as “relates to improving the quality of life in low-income communities”? Since it is up to the responder to decide, we really don’t know.
The way these questions are worded, something I’m really wondering is: What percent of college graduates, in general, would answer yes to one or both of these questions. Since we don’t have this control group to compare to the TFA group, it is hard to know if 85% is actually impressive.
Another thing kind of ironic about the 85% number is that, in general, 85% is the percent of corps members who don’t quit during their first two years of teaching. Since this statistic is just about ‘alumni’, those people who quit are not included and that further skews the numbers.
In the past six years TFA has found a way to say that this number has grown from 80% to 85%. Who knows what they will be saying it is six years from now. Whatever it is, people should know, as TFA absolutely does, that this number is complete nonsense, something that it would not even be fair to call a half-truth. If TFA continues to use this 85% number and they didn’t realize it was bogus before, they know now and they will be purposely using something they know is misleading at best.
There are two ways to get more accurate data. One is for TFA to try to fully account for all 51,000 alumni. This is a difficult thing to do and something that TFA is not going to invest the time and money into since it can only make that number more accurate and make them look worse for it. The other way is to do a random sampling where they pick about 10,000 people who started with TFA out of the about 65,000 alumni plus quitters. Then they would have to track down all 10,000 of those and that would be a pretty good random sample, I think. They won’t be willing to to this either.
So I’ve decided to do a little crowd sourced experiment. In the early days of TFA, they sent me an alumni directory with the name of every corps member from 1990 to about 2000. So here is my experiment. There were 522 corps members in the first cohort of 1990. I had WolframAlpha picke 100 numbers between 1 and 522.
Then I alphabetized the 522 corps members from 1990 and assigned each a number and then picked the 100 people who corresponded with the 100 numbers that were randomly chosen. This is a true random sampling and it is about 20% of the total population which is a pretty good size sample actually.
|236||Jones , Stephanie|
|56||Brooks , Daniel|
|27||Beck , John|
Hendricks Richman, Susan
|106||Cox , David|
OK, now I did not then go through and start tracking down each of these 100 people. There was a time a few years back where I may have had the energy for such a project. But, with six degrees of separation and all that perhaps some readers will know some of these people who are all about 51 years old right now and graduated college in 1990. Or maybe readers can pick someone off the list at random and write a comment, something like “297 is a banker at Wells Fargo” with some kind of link to prove this. I haven’t really thought this through fully, but if these 100 people can be researched, it would be interesting to see if approximately 85 of them are “working in education or careers serving low-income communities.”