Are 85% of TFA alumni really “working in education or careers serving low-income communities”?

In a recent podcast on ‘Getting Smart,’ they interviewed TFA CEO Elisa Villanueva-Beard.  At the 0:56 mark the host gave this series of numbers, straight from the TFA PR department, “Of the 53,000 alumni, 85% work in education or in careers serving low-income communities. That includes 1,260 school leaders, 471 school system leaders, 500 policy and advocacy leaders and 200 social entrepreneurs.”

A major critique of TFA is that the teachers use TFA as a way to pad their resumes — that they teach for two or maybe three years and then go on to law school, medical school, or business school.  If this 85% number is accurate, it would serve as a great counter to any critique of TFA that the corps members do not commit long enough.

Six years ago in a HuffPost editorial, Elisa Villanueva-Beard said that it was 80%.  Over the last few years this has grown, at least in theory, to 85% and it is something that is now quoted on the Teach For America website in the section about their impact.

Teach For America has a 28-year track record of advancing educational excellence and equity in the United States through our network of remarkable and diverse leaders working to expand opportunity and access for all children. With nearly 60,000 alumni and corps members in 51 regions around the country, our network now includes 14,000 teachers; 3,700 school principals, assistant principals, and deans; more than 300 school system leaders; 500 policy and advocacy leaders; nearly 200 elected leaders; and almost 200 social entrepreneurs. And while only one in five Teach For America corps members had plans to teach before applying to TFA, 85% of alumni are now working in education or careers serving low-income communities.

This 85% statistic, if to be taken literally, would mean that 43,350 out of 51,000 TFA alumni have a career in education or serving low income communities.

The first question to ask is:  How was this data collected?  Did TFA track down all 51,000 alumni?  Did they do some kind of random sampling?  Or is this based on their alumni survey?  I know it is based on their alumni survey since when I fill mine out there is this question:

So what they should say is that out of the people who self-selected to take the alumni survey, 85% of the responders answered yes to one or both of these questions.  There are two types of bias at work here.

The first is selection bias since this is not a random sampling — it is the people who choose to answer which likely has a higher percent of people likely to answer ‘yes’ to these questions.  We don’t really know what percent responded.  Since there is so much selection bias it probably doesn’t matter if the response rate is 60% or 70%.  But if the response rate is something like 10%, that would make the statistic even less reliable.

The other type of bias comes from the wording of these questions.  What qualifies as “relates to improving the quality of life in low-income communities”?  Since it is up to the responder to decide, we really don’t know.

The way these questions are worded, something I’m really wondering is:  What percent of college graduates, in general, would answer yes to one or both of these questions.  Since we don’t have this control group to compare to the TFA group, it is hard to know if 85% is actually impressive.

Another thing kind of ironic about the 85% number is that, in general, 85% is the percent of corps members who don’t quit during their first two years of teaching.  Since this statistic is just about ‘alumni’, those people who quit are not included and that further skews the numbers.

In the past six years TFA has found a way to say that this number has grown from 80% to 85%.  Who knows what they will be saying it is six years from now.  Whatever it is, people should know, as TFA absolutely does, that this number is complete nonsense, something that it would not even be fair to call a half-truth.  If TFA continues to use this 85% number and they didn’t realize it was bogus before, they know now and they will be purposely using something they know is misleading at best.

There are two ways to get more accurate data.  One is for TFA to try to fully account for all 51,000 alumni.  This is a difficult thing to do and something that TFA is not going to invest the time and money into since it can only make that number more accurate and make them look worse for it.  The other way is to do a random sampling where they pick about 10,000 people who started with TFA out of the about 65,000 alumni plus quitters.  Then they would have to track down all 10,000 of those and that would be a pretty good random sample, I think.  They won’t be willing to to this either.

So I’ve decided to do a little crowd sourced experiment.  In the early days of TFA, they sent me an alumni directory with the name of every corps member from 1990 to about 2000.  So here is my experiment.  There were 522 corps members in the first cohort of 1990.  I had WolframAlpha picke 100 numbers between 1 and 522.

Then I alphabetized the 522 corps members from 1990 and assigned each a number and then picked the 100 people who corresponded with the 100 numbers that were randomly chosen.  This is a true random sampling and it is about 20% of the total population which is a pretty good size sample actually.

215	Holifield, Erin
278	Lienhard, Bill
111	Darby, John
308	McGlone, Thomas
19	Arsuaga, Maritere
461	Tan, Chin
425	Simes, Jeffrey
115	Davis, Geoff
253	Koo, Chiray
171	Gomez, Carlos
184	Groom, Ileetha
392	Rivera, Richard
370	Polen, Michael
277	Lewis, Kimberly
299	Martinez, Jane
271	Lerntouni, Tank
113	Davis, Andrew
60	Brown, Daryl
320	Miller, John
401	Sabin, Caroline
3	Abell, Jennifer
273	Levine Grimaldi,
72	Can, Kristen
95	Cobb, Kendall
265	Lay, Corey
384	Rehl, Michael
281	Livingston, Therese
485	Wade, Andrea
122	Dennis, Terrence
379	Ramsey, Lukman
512	Winiecki, Marc
236	Jones , Stephanie
397	Roth, Sharon
429	Skolaslci, Renee
499	Wickliff, Derek
364	Phoa, Cynthia
193	Hamilton, Donna
413	Seligman, Miklci
95	Cobb, Kendall
125	Dineen, John
467	Thompson, Julia
56	Brooks , Daniel
361	Peterson, Lisa
99	Collins, Philip
502	Willey, Kristin
136	Ebby, Rachel
514	Wright, Ernest
41	Boatright, Laura
194	Harrigan, Lisa
505	Williams, Brandi
501	Wilkinson, Wendy
173	Gonzales, Emilio
373	Price, Wendy
20	Aumou, Elizabeth
367	Plaman, Kathryn
180	Grado, Danielle
399	Ruvoli-Gruba, JoAnne
444	Steensland, Lara
478	Utley, Stephen
57	Brooks, Hoff
27	Beck , John
517	Yudell, Michael
384	Rehl, Michael
185	Guerrero, Scott
116	Davis, Lorna
350	Palazzolo, Rayann
513	Wolf, Matthew
467	Thompson, Julia
203	Heitmann, Noel
205	Hendricks Richman, Susan
68	Bushnaq, Faith
109	Crean, William
129	Donoho, Lori
186	Gulling, Egypt
339	Nicholas, Robert
210	Heyl, Densie
257	Kruse, Jennifer
137	Edge, Kecia
204	Held, Robert
521	Zimmerman, Andrea
313	McPherson, Maria
224	Israel, Todd
63	Brown, Michael
106	Cox , David
251	Klender, Kimberly
336	Newkirk, Jennifer
64	Buckley, Michael
333	Nagler, Mary
25	Basich, David
181	Graham, Elliott
299	Martinez, Jane
146	Eppolito, Veronica
201	Haynes, Michael
216	Holmes, Tiffany
464	Taylor, Olu
437	Snyder, Christina
460	Tabb, Kathleen
226	Jacobs, Sandi
297	Marie!, Kecia
237	Jones, Brian

OK, now I did not then go through and start tracking down each of these 100 people.  There was a time a few years back where I may have had the energy for such a project.  But, with six degrees of separation and all that perhaps some readers will know some of these people who are all about 51 years old right now and graduated college in 1990.  Or maybe readers can pick someone off the list at random and write a comment, something like “297 is a banker at Wells Fargo” with some kind of link to prove this.  I haven’t really thought this through fully, but if these 100 people can be researched, it would be interesting to see if approximately 85 of them are “working in education or careers serving low-income communities.”