Embarrassing Paper from the University of Arkansas by Thomas Ultican
Masquerading as a serious education research paper, “Making it Count: The Productivity of Public Charter Schools in Seven U.S. Cities” is little more than a flawed propaganda screed. The authors from the University of Arkansas College of Education and Health Professions claim the superiority of “public charter” schools over traditional public schools (TPS) by employing thoroughly discredited methods. They repeat the same data manipulation malfeasance that has been debunked multiple times over the past decade.
The paper was produced by the School Choice Demonstration Project which resides within the Department of Education Reform. That department was established by the University Arkansas’s College of Education and Health Professions in 2005. Arizona State Professor Eugene Glass commented that the department is, “one of the strangest I have ever seen.” Glass reports that the Department of Education Reform was made possible by a $10 million dollar gift from the Walton Family Foundation plus another $10 million from the University of Arkansas’s matching gift program.
Subsequent to the big 2005 grant, Walton Family Foundation tax records (EIN: 13-3441466) reveal more than $30 million in grants to the University of Arkansas Foundation and school administration.
Inappropriate Numerators and Specious Denominators
The people at the School Choice Demonstration Project insist on using “public charter school” and “traditional public schools” to differentiate between charter schools and public schools. Charter schools are private companies that have a government license to provide schooling. Public schools are controlled by elected public representatives and must accept all students. Charter schools are not required to meet all public school rules of operation. Referring to charter schools as “public charter schools” and public schools as “traditional public schools” reveals a strong bias.
The authors attempt to compare testing results per $1,000 dollar funding between charter and public schools. They use manipulated National Assessment of Education Performance (NAEP) data. Another Walton Family creation, the Center for Research on Education Outcomes (CREDO) at Stanford University, used a “virtual twin” scheme to create the comparison data employed.
Business writer Andrea Gabor states that CREDO studies which compare charter schools with public schools start with two key assumptions “A) That standardized-test scores are an adequate measure of school quality and B) that creaming in charter schools does not exist.” Many studies dating back to the eugenics movement debunk standardized-testing for mistaking correlation versus causation and for not being able to compensate for the problem of error. A glance at state enrollment records confirms that charter schools practice creaming.
The CREDO method does not compare charter school test performance to actual public schools; rather it creates mathematical simulations. They use arithmetic magic to create the “virtual twin” schema that stands in for a public school students. This method introduces several biases that advantage charter schools.
The “Making it Count” paper gets its “inappropriate numerators” from CREDO. It is also where they get the theory for their wild claim that “public charter schools in our sample would produce $487,177 more in lifetime earnings than the TPS.”
The denominators in the paper’s calculations are based on school spending per student. The data used comes from a 2020 paper also produced by mostly the same authors; “Charter School Funding: Inequity Surges in the Cities.” In this update of a previous report on the 2013-14 school year, they state, “Our most recent report updated that analysis by drawing upon data from the 15 metropolitan areas for the 2015-16 school year.” In other words, no methodology changes just an update.
One of America’s leading authorities on education finance, Rutgers University’s Bruce Baker, reviewed the 2013-14 report for the National Education Policy Center. In the introduction, Professor Baker wrote,
“A district’s expenditure can be a charter’s revenue, since charter funding is in most states and districts received by pass-through from district funding, and districts often retain responsibility for direct provision of services to charter school students—a reality that the report entirely ignores when applying its resource-comparison framework. In addition, the report suffers from alarmingly vague documentation regarding data sources and methodologies, and it constructs entirely inappropriate comparisons of student population characteristics. Simply put, the findings and conclusions of the study are not valid or useful.” (Emphasis added)
Professor Baker noted that the report’s authors knew about their errors because “these issues have been extensively explored for decades and are fundamental and long-accepted components of virtually every state’s finance system.” He also pointed out that he and other critics have previously highlighted these flaws in papers produced by the same authors. This was the genesis of the “specious denominators.”
Baker addressed using the paper’s data for return on Investment claims,
“In only a handful of states are the majority of charter schools ostensibly fully fiscally independent of local public districts. This core problem invalidates all findings and conclusions of the study, and if left unaddressed would invalidate any subsequent ‘return on investment’ comparisons.“
The Authors
Corey A. DeAngelis is the national director of research at the American Federation for Children (DeVos), the executive director at the Educational Freedom Institute (Friedman), an adjunct scholar at Cato Institute (Koch), and a senior fellow at Reason Foundation. He was named on the Forbes 30 Under 30 list for his work on education policy and received the Buckley Award from America’s Future in 2020.
Patrick J. Wolf is principal investigator of the School Choice Demonstration Project where he has led or is leading major studies of school choice initiatives including longitudinal evaluations of school voucher programs in Washington, DC; Milwaukee, WI; and the state of Louisiana.
Cassidy Syftestad is a Doctoral Academy Fellow, Department of Education Reform at the University of Arkansas. Her previous experience includes working at the Koch Institute, the Heritage Foundation, the Heartland Institute and Hillsdale College.
Larry D. Maloney is president of Aspire Consulting.
Jay F. May is founder of and senior consultant for EduAnalytics.
