Fighting for Reliable Evidence

Chapter 8: Expanding the Agenda: The Role of Basic Education

CHAPTER 8 Expanding the Agenda: The Role of Basic Education* Under the guise of research to determine how well the program is working, roughly 15–20 percent of the applicants . . . will be shunted into a control group that will receive no job counseling or support services. . . . This is shameful and inhumane, and shouldn’t be allowed to happen. —St. Petersburg (Fla.) Times1 If we’re going to put thousands of people through something, we ought to be willing to find out whether or not it works. —Don Winstead2 We have seen that real-world experiments proved feasible and credible; that most welfare-to-work programs were successful; that politicians and public officials listened and acted; that states could be willing partners; that a major foundation sparked these changes; and that several government agencies endorsed experiments as uniquely reliable. If welfare research had gone no further , it would already be unusual in its influence and in the use of multiple, large-scale experiments to determine the potential of a reform strategy. But the story did not end there. What makes the saga exceptional is that the momentum was sustained and resulted in a comprehensive and coherent body of reliable evidence about a range of policy alternatives. In the process, *Chapter 8 authored by Judith M. Gueron. EXPANDING THE AGENDA 263 welfare researchers demonstrated the potential of using random assignment in increasingly demanding environments and of learning more from such studies. In the ten years from 1986 to 1996, these distinctive new contributions began to emerge. There was a flowering of experiments, a sort of golden age—complete with fresh victories and intensified battles—that generally followed the new paradigm laid out in the early 1980s but with four key differences that interacted to create the larger whole: the entrepreneurs, who were the sources of innovation; the policies tested; the questions asked; and the scale of the programs and the experiments. During these years, three parties took the lead in shaping the knowledgebuilding agenda: the federal government, MDRC, and the states. At the federal level, the Department of Health and Human Services (HHS) funded many of the major studies and brought researchers at several organizations, particularly Mathematica Policy Research and Abt Associates, into the field. At the state level, although governors sought to change welfare, not to conduct research, they sometimes ended up in random assignment studies because they needed federal waivers and encountered staff at HHS and OMB, who made budget neutrality, as measured by an experiment, the price for flexibility. In other cases, exceptional state officials who learned about the growing success of such studies decided that this approach held enough promise to be worth the political risk, in the process becoming determined converts and occasionally even writing random assignment into law. As a result, for the first time, some of the champions of random assignment—the people who bore the brunt of the fight—were state staff. These three parties knew about the modest but positive effects of the first generation of welfare-to-work programs and asked whether something else might work better. For some, doing better meant producing larger impacts and reaching the more disadvantaged by increasing the focus on education and training. For others, it meant helping single mothers not only obtain work but move out of poverty by supplementing their earnings, getting them higher-paying jobs, or increasing child support payments. For still others, it meant extending services and obligations to a wider range of people, including mothers with younger children, teen parents, and fathers who were not living with their children on welfare. For yet others, it meant tougher mandates and limits on how long someone could receive welfare. Whereas early 1980s studies addressed the first-order queries about welfareto -work programs—Do they increase work and reduce welfare? At what 264 FIGHTING FOR RELIABLE EVIDENCE cost?—people now sought answers to more refined questions: What works best for whom, under what conditions, and why? Were there trade-offs, with different policies producing different benefits at different costs for different people? What metrics should be used to judge success: more work, greater participation, higher wages, reduced poverty, reduced dependency, lower costs, or better outcomes for children? How much did different factors—design , implementation, context, who participates—explain why some programs are more effective than others? Earlier experiments assessed special demonstrations of programs designed by researchers or funders (such as Supported Work or...