We cannot verify your location
Browse Book and Journal Content on Project MUSE

Fighting for Reliable Evidence

Judith M. Gueron, Howard Rolston

Publication Year: 2013

Once primarily used in medical clinical trials, random assignment experimentation is now accepted among social scientists across a broad range of disciplines. The technique has been used in social experiments to evaluate a variety of programs, from microfinance and welfare reform to housing vouchers and teaching methods. How did randomized experiments move beyond medicine and into the social sciences, and can they be used effectively to evaluate complex social problems? Fighting for Reliable Evidence provides an absorbing historical account of the characters and controversies that have propelled the wider use of random assignment in social policy research over the past forty years.Drawing from their extensive experience evaluating welfare reform programs, noted scholar practitioners Judith M. Gueron and Howard Rolston portray randomized experiments as a vital research tool to assess the impact of social policy. In a random assignment experiment, participants are sorted into either a treatment group that participates in a particular program, or a control group that does not. Because the groups are randomly selected, they do not differ from one another systematically. Therefore any subsequent differences between the groups can be attributed to the influence of the program or policy. The theory is elegant and persuasive, but many scholars worry that such an experiment is too difficult or expensive to implement in the real world. Can a control group be truly insulated from the treatment policy? Would staffers comply with the random allocation of participants? Would the findings matter?Weaving history, data analysis and personal experience, Fighting for Reliable Evidence offers valuable lessons for researchers, policymakers, funders, and informed citizens interested in isolating the effect of policy initiatives. It is an essential primer on welfare policy, causal inference, and experimental designs.

Published by: Russell Sage Foundation

Title Page, Copyright Page

pdf iconDownload PDF (39.6 KB)
pp. i-iv


pdf iconDownload PDF (23.3 KB)
pp. v-vi

Tables and Figures

pdf iconDownload PDF (23.2 KB)
pp. vii-viii

About the Authors

pdf iconDownload PDF (25.8 KB)
pp. ix-x

read more


pdf iconDownload PDF (27.7 KB)
pp. xi-xiv

Welfare policy in this country is unique among all areas of social policy in having had forty-five years of uninterrupted, large-scale random assignment studies that assessed the effectiveness of reform initiatives. This book tells the story of our participation in that history. It is not an impartial account. For most of these years, we were actors in the story. Judy helped to build and then...

read more

Acknowledgements and Dedication

pdf iconDownload PDF (40.2 KB)
pp. xv-xviii

Many people over many years made this book possible. Some of them were responsible for the story itself, and the fact that it reports success in using experiments to test social policies; others directly supported our work on this volume. Those in the first category fill the pages of this book. They are the state and community officials and staff who, by participating in random assignment studies, let us redefine, expand on, and make a reality of Justice...

read more

Chapter 1: Introduction: The Issue, the Method, and the Story in Brief

pdf iconDownload PDF (172.2 KB)
pp. 1-21

How can we know whether social programs do what they are designed to do? Can that question even be answered convincingly? A simple example illustrates the problem: The governor of a large midwestern state is under pressure. Welfare rolls are rising. Legislators are pushing for action. Armed with a report from a blue-ribbon panel, she recommends a new program with a catchy name: WoW, for...

read more

Chapter 2: They Said It Couldn’t Be Done: The National Supported Work Demonstration

pdf iconDownload PDF (187.3 KB)
pp. 22-65

The Supported Work project started out as a small local program offering subsidized jobs to two hard-to-employ groups (former prisoners and former addicts) in a structured environment, followed by assistance in finding an unsubsidized job. The national Supported Work demonstration, the subject...

read more

Chapter 3: Bridge to the 1980s: Implementing Random Assignment in Work Incentive Program Offices and Testing a Job Guarantee

pdf iconDownload PDF (114.4 KB)
pp. 66-87

This chapter tells the story of two initiatives that helped MDRC not only survive as an institution in the dramatically different policy and policy- research environment of the Reagan era but also continue using random assignment to learn more about program evaluation generally and the work-...

read more

Chapter 4: Inventing a New Paradigm: The States as Laboratories

pdf iconDownload PDF (217.6 KB)
pp. 88-140

Political transitions can sound portentous but may bring little change. For our story, however, Ronald Reagan’s election in 1980 was a true revolution, marking a dramatic turning point in welfare policy, the role of the states, and the nature and origin of welfare research. Support for a welfare entitlement and for voluntary programs such as Supported Work yielded to the harsher language of mandates and obligations. Policy initiative shifted from Washington...

read more

Chapter 5: The Reagan Revolution and Research Capacity Within the Department of Health and Human Services: From Near Destruction to New Growth

pdf iconDownload PDF (177.2 KB)
pp. 141-180

When Ronald Reagan won the presidency in 1980, the Department of Health and Human Services (HHS) had already played a substantial role in what was then the relatively brief history of social experiments. Although the first such experiments were initiated by the Office of Economic Opportunity, through...

read more

Chapter 6: The Work/Welfare Demonstration: Lessons About Programs, Research, and Dissemination

pdf iconDownload PDF (220.7 KB)
pp. 181-216

The Work/Welfare demonstration was the major turning point in our story. If Supported Work and the WIN Labs put a toe in the real world, this project was an immersion. At the start, we at MDRC fretted over all of the prerequisites to success: recruiting the states, getting the money, gaining federal coop-...

read more

Chapter 7: Waiver Evaluations: How Random Assignment Evaluation Came to Be the Standard for Approval

pdf iconDownload PDF (194.9 KB)
pp. 217-261

Throughout the first Reagan administration, the Department of Health and Human Services (HHS) applied the approval standard described in chapter 5 to section 1115 waiver applications: it approved waiver requests from states whose policy proposals were consistent with administration policy and denied those from states that wished to try anything different. Evaluation requirements were minimal, which was plainly permissible as a legal matter, given the...

read more

Chapter 8: Expanding the Agenda: The Role of Basic Education

pdf iconDownload PDF (211.5 KB)
pp. 262-310

We have seen that real-world experiments proved feasible and credible; that most welfare-to-work programs were successful; that politicians and public officials listened and acted; that states could be willing partners; that a major foundation sparked these changes; and that several government agencies endorsed experiments as uniquely reliable. If welfare research had gone no further,...

read more

Chapter 9: The JOBS Evaluation: Cumulative Evidence on Basic Education Versus Work First

pdf iconDownload PDF (199.8 KB)
pp. 311-353

Along with the Greater Avenues for Independence program (GAIN) evaluation (later, California’s JOBS program), the Job Opportunities and Basic Skills Training (JOBS) evaluation would profoundly alter the views of policy makers, program operators, and researchers about the effectiveness of alternative...

read more

Chapter 10: Expanding the Agenda II: Three Experiments to Make Work Pay

pdf iconDownload PDF (173.2 KB)
pp. 354-387

With the welfare debate shifting rapidly toward the right in the early 1990s, work-for-your-grant (workfare) and related policies that had been denigrated as slavefare ten years earlier looked benign next to calls to end support. The tipping point came when the Democratic presidential candidate, Bill Clinton, made “ending welfare as we know it” and “making work pay” central parts of...

read more

Chapter 11: The End of Our Story: Welfare Reform Recedes from the Spotlight of Random Assignment Program Evaluation

pdf iconDownload PDF (168.7 KB)
pp. 388-422

Both Judy and I have heard people say, even experts very familiar with the welfare evaluation field, that the rigor of the large body of welfare experiments was a result of the federal government’s use of its 1115 waiver demonstration authority to require an experimental evaluation in exchange for programmatic...

read more

Chapter 12: Conclusion and Lessons

pdf iconDownload PDF (156.9 KB)
pp. 423-454

In the preceding chapters, we tell of our long struggle to show that a relatively untried technique—large-scale, random assignment experiments (the social policy analogue to clinical trials in medicine)—could be used to answer important questions about complex social programs. We also show that the results were considered singularly credible and, in part because of that, were...

read more

CODA: Random Assignment Takes Center Stage

pdf iconDownload PDF (91.6 KB)
pp. 455-472

This book describes the struggle to promote randomized experiments as a campaign waged over many years, mostly by people outside the normal academic reward system. In recent years, this has changed, both in the United States and internationally, prompting me to ask a number of people (see the appendix) close to the past conflict or current transformation: How would...


pdf iconDownload PDF (57.5 KB)
pp. 473-478


pdf iconDownload PDF (23.6 KB)
pp. 479-480


pdf iconDownload PDF (212.7 KB)
pp. 481-536


pdf iconDownload PDF (40.2 KB)
pp. 537-538


pdf iconDownload PDF (82.5 KB)
pp. 539-552


pdf iconDownload PDF (122.5 KB)
pp. 553-576

E-ISBN-13: 9781610448130
E-ISBN-10: 1610448138

Page Count: 593
Publication Year: 2013

Edition: 1