Learning More from Social Experiments

Evolving Analytic Approaches

Publication Year: 2006

Policy analysis has grown increasingly reliant on the random assignment experiment—a research method whereby participants are sorted by chance into either a program group that is subject to a government policy or program, or a control group that is not. Because the groups are randomly selected, they do not differ from one another systematically. Therefore any differences between the groups at the end of the study can be attributed solely to the influence of the program or policy. But there are many questions that randomized experiments have not been able to address. What component of a social policy made it successful? Did a given program fail because it was designed poorly or because it suffered from low participation rates? In Learning More from Social Experiments, editor Howard Bloom and a team of innovative social researchers profile advancements in the scientific underpinnings of social policy research that can improve randomized experimental studies. Using evaluations of actual social programs as examples, Learning More from Social Experiments makes the case that many of the limitations of random assignment studies can be overcome by combining data from these studies with statistical methods from other research designs. Carolyn Hill, James Riccio, and Bloom profile a new statistical model that allows researchers to pool data from multiple randomized-experiments in order to determine what characteristics of a program made it successful. Lisa Gennetian, Pamela Morris, Johannes Bos, and Bloom discuss how a statistical estimation procedure can be used with experimental data to single out the effects of a program’s intermediate outcomes (e.g., how closely patients in a drug study adhere to the prescribed dosage) on its ultimate outcomes (the health effects of the drug). Sometimes, a social policy has its true effect on communities and not individuals, such as in neighborhood watch programs or public health initiatives. In these cases, researchers must randomly assign treatment to groups or clusters of individuals, but this technique raises different issues than do experiments that randomly assign individuals. Bloom evaluates the properties of cluster randomization, its relevance to different kinds of social programs, and the complications that arise from its use. He pays particular attention to the way in which the movement of individuals into and out of clusters over time complicates the design, execution, and interpretation of a study. Learning More from Social Experiments represents a substantial leap forward in the analysis of social policies. By supplementing theory with applied research examples, this important new book makes the case for enhancing the scope and relevance of social research by combining randomized experiments with non-experimental statistical methods, and it serves as a useful guide for researchers who wish to do so.

Published by: Russell Sage Foundation

Contents
Search Inside This Book
Book Details

This book is founded on a commitment to promote evidence-based policy-making in human affairs by developing and refining the research tools available to social scientists. Specifically, the book seeks to advance the science of evaluation research by presenting innovative ways to address the following high-stakes...

Chapter 1 Precedents and Prospects for Randomized Experiments

pdf icon Download PDF (280.0 KB)
pp. 1-36

When families move to low-poverty neighborhoods, their teenage children are less likely to commit crimes (Ludwig, Hirschfield, and Duncan 2001). Couples therapy and family therapy are equally effective at improving marital relationships (Shadish et al. 1995). Increasing welfare benefit amounts by 10 percent...

Chapter 2 Modeling Cross-Site Experimental Differences to Find Out Why Program Effectiveness Varies

pdf icon Download PDF (288.6 KB)
pp. 37-74

Charged with planning a new social program, senior administrators in a state human services agency pore over stacks of evaluation research, seeking knowledge and insights that can help them design the new initiative. The evaluations provide them with lots of information about the effects of particular programs on particular people in particular settings. And having used random assignment to measure program effects, or impacts, the studies...

Chapter 3 Constructing Instrumental Variables from Experimental Data to Explore How Treatments Produce Effects

pdf icon Download PDF (325.2 KB)
pp. 75-114

A random-assignment study can provide the most compelling evidence possible about how an intervention‒be it social, economic, legal, or medical‒affects the people to whom it is targeted. Randomization entails using a lottery-like process to assign each eligible sample member either to a group that is offered the...

Chapter 4 Randomizing Groups to Evaluate Place-Based Programs

pdf icon Download PDF (498.9 KB)
pp. 115-175

Social interventions such as community improvement programs, school reforms, and employer-based efforts to retain workers, whose aim is to change whole communities or organizations, are often called place-based initiatives. Because such programs are designed to affect the behavior of groups of interrelated people rather...

Chapter 5 Using Experiments to Assess Nonexperimental Comparison- Group Methods for Measuring Program Effects

pdf icon Download PDF (446.9 KB)
pp. 173-235

The past three decades have seen an explosion in the number of social program evaluations funded by government and nonprofit organizations. These evaluations span a wide range of policy areas, including education, employment, welfare, health, criminal justice, housing, transportation, and the environment. Properly evaluating...

Index

pdf icon Download PDF (126.8 KB)
pp. 237-246

E-ISBN-13: 9781610440707
Print-ISBN-13: 9780871541338
Print-ISBN-10: 0871541335

Page Count: 264
Publication Year: 2006

Learning More from Social Experiments

Title page

Copyright

Contents

Contributors

Preface

Chapter 1 Precedents and Prospects for Randomized Experiments

Chapter 2 Modeling Cross-Site Experimental Differences to Find Out Why Program Effectiveness Varies

Chapter 3 Constructing Instrumental Variables from Experimental Data to Explore How Treatments Produce Effects

Chapter 4 Randomizing Groups to Evaluate Place-Based Programs

Chapter 5 Using Experiments to Assess Nonexperimental Comparison- Group Methods for Measuring Program Effects

Index

Research Areas

Welcome to Project MUSE

Connect with Project MUSE

Cover 1

Cover 2

Cover 3

Learning More from Social Experiments

Title page

Copyright

Contents

Contributors

Preface

Chapter 1 Precedents and Prospects for Randomized Experiments

Chapter 2 Modeling Cross-Site Experimental Differences to Find Out Why Program Effectiveness Varies

Chapter 3 Constructing Instrumental Variables from Experimental Data to Explore How Treatments Produce Effects

Chapter 4 Randomizing Groups to Evaluate Place-Based Programs

Chapter 5 Using Experiments to Assess Nonexperimental Comparison- Group Methods for Measuring Program Effects

Index

Research Areas

Recommend

Related Content

Welcome to Project MUSE

Connect with Project MUSE

Cover 1

Cover 2

Cover 3