Getting to Scale: How to Bring Development Solutions to Millions of Poor People

10. Institutional Challenges to Scaling Up Learning in Kenya

262 10 Institutional Challenges to Scaling Up Learning in Kenya tessa bold, mwangi kimenyi, germano mwabu, alice ng’ang’a, and justin sandefur Education is a fundamental component of the development process, and governments in developing countries invest heavily in education. Despite this commitment, learning outcomes remain poor in low-income countries. In Kenya, a highly publicized study by the advocacy organization Uwezo demonstrates the severity of the learning crisis, with only a third of grade three pupils able to read simple sentences and very slow improvements in reading and math skills as students progress to higher grades.1 To cite one example from the enormous policy literature on this learning failure, the recent Africa Learning Barometer discusses the many reasons that learning outcomes on the continent are inadequate. These include resource constraints such as books, facilities such as toilets and classrooms, and the quality and number of teachers. But resources are just part of the story—the incentives provided to parents, students, and teachers also matter, and these incentives are shaped by curriculum design, testing systems, school governance structures, and the types of mechanisms used to monitor school performance. How can policymakers find out what works? Small-scale experiments provide a means to pilot innovative approaches to improve learning outcomes and subject them to rigorous evaluation. In the United States, the Department of 1. Uwezo (2011). Scaling Up Learning in Kenya 263 Education’s “What Works Clearinghouse” provides a catalogue of school management and teaching practices that have been shown to improve learning. In the developing world, research in Western Kenya has led the way in using randomized controlled trials (RCTs) to demonstrate the effectiveness of schoollevel reforms to incentives. This broad research program suggests a number of low-cost ways to improve learning in Kenyan primary schools, including tracking students by ability, offering merit-based scholarships as an incentive to students , and hiring teachers on fixed-term, renewable contracts.2 Although these and many other small-scale interventions to improve educational or other development outcomes have been found to be effective in a variety of settings, it is not quite obvious that such successes can be replicated when the interventions are scaled up to entire regions or even a country. We review a number of, by now, familiar concerns that must be addressed when extrapolating from small-scale experiments to forecast the impact of scaled-up policies. We devote special attention to one obstacle to scaling-up experimental results that has received relatively less attention in the literature to date. This stems from the fact that, in many cases, the implementation of experimental pilots is undertaken by nongovernmental organizations. There are clear logistical and cost reasons for this. But there are also drawbacks to piloting with NGOs, especially considering that large-scale delivery of education and other social services is dominated by the government in most developing countries. Thus the most feasible and sustainable approach to scaling up successful interventions will necessarily be through government implementation. As we discuss in detail below, it cannot be assumed that implementation by NGOs and the government will yield similar outcomes. We illustrate this point using our own prior work.3 The idea is to explore, first, whether the results observed under small-scale interventions hold when scaled up and, second, whether the outcomes differ whether the intervention is by an NGO or the government. The results of the interventions reveal that indeed there are differences in outcomes between NGO and government implementation . The results suggest that we need to look beyond the intervention— the contract teacher—to the institutions of implementation. It is by getting a clear understanding of the dynamics of the institutions of implementation— political economy, so to say—that we can then know the conditions under which scaling up will succeed. 2. The program is summarized in Kremer and Holla (2009). See also Duflo, Dupas, and Kremer (2011, 2012); Kremer, Miguel, and Thornton (2009). 3. Bold and others (2012); this work attempts to replicate the results of the NGO project described in Duflo, Dupas, and Kremer (2012) in other contexts, using both NGO and government implementation. 264 T. Bold, M. Kimenyi, G. Mwabu, and others Why Experiment? The Challenge of Internal Validity Randomized controlled trials are ideally suited for measuring the causal effects of a particular policy in a particular setting at a particular time. Joshua Angrist and Jorn-Steffen Pischke give a useful definition of...