Getting Inside Accountability: Lessons from Chicago

THE PASSAGE OF No Child Left Behind (NCLB) in 2001 ensures that school accountability will dominate the educational landscape in this country for the foreseeable future. ¹ NCLB requires states to test all children in grades three to eight in reading and mathematics in each year and report the percentage of students meeting state-defined proficiency levels in each school. Results must be broken down by poverty, race, ethnicity, disability, and limited English proficiency. Schools will be required to raise the proportion of students meeting these targets each year according to a schedule that ensures that all students are proficient by 2014. If schools fail to make "adequate yearly progress," they may be subject to increasingly severe interventions, culminating with the closure or reconstitution of the school. Although it is more comprehensive in scope, the federal legislation resembles policies adopted by a number of states and school districts in recent years.

While there is a growing body of research on how accountability policies influence student achievement, dropout rates, and other outcomes, there is little evidence on the mechanisms through which accountability policies operate. Proponents contend that high-stakes testing will lead teachers and students to work harder, increase parental involvement, and force schools to become more efficient by, for example, instituting new instructional strategies or reorganizing the school day. ² Critics respond that it will merely lead schools to shift attention away from nontested subjects and cause teachers to focus excessively on narrowly tailored test preparation strategies. ³

Understanding the mechanisms through which accountability policies work is important for several reasons. For one thing, it can help us to interpret the impact of these reforms and assess the extent of any unintended consequences. Suppose we see science performance decline after the introduction of math and reading standards. Some observers have suggested that this event might simply be owed to the diminished stature of the science exam. If this decline is accompanied by a decrease in the number of science teachers employed by a school system, however, we might be less willing to accept such benign views. ⁴ Second, knowledge of the mechanisms will help us better understand the potential and limitations of this reform strategy. If the reform works primarily through increasing student effort or parent involvement, for example, without changing the technology of schooling, we might expect student performance to plateau after initial increases. Third, knowledge of the policy mechanisms may shed light on the education production function. If, for example, we see that low-achieving schools that showed considerable improvement in student performance under accountability spent less money on teacher aides and more on professional development, we might take this result as evidence suggesting the efficacy of teacher training. ⁵

In this paper, I examine the recent reforms in the Chicago Public Schools (CPS) in an effort to learn more about the mechanisms underlying test-based accountability policies. Chicago provides an excellent case study. It was one of the first large, urban school districts to institute high-stakes testing. There is evidence that the program led to substantial increases in math and reading achievement, particularly among low-performing students and schools. ⁶ Finally, there has been a considerable amount of high-quality research on the Chicago policy, largely because of a unique collaboration between district officials and a university-based research organization, the Consortium on Chicago School Research (CCSR).

To help make sense of the myriad ways in which individuals and schools may have responded to the reforms, I present a simple model of education production that distinguishes between changes in educational inputs (for example, student effort, financial resources) and changes in the technology of schooling (for example, instructional practices, school organization). In the context of this framework, I discuss a variety of potential mechanisms, drawn from theory and existing research, that might have led to achievement gains under accountability. After reviewing the existing evidence on the Chicago reforms, I...