The Education Gap: Vouchers and Urban Schools

Chapter 6: The Urban Test Score Gap

Switching to a private school changed the educational experiences of inner-city low-income students in important ways. Compared with their peers in public schools, voucher students were taught in smaller classes located in much smaller schools. They received more homework assignments, faced fewer disruptions, and abided by stricter dress codes. Communication between their school and parents was more extensive. Meanwhile, students in public schools enjoyed more physical resources and academic programs, and they were subject to closer supervision when they moved throughout the school building. Still other aspects of schooling changed hardly at all. Suspension rates, parents’ involvement in their child’s education, students’ selfesteem , students’ friendship patterns, and schools’ racial composition all remained much the same, regardless of whether a student attended a public or private school. How do these changes translate into learning? Or, to put it more precisely, do inner-city students who attend private schools score higher on standardized tests than their public school peers? Standardized Test Performance Students’ performance on standardized tests in reading and math may say very little about the quality of their elementary and middle schools; the multiple-choice questions that constitute those tests surely do not capture all that students learn in school. Although scholars have found that the perfor140 The Urban Test Score Gap 6 file 03 ch04-06 pp90-167.qxd 3/14/02 2:07 PM Page 140 mance of high-school students on such tests does a fairly good job of predicting their future success in the labor market,1 few if any studies have shown that the performance of elementary and middle-school students on standardized tests has important downstream consequences. It also is difficult to know whether changes in test scores from one elementary year to the next are meaningful. Such test score gains (or losses) are known as value-added measures of school effectiveness because they measure the change in the level or value of the test scores from one period to the next. Such measures are subject to a good deal of natural variation, and any effort to draw strong conclusions from annual shifts is problematic. With his colleague , Thomas Kane, an economist at the University of California in Los Angeles, has shown that value-added analyses used to evaluate schools in North Carolina displayed a good deal of random noise that bedeviled those who attempted to make policy on the basis of their results.2 In their words, “one-time factors . . . lead to temporary fluctuations in test performance. Some of these factors are likely to be unrelated to the educational practices of a school. For instance, a dog barking on the day of the test, a severe flu season , or one particularly disruptive student in class could lead scores to fluctuate .”3 Such distractions are especially likely to cause havoc with research findings whenever sample sizes are small. In their research, Kane and others find that smaller elementary schools experience greater year-to-year test score swings (positive and negative) than larger schools. For this reason, the researchers note, state accountability regimes reward and punish small schools more frequently, not because their students are experiencing actual learning gains or losses, but because the estimates of their achievement levels are less stable. Education statistician Anthony Bryk makes much the same point when he cautions against drawing conclusions about the impact of a school intervention from “single grade information. . . . Judging a school by looking at only selected grades can be misleading. We would be better off, from a statistical perspective, to average across adjacent grades to develop a more stable estimate of school productivity.”4 Test score findings from randomized field trials (RFTs), such as those presented in this chapter, are value-added measures. The treatment and control groups were similar at baseline, so any differences observed subsequently provide estimates of the value added by the voucher intervention. Further, to increase the precision of our estimates, all models explicitly controlled for baseline test scores. To minimize random error in these value-added measures , we combined test scores in reading and math in the analyses presented in this chapter because, by so doing, estimates rely upon a larger number of the urban test score gap 141 file 03 ch04-06 pp90-167.qxd 3/14/02 2:07 PM Page 141 test items, thereby further reducing random fluctuation. We also based our interpretation on results from all students tested, regardless of grade...