Meaningful Urban Education Reform: Confronting the Learning Crisis in Mathematics and Science

Chapter 7

Arguably the most important aspect of the work we undertook over a three-year period in the four urban sites was to address the question of how well USI reforms in mathematics and science worked to close achievement gaps between groups of underserved students and their more privileged counterparts. Our aim throughout this research was to determine the impact of the NSF reform agenda on student achievement outcomes using NSF’s six-driver model described in chapter 1. To answer this question we used a series of analyses combining qualitative measures such as classroom observations with teacher and student reports of instructional practice (see chapter 5) and measures of student achievement. Steps along the way included a path analysis for Miami-Dade and Chicago using a variety of data sources developed by reducing fifty-two different indicators into seventeen components to model systemic reform in urban schools. We surveyed teachers in fortysix schools to ascertain the level of professional community, a potential correlate of achievement (Louis, Marks, and Kruse, 1996). From the Survey of the Enacted Curriculum (SEC) data, we identified five instructional factors (problem-solving activities, small-group work, hands-on activities, use of technology, and use of equipment), two assessment factors (performance and testing), four professional 7 Closing the Achievement Gap 155 development factors (standards-based instruction tools, standardsbased instructional methods, time in professional development, and preparation coursework), three teacher opinion factors (traditional beliefs, standards-based beliefs, and beliefs about sharing), three teacher preparation factors (equity, students with special needs, and use of standards-based practices), and two instructional influences factors (national and state standards, students and parents). Cronbach’s alpha was calculated for each factor as a measure of reliability, and those factors considered in the next phase had alpha coefficients ranging from 0.59 to 0.95. In this chapter we will describe quantitative analyses used in our three-year evaluation of NSF’s Urban Systemic Initiative in the four cities, provide a discussion of path analyses of two models of systemic reform, and examine the results of these analyses as they address the question of what factors or indicators increase student achievement and close the achievement gap most effectively. Finally, a cross-site analysis provides indications of the extent to which our results may generalize to other operationalizations of systemic reform. These aspects of our work are critically important to educational policy makers at several levels in the system (Cohen and Ball, 1999; Cohen and Hill, 2000). Causal Modeling Path analysis was developed by Sewall Wright (1934) as a technique for disentangling direct and indirect effects of certain variables (hypothesized to be “causes”) on other variables assumed to be dependent upon the values of causal variables. The technique is not intended to discover causal relationships, but rather to evaluate the tenability of a causal model that is identified a priori. The six-driver model proposed by NSF is one such a priori model that was evaluated using the data collected in the four USI sites (see Figure 7.1). An alternative model, incorporating a seventh driver, was also investigated. See chapter 8 for more information regarding the seventh driver, school culture. Under a set of important assumptions (for example, an accurately specified model with linear, additive relationships among variables; fully recursive relationships; freedom from measurement error), the observed covariation between variables may be decomposed into direct, indirect, and spurious components by translating the causal model into a set of linear equations. If the assumptions are tenable, the parameters of these equations (path coefficients) represent the direction and magnitude of the causal connections between variables. 156 MEANINGFUL URBAN EDUCATION REFORM DEVELOPMENT OF THE MODEL The variety of data sources incorporated in our study allowed us to develop multiple indicators for each of the NSF drivers (see Table 1.2 in chapter 1), to do so over a three-year period, and ultimately to execute a comprehensive analysis of multiple salient features fostering or inhibiting student attainment. In turn, these indicators informed our conceptualization of the NSF driver model, including a hypothesized seventh driver. Indicators included both organizational and individual level factors such as student achievement and student engagement (D5 and D6); teachers’ reports of their professional development experiences , use of technology in the classroom, involvement in decisionmaking processes, and so on (D1 and D3); school district assessment practices and school-level support structures (D2); teachers’ classroom practices (D1); the nature of community-school partnerships...