Meaningful Urban Education Reform: Confronting the Learning Crisis in Mathematics and Science

Appendix A:

This appendix contains the instruments designed or modified by the USI research staff during the course of the evaluation research grant from the National Science Foundation. Because the project was designed both to determine the impact of USI reform in four cities and to test the NSF sixdriver model for reform, it was necessary to create instruments that would elicit information to meet both research goals. The instruments were designed to measure aspects of NSF’s six drivers (see chapter 1, Table 1.2). The appendix is organized into three sections corresponding to the three studies that constitute the scope and focus of our research. The three studies are: the Mathematics and Science Attainment Study; the Study of the Enacted Curriculum; and the Policy Study. Mathematics and Science Attainment Study The Mathematics and Science Attainment Study was geared toward understanding the types of instructional practices in mathematics and science classrooms and how these practices impact student achievement. Through classroom observations and teacher and student surveys of instructional practices (from the Study of the Enacted Curriculum), we were able to document the types of instructional practices being used during the USI reform. The student engagement study was conducted to determine which types of instructional practices kept students engaged Appendix A: Instrumentation 223 224 MEANINGFUL URBAN EDUCATION REFORM during a lesson, and the School Culture Quality Survey sought to document the type of school culture present and whether or not it was supportive for reform activities. A brief discussion of each of these data collection methods is presented below along, with the instrument used to gauge it. Classroom Observation Protocol. In the fall of 1999 the research team designed the classroom observation protocol to collect information related to Driver 1, standards-based instruction. The process began by identifying the categories of standards-based practices for observation, using standards developed by the National Council of Teachers of Mathematics (NCTM) and the National Academy of Sciences as guides. The standards developed by the NCTM and the National Academy of Sciences emphasize high-quality mathematics and science education “for all students.” They are also tied to the notion of the importance of resources to the enhancement of classroom instruction. Students cannot be taught with equity and fairness unless schools and classrooms are equipped with the necessary technological support, in addition to instruction that is standards-based and carried out by teachers who have appropriate certification and professional development experience. Based upon that identification, the research team organized the protocol into three parts to collect data on instructional practices occurring in the classrooms of participating teachers. The first part included information about the class such as the number of students and the ethnicity and gender of the students and teacher. The second part identified the instructional materials, including technology, used during the observed lesson, as well as how the class was arranged. Finally, classroom events, such as teacher and/or student conversations, interactions , and activities were noted. Classroom Observation Coding Checklist. The classroom observation checklist instrument was used to code the classroom observations. The checklist consisted of mathematics and science standards as well as components of other instruments designed to identify five mathematics and science standards-based practices (NCTM, 2000; NRC, 1996): • Communication is a way of sharing ideas and clarifying understanding. • Problem solving is engaging in a task for which the solution is not known in advance. (Problem solving and inquiry were used interchangeably in this study.) • Representation applies to processes and products that are observable externally as well as to those that occur “internally” in the minds of people, such as diagrams, graphical displays, and symbolic expressions. Checklist of Observed Standards-Based Practices 1. Observer’s Name ___________ 8. Teacher Gender ________ _____________ 2. Date and Time __________ ____ 9. Teacher Ethnicity ___________________ 3. School Name _______________ 10. Teacher Age (approx) ___ _____________ 4. Teacher Name ______________ 11. Class Size ______________ ____________ 5. Grade Level ________________ 12. Ethnicity of Students (#’s) _______ _____ 6. Subject ____________________ 13. Gender of Students (#’s) _____________ 7. Class Academic Level (if applicable) __________ _____ Please indicate the emphasis placed on each of the statements: 0 ⫽ None, 1 ⫽ Minimal, 2 ⫽ Moderate, 3 ⫽ Major Communication 0 1 2 3 Modeling appropriate use of mathematical and/or scientific language to communicate mathematical and/or scientific concepts. (Look for effective communication techniques, oral, written, and visual forms). 0 1 2 3 Providing students with the opportunity to communicate mathematical and...