Comment

David P. Baker makes a significant contribution to the understanding of cross-national education differences by synthesizing many studies that he and his colleagues have done with the Third International Mathematics and Science Study (TIMSS) data and by drawing policy implications from that synthesis. This paper not only addresses the issue of poor performance of the average American student from an international perspective, but also pays attention to the poorer achievement of disadvantaged students in the United States. Baker's paper provides a comprehensive look into school effect by combining a complex array of the TIMSS data on school input, process, context, and outcomes. However, the paper also reveals the limitations of the current TIMSS data, as well as limitations in Baker's studies as they relate to critical policy questions.

Issues of Selection for Review and Comparison

In reviewing and synthesizing past TIMSS studies, Baker selects almost exclusively his own studies without referencing other studies on the same topic. Providing a comprehensive and balanced literature review, including earlier international studies using non-TIMSS data as well as the studies of TIMSS data by other researchers, would have created a scholarly framework within which his findings could be better evaluated. In the comparative research field, different studies often produce findings that are not always congruent with each other. Much depends on which countries and variables are selected and how they are analyzed and compared.

Who is being compared with whom is critical in this kind of comparative study. The validity of selection and comparability is not discussed in this paper. Baker is not consistent in his selection of countries for comparison. In the first part of his paper, when examiningaverage achievement, Baker includes all TIMSS countries for comparison. But in the second part of his paper, where he examines the achievement of disadvantaged students, Baker selects only developed, wealthy countries. Moreover, he uses both eighth- and twelfth-grade data for the former comparison but only twelfth-grade data for the latter comparison. It is not clear why he chose to use different sets of countries or grades for comparisons and how such different selection criteria might have influenced his findings and conclusions.

Missing Variables and Measurement Issues

The studies that Baker reviews use several indicators to measure school effects, but some important variables are missing in those analyses. To begin with, information on the rigor of the curriculum is lacking. Based on the TIMSS teacher survey results, Baker points out that the average U.S. eighth-grade math class has more coverage and repetition of topics and that these variables are not related to achievement variations across the TIMSS countries. Without considering other key instructional variables, however, how the breadth of instruction alone can be related to achievement outcomes is not clear. The level of implemented curriculum should have been considered. TIMSS curriculum studies showed that the U.S. curriculum is not only less focused but also less advanced.³² Further, Baker's comparison of implemented curriculum is limited to the eighth-grade data, and inferring that the American high school is not much different from an international average based on the eighth-grade level data is misleading. Simply comparing the national averages also conceals substantial variations in the type of courses offered by American high schools and taken by their students.

Teacher quality and training are another important set of missing variables in Baker's review. He acknowledges that the TIMSS data provide little information on the training of teachers. In fact, teacher quality can be the most important factor that determines the quality of instruction. Even when teachers say that they cover the same thing to the same extent, their differences in content and pedagogical knowledge should result in significant variation in the quality of teaching and learning.

In addition to these kinds of key schooling variables, much broader cultural and institutional differences must be considered. East Asian countries such as Japan and South Korea could not have performed best without their...