Comment

John H. Bishop, Ferran Mane, Michael Bishop, and Joan Moriarty are to be commended for bringing sophisticated empirical analysis to an issue that generates considerable ill-informed debate: whether standards-based testing with genuine stakes helps or harms students. More specifically, they have provided an array of carefully conducted analyses designed to examine the impact of two types of standards-based testing (end-of-course exams and minimum competency exams) on several widely watched indicators of student achievement: high school graduation, college enrollment, achievement test scores, and earnings. They have tested a series of well-developed hypotheses using both cross-sectional and longitudinal data and with different data sets. By and large the results are encouraging to those who support standards-based reforms and problematic for those who oppose them. Students, for the most part, do not drop out in large numbers when they are required to pass exams to graduate or receive course credit, and some evidence exists that standards-based testing may increase college attendance and post-high school earnings. On the bases of their analyses, the authors conclude that the most effective type of standards-based reform is one that combines the use of end-of-course exams with the implementation of minimum competency exams.

Is standards-based reform working? As answered in this paper, it depends, in at least three ways. First, standards-based reform has many meanings and many components, and the impact of standards-based reform depends on which component is being examined. Some (such as the use of end-of-course examinations) have stronger effects than others, and others (such as the use of published report cards on schools) apparently have no effects at all. Without clear guidance about which elements of standards-based reform a state or school district ought to implement, a state or district could come away empty-handed and discouraged after attempting to implement a standards-based policy that has good intentions but few discernible consequences.

Second, the impact of standards-based reform depends on which outcome is being assessed. The same reform may have positive effects on one outcome, negative effects on another, and no effects on a third. For instance, in the analyses presented by Bishop and his colleagues, the use of minimum competency testing has a negligible effect on high school enrollment or graduation, once general equivalency diplomas (GEDs) are taken into account; a small positive effect on the achievement of poor students (those with a grade point average [GPA] of C- or below) but a small, negative effect on the achievement of good ones (those with a GPA of A); a positive effect on college enrollment, especially among poor and average students; and a surprisingly significant impact on post-high school employment and earnings, especially among good students. A state or district that failed to evaluate standards-based reform with multiple indicators might come away with a conclusion that was excessively optimistic or excessively pessimistic. Surely the fact that minimum competency testing has stronger effects on college attendance and earning than on achievement is a mixed blessing.

Third, the impact of standards-based reform depends on which students are the focus of the analysis. The same reform may affect a specific outcome differentially among poor students than among affluent ones, among minority students differentially than among white ones, and among poor achievers differentially than among high achievers. Some evidence indicates that the implementation of minimum competency testing may provoke schools to focus more energy on poor-achieving students, but that this laudable response may be accompanied by a troublesome decline in achievement among students with relatively higher grades. Raising the achievement of students at the bottom of the distribution is important, but schools need to find a way to do this without compromising the already mediocre test performance of the best students.

The paper does not say much about the differential impact of various reforms as a function of student race, which...