Comment

The paper by Michael T. Nettles, Catherine M. Millett, and Douglas D. Ready addresses one of the most important issues in current education policy: reducing inequality in educational outcomes, particularly between African American and white students. The paper follows in the tradition of previous studies that try to identify those factors that have a causal effect on the black-white achievement gap, although Nettles and his colleagues appropriately note that their paper cannot identify causal relationships. I elaborate here on why causal inferences cannot be drawn from these findings. I also note that even if strong conclusions could be made about what drives the black-white gap from these findings, the implications for public policy are not obvious.

Causes of the Black-White Test Score Gap

Causal inference is complicated primarily for two reasons, both of which relate to the role of self-selection. First, SAT takers are not a random sample of American adolescents. Those students who are most likely to take the SAT are also disproportionately likely to have well-educated parents or good high school grades.⁴⁸ All else equal, when SAT-taking rates increase, the average level of socioeconomic and academic advantage of SAT-takers declines. The result is that average SAT scores and the proportion of students who take the SAT are negatively correlated, both across states at a particular time and within states over time.⁴⁹ This pattern poses a problem for Nettles and his colleagues, because the same factors that affect the academic achievement of black and white students may also affect the likelihood that students take the SAT. In this case, some policy lever may improve overall student achievement but not increase average SAT scores if the intervention also induces a substantial increase in the proportion of students who take the SAT.

To illustrate the problem, assume for simplicity that white students always take the SAT and always score a combined 900 on the math and verbal tests. Suppose further that some school characteristic (smaller class sizes) has little effect on how much white students learn in school but increases how much African American students learn and improves their odds of considering college and thus taking the SAT. In this example, the size of the black-white SAT gap may be no smaller—and may even be larger—in schools with small classes, even though the overall population of black students learns more with small classes both absolutely and relative to whites. In reality, school characteristics may affect the academic achievement and SAT-taking rates of both whites and blacks, so in practice the magnitude and even sign of bias that results from inadequately controlling for the nonrandom self-selection of SAT takers is difficult to predict.

One way to begin to address this problem is to adjust for the SAT-taking rates separately of white and black students within each school, with some care paid to the functional form of the relationship between average black (or white) SAT scores and the proportion of black (white) students taking the SAT.⁵⁰ This type of procedure has limitations but would at least be a step in the right direction.

A second problem, which is more fundamental to education research generally, is that students are not randomly assigned to schools with different characteristics such as class sizes. Many parents care a great deal about where their children go to school, as evidenced by the relationship between local school spending or quality and house prices.⁵¹ The result is some systematic relationship between the family backgrounds of students and the schools that they attend. Parent and student characteristics may also be related to the child's school inputs because educational administrators assign additional resources to schools or classrooms that disproportionately serve low- (or high-) achieving students.

The paper by Nettles, Millett, and Ready attempts to address this version of the self-selection problem by regression adjusting for observed student and family characteristics. The standard regression-based...