The 2009 Brown Center Report on American Education: How Well Are American Students Learning?

Do Schools Ever Change? An Empirical Investigation

20 The Brown Center Report on American Education Part II: Do Schools Ever Change? An Empirical Investigation How many schools that were languishing at the bottom in 1989 joined the state’s top-performing schools by 2009? T HE OBAMA ADMINISTRATION HAS MADE FIxING PERSISTENTLY failing schools a primary focus of its education policy. “Turnarounds ” are a hot topic. States applying for Race to the Top grants in 2010 will receive more favorable treatment if plans for turning around failing schools are included in the application. Much of the rhetoric on turnarounds is pie in the sky—more wishful thinking than a realistic assessment of what school reform can actually accomplish. That said, schools do indeed change. Turnarounds are not unicorns. But how likely are they to be seen? Background Let’s cover a few details about the study before turning to the data. California has a long history of testing. Using standardized tests to annually measure student progress began in 1962. Complaints arose that nationally normed commercial tests, although informative for revealing where the state stood relative to the rest of the country, did not adequately reflect California’s curriculum and took too long to administer. Beginning in 1973, California tested students in Grades 3, 6, 8, and 12 on the California Assessment Program (CAP), the state’s own test that employed the then novel technique of matrix sampling. Eighth graders were This section of the Brown Center Report presents an investigation into the probability of turning around failing schools. It compares the 1989 and 2009 test scores of 1,156 California schools, all of the schools that contained an eighth grade in 1989 and were still operating in 2009. The turnaround question is one of several addressed in the analysis. How much did any school’s performance change over this twenty-year period? How many schools that were languishing at the bottom in 1989 joined the state’s top-performing schools by 2009? How many made even a little bit of progress? Conversely, how many schools fell from the ranks of high-performing schools to the lower end of the continuum? The Brown Center Report on American Education 21 tested in the spring, and results for every school were published to great fanfare in the fall.13 In the 1980s, the scores were an important element of the state’s accountability system, the Program Quality Review, which also included periodic visits from a team of reviewers who made recommendations for improvement.14 CAP testing lasted until its funding was vetoed by Governor George Deukmejian in 1990. For the current study, 1989 eighthgrade CAP scores were available for reading, math, history/social science, and science. The 2009 scores are in English/language arts (which is paired with the 1989 reading score), math, history, and science.15 For both 1989 and 2009, we created a composite score for each school—simply the average of the four subject scores—and computed percentiles for the composite scores. As a check on whether the results might be influenced by weighting, a composite score using differential weighting of subjects (counting reading , for example, as more important) was also calculated to mirror the formula of California ’s Academic Performance Index. The results were not significantly affected. Using scale scores or percentile scores also did not produce appreciably different results. The analysis uses percentiles, which place performance on a common scale. Imagine a list of schools ranked from 1 to 99 by their scores on a test. Percentiles describe where in that order a particular school falls, with the 99th rank assigned to the highest-performing school and 1 assigned to the lowest-performing school. Percentiles readily demarcate quartiles of performance, the cut points being at the 75th, 50th, and 25th percentiles. Limitations of the study should be noted. Schools that closed or did not have test scores available in either year are omitted . That means schools that opened since 1989—and California opened hundreds of them—are not part of the study. In addition, this is an empirical investigation summarizing an historical pattern so it assumes that the past has something germane to say about the future. That any twenty-year period of the past can be instructive on the question posed by the study’s title, whether schools “ever” change, is admittedly debatable. Perhaps circumstances in the next twenty years will be so different from the 1989–2009 period that any inference...