Comment

In commenting on G. Alfred Hess Jr.'s paper, I use Chicago as an example to highlight what the Consortium for Policy Research and Education, the organization where I do my work, calls the second-generation problems of standards-based reform. That is, if the principle of continuous improvement is applied to an accountability system, what sort of problems has arisen in Chicago? First, however, I would like to make a couple of contextual remarks. Chicago, as a case, is a powerful model in ways that have not been explicitly stated. The Chicago experience is one of a relationship between the academic community and the school system around issues of reform. This relationship has not been altogether smooth. In fact, it has often been highly troubled. But the best academic minds in Chicago, which are among the best academic minds in the country, have worked on school reform there. This unusual situation has reached only tepid approximation in Boston. When the likes of Anthony Bryk, Alfred Hess, and others are willing to devote substantial amounts of their time to making reform work, a unique set of circumstances is created.

Hess is an authority on accountability. His paper contains a certain smoothness of narrative that glides over many historical and substantive complexities of Chicago school reform. Such a conceit develops when an author gets into a historical voice. Things tend to get smoothed out. Some issues discussed in the paper address how the narrative is constructed.

Hess has written a kind of teleological paper, communicating a sense of manifest destiny. Things are generally getting better in Chicago, and progress is based on a rational, straight-ahead model.

So, if my remarks reveal some questioning of the inevitability of success and the flawlessness of how the pieces fit together, they are not a critique of Hess but an allusion to the saying about how legislation, like sausage, is best not seen in the making. History is best not seen in the making, I suppose is the analogy.

The first problem with Chicago's accountability system is purpose and focus. Based on the evidence, Chicago is a classic case of mixed signals. Part of the second-generation reform must be focusing the signal more clearly and dealing with the consequences of focusing the signal.

Chicago must bring its students up to the national norms on the Iowa Test of Basic Skills (ITBS). As Hess suggests in his conclusion about implicit and explicit norms, this is what I would characterize as the "Lake Good Enough" phenomenon. Lake Wobegon ("where all the children are above average") anchors one end of the distribution, and Milwaukee and Kansas City anchor the other end. The problem with a norm-referenced standard for success is that it is an artificial idea. However, having test results to use as a benchmark for performance does make a certain amount of sense. But the ITBS is an inappropriate choice for Chicago. So the problem is not so much with using the ITBS or any norm-referenced test. The problem is with the target and with the idea of a norm-referenced standard as the target. It is an artificial and inappropriate construct for improvement.

Sooner or later, this is going to become a huge problem. An indication is the talk about end-of-course tests (which are geared to the standards developed outside of the school system), ways of validating those tests and standards, and how to think about alignment of the curriculum to the performance measures. All of those are imperative second-generation changes for Chicago. If, five years from now, ITBS scores are the focus of discussion in the case of Chicago, reform will not have worked.

Another issue arising from the mixed signals is the tremendous institutional conflicts embedded in the system between the first-generation and the second-generation reform. I have characterized the 1988 reform not as a reform, but as a return to ward politics in Chicago. Local...