How Universities Work

CHAPTER 11 Quality

115 Chapter 11 Quality Quality is among the most elusive of academic measurements. Everyone speaks about it, everyone is in favor of it, and most academics believe they know it when they see it. But when we ask for specific measures of quality for departments, colleges, and universities, the task immediately becomes remarkably complex. The Top American Research Universities publication groups research universities into clusters defined by nine indicators of quality. For the purposes of identifying clusters of high-quality research university campuses, this methodology serves admirably , even if it is not without controversy. However, the measurement of quality within the university at high enough levels of specificity and reliability to drive an effective budget process poses an additional challenge. Budgeting to drive improvements in quality requires reasonably robust measures if it is to allocate dollar rewards effectively . Unless the university can measure something, it cannot reward its improvement. Incentives and the behaviors they generate require clear measurement. 116 HOW UNIVERSITIES WORK Quality: The Main Event Although quality and productivity receive equal treatment in measurements for the budget, they do not play the same role in institutional improvement. Quality is the main event because the best universities are quality engines acquiring the highest-quality students, faculty, and educational and research programs. Quality is an end in itself. In theory, perhaps, if an institution had one faculty member and one student, both of the highest quality in the world, it could be satisfied. This is the theory of extremes that drives sports and other winner-take-all enterprises . In universities, however, this approach is not effective. Universities do not have seasons, and they do not win championships for their academic performance. Universities serve as stable, permanent, and continuing generators of knowledge, learning, and associated services and benefits to society. What matters to universities is sustained high performance in both productivity and quality. This is why we call universities quality engines. Productivity speaks to the university’s commitment to deliver a substantial volume of goods and services in exchange for investment by individuals, state and federal governments, and other support groups. Quality speaks to the university’s commitment to deliver products and services that are nationally and internationally competitive. The institution’s success as an enterprise comes from its ability to compete in the marketplace for a substantial share of quality. Quality is scarce, and the competition for it intense. The relationship between productivity and quality often escapes academic observers, primarily because they underestimate the importance of both productivity and quality in acquiring the money needed to excel within the national and international quality marketplace. Quality 117 Context for Measuring Quality Performance budgeting approaches quality from two perspectives : national and internal. Quality for research universities must be measured in a national context. Quality is not an abstraction but a specific measurement against a national standard . Each guild has its own national standards of quality, and what counts as quality for one guild may well be irrelevant for another. To take a simple example, historians look to books and booklength monographs as the primary quality products they evaluate . Journal articles are significant, but they do not substitute for books. In scientific fields, however, the journal article is the primary vehicle for demonstrating research quality. In some fields, single-author publications determine quality, where in others multiple author publications are the norm. Science fields measure grants, while fine arts, theater, and music look to exhibits and performances. The variations on the definitions of quality multiply along with the expansion of fields and subfields within each guild and its subdisciplines. Although measuring the quality of this complex mix of academic products may appear hopeless, university people are quite good at evaluating the quality of their colleagues’ work in the same fields in other institutions. They tend not to explain the process to outsiders clearly, but they know how to measure quality, and if pushed, they can be explicit in the evaluation procedure. If a university adopts performance budgeting and if it provides significant rewards for demonstrated quality improvement , the faculty will develop effective quality measurement. The measurement will not be perfect, of course, but it will be close enough to provide a fair basis for distributing rewards. Experience demonstrates that quality measurement requires two dimensions: an external benchmark relative to the national competition and a local measurement of improvement. 118 HOW UNIVERSITIES WORK National Benchmarking of Research Ideally, all quality measurement would involve...