The Search for New Measures: The ARL LibQUAL+Project--A Preliminary Report

At its October meeting last year, the Association of Research Libraries launched a pilot project to assess service quality among research libraries. The project drew its impetus from the New Measures initiative of ARL, a joint undertaking of the Statistics and Measurement Committee and the Research Library Leadership and Development Committee. At a meeting in Tucson the winter of 1999, a gathering of library leaders from across North American research libraries articulated the need to develop alternatives to expenditure metrics as measures of library performance. ¹ While the descriptive statistics that ARL has developed over the decades remain the most important, and virtually only metric of assessment--they help identify those universities that have followed a practice of sustained investment over time and they document improvements and declines in resource allocation among member libraries--there is no necessary correlation between expenditures and service quality. In fact, the relationship that emerges between expenditures and institutional excellence is widely assumed but not empirically proven. The reality of such expenditure-based metrics is that rising in the ARL rankings only requires spending more.

At the New Measures conference, participants recognized the focus on expenditures to be widely at variance with new demands for evaluation and accountability. The pilot project proposed and endorsed at the October annual meeting, derived from SERVQUAL (for SERVice QUALity), addresses user assessments of service delivery. By measuring the relationship between service delivery and user satisfaction, librarians hope to control costs by directing resources to those service quality issues identified by users as most important and most in need of attention.

The pilot project reexamines traditional methods of assessing effectiveness while testing new theories to measure delivery of services to the consumers of research library resources. It is vital to recognize that research libraries are not alone in this predicament. The libraries at all postsecondary institutions face the same dilemma. Community colleges, four year liberal arts colleges, and the many other colleges and universities supporting graduate study must also rely upon expenditure metrics to assess their libraries. Thus, the need for new measures is common to all libraries. While the project's research is grounded in the research library community, the emergent tools, with further research in other library cohorts, could be replicated at postsecondary libraries generally.

In the ARL-endorsed design, the LibQUAL+ instrument (an enhanced derivative of SERVQUAL) is being tested for score validity in comparison across libraries and across contexts, freeing institutions from reliance upon locally developed assessments. The project proposes to establish a national perspective, providing local institutions with data fine enough to diagnose local management issues. Where there are deficits, libraries will have the opportunity to make improvements that fit the local situation. A cohort of best practices across all the dimensions that define library quality may emerge, facilitating the efforts of administrators to tailor available resources to the institutional mission. Trends across the dimensions can be identified at the national level, placing local results in an important context for librarians and campus administrators alike.

Setting

At the ARL annual meeting, investigators asked for volunteers to participate in the pilot project, and thirty institutions responded. The majority of the costs for the pilot program was borne by Texas A&M University, where six years of experience with the administration of SERVQUAL provided the foundation for the study. However, each of the pilot libraries agreed to underwrite $2,000 of the costs, payable upon receipt of deliverables. We set an ambitious timeline for the project, calling for administration of the survey in April 2000 and delivery of the preliminary results at the July annual meeting of the American Library Association in Chicago.

Between October and April, Texas A&M administrators implemented a series of steps in fast-track fashion. From the institutions volunteering for the pilot, we...