Universities and Colleges as Economic Drivers

4. On the Measurement of University Research Contributions to Economic Growth and Innovation

97 4 ON THE MEASUREMENT OF UNIVERSITY RESEARCH CONTRIBUTIONS TO ECONOMIC GROWTH AND INNOVATION MaRyann P. fELDMan, aLLan M. fREyER, anD LauREn LanaHan aBstraCt The increasing complexity of university scientific research and the increasing pressure for accountability in public expenditures creates a challenging environment for the measurement of scientific productivity . Of course, the choices of metrics employed create different incentives for institutional actors at the federal, state, and university levels. This chapter uses the example of the Center for Environmentally Responsible Solvents and Processes, an NSF-funded science and technology center over its ten-year span to assess economic impact, research outcomes, and public benefit. First, we conduct an economic impact assessment using the IMPLAN methodology for input-output analysis. Next, we analyze conventional knowledge and technology transfer metrics. Our third analysis attempts to consider the social, cultural, and educational public benefits from the research center. Then we employ ex post thought experiments to compare the center with other modes of federal research funding. We advocate a broad, holistic, and case-specific approach to the evaluation of multidisciplinary research centers, combining traditional research outcome 98 FELDmaN, FrEyEr, aND LaNahaN and knowledge transfer metrics with social, cultural, and educational measures. INtroDuCtIoN The mission of universities has become more complex, encompassing commercialization goals and economic development along with the traditional goals of educational and research excellence. Rather than constituting the model of a single-investigator project, the nature of university research practice has come to encompass a range of organizational forms, such as multidisciplinary and interdisciplinary collaborative research endeavors that span across universities, government labs, and industry (Aboelela et al., 2007). The motivation is that the applications with the greatest potential for economic growth often exist at the intersection of disciplines and require the integration of diverse forms of knowledge (Metzger & Zare, 1999). To increase the practical impact of university research, many state and federal programs require university research projects to involve industrial actors, which may include large firms, industry consortium, and new start-ups firms (Hagedoorn, 2002). Increasing in both scale and scope, the university research enterprise not only supports fundamental research but also delivers results in the form of economic outcomes and societal impacts (Mallon & Bunton, 2005). The result is that the academic research enterprise has evolved substantially to develop new multifaceted capabilities. Alongside these formative changes, state and federal funding agencies have come to demand greater accountability of the results of public investments in academic research. These demands for accountability to political authority, especially to the U.S. Congress— building on the 1993 Government Performance and Results Act (GPRA)—have remained central over the past decade. The Office of Management and Budget’s recent 2009 memorandum, “Increased Emphasis on Program Evaluations,” is one recent example of pressure for research evaluation and accountability (Chubin et al., 2009). These requirements, however, present particular challenges for the evaluation of multidisciplinary and interdisciplinary collaborative university research programs (Bozeman & Boardman, 2004). Measurement of University Research Contributions 99 Traditional measurements of research outcomes have relied on assessments of economic impact, such as the multiplier effects associated with expenditures, or focused on metrics of knowledge creation and technology transfer, such as publications, patents, and spinoff companies. These metrics capture the more tangible outcomes that result from university research; however these measures alone fail to capture the spectrum of results (Cozzens & Melkers, 1997; Ruegg & Feller, 2003; Wagner et al., 2011). A given university research project has the potential to span many years, leveraging additional resources —both public and private, training and education, knowledge generation and transfer, dissemination via publications, technology transfer products, job creation, and greater societal gains. Moreover, traditional performance indicators limited to economic impact assessments and technology transfer may underestimate the total outcomes of the research project (Wagner et al., 2011). This formative change and evolution in university research practices demands new evaluation metrics that look beyond the direct tangible outcomes (Wagner et al., 2011; Kabins, 2011; Chubin et al., 2009). The choice and use of metrics are critical as they will subsequently influence the conduct of scientific research and define the nature of the research that is undertaken and may even delimit the resulting contributions. A focus on easily measurable outputs may skew attention toward less risky and more immediately realized countable metrics. The nature of large-scale university research projects inherently incorporates learning and adaptation as it proceeds. A lack of immediate results may indicate the identification of new and relevant problems...