The Workforce Investment Act: Implementation Experiences and Evaluation Findings

8 - Lessons from the WIA Performance Measures

209 8 Lessons from the WIA Performance Measures Burt S. Barnow George Washington University Since the late 1970s, major federal workforce development programs in the United States have included performance management systems that assess how well the programs are performing at the national , state, and local levels. The use of performance management in workforce programs predates the more general congressionally mandated performance requirements of the Government Performance and Results Act (GPRA). This chapter draws on the previous work of the author and others in assessing the lessons of the past 30 years of experience with performance management in workforce programs. Although the chapter focuses on the U.S. system, the lessons should apply to programs in other countries as well. The chapter first discusses what performance management is in the context of workforce programs. Next, performance management is contrasted and compared with program evaluations. Policy officials would like to implement performance measures that are based on program impact ; the next section describes why that is generally not possible to do and presents empirical findings on the success of such efforts. The following section describes how the performance management system used for U.S. workforce programs can lead to unintended results and summarizes some of the research on this topic. This is followed by a discussion of whether standards should be absolute or adjusted for factors such as participant characteristics and economic conditions. The final section presents lessons for countries that are considering establishing a performance management system. Although related, the concepts of performance measurement and evaluation are distinct and serve different purposes (see Blalock and Barnow [2001]). Performance measurement is a management tool that 210 Barnow is used to monitor implementation on a real-time basis. Performance measures may track data that indicate fidelity in program implementation , inputs (such as participant characteristics) that are considered important to the program’s purpose, process measures (e.g., use of “best practices”), outputs expected from the program, and sometimes shortterm gross outcome measures. Program evaluation, on the other hand, is intended to answer specific questions about programs. Process studies document what happened while the program was implemented, impact evaluations assess what difference in outcome measures was due to the intervention, and cost-benefit analyses assess whether the benefits of a program exceed the costs. If the program has limited capacity, participant characteristics may be a useful performance measure, and one or more measures could be established to track the characteristics of customers served.1 Process measures rather than output or outcome measures are sometimes used. For example, if particular practices are known to be more effective or less expensive than the alternatives, a case can be made for including process measures of performance. In the current health care reform debate in the United States, some advocates argue that costs can be driven down by requiring providers to use best practices or by providing financial incentives to do so; similar arguments can be raised in setting standards for education. In the past, however, some in the workforce field have argued that so long as the grant recipients are held accountable for the desired results, they should be free to adopt the approach they believe is best rather than relying on processes prescribed by the federal government.2 A reasonable approach might be to monitor use of best practices and provide technical assistance, rewards, and sanctions only when an organization fails to achieve satisfactory outcomes. In a system characterized by delegation of authority from the central government to lower levels of government (state and local government for many U.S. programs, but the concepts apply to a system of grantees or for-profit contractors as well), the goals of the level of government providing the funds may not be aligned with the goals of the level of government providing the services. By instituting a performance management system that provides rewards and sanctions based on how well the lower level of government meets the goals of the funding agency, the so-called principal-agent problem can be (in theory) resolved. Lessons from the WIA Performance Measures 211 The differences between performance measurement and evaluation often are matters of depth of analysis and causality. Because of the need for rapid feedback, performance measurement activities generally track easy-to-collect data on inputs, activities, and outputs. Data for performance measurement generally come from management information systems maintained by the programs and from administrative data collected for...