The Workforce Investment Act: Implementation Experiences and Evaluation Findings

9 - Recent Advances in Performance Measurement of Federal Workforce Development Programs

233 9 Recent Advances in Performance Measurement of Federal Workforce Development Programs Randall W. Eberts Timothy J. Bartik Wei-Jang Huang W.E. Upjohn Institute for Employment Research The purpose of performance measurement is to enable federal, state, and local workforce agencies to track the progress of program participants in achieving the core goals of programs under WIA: finding a job, retaining a job, and receiving adequate earnings. Performance measures are also used to hold management accountable for the effectiveness of the services delivered to help participants achieve those goals. The ETA has established three measures to capture these three goals for adult and youth programs: 1) entered employment, 2) job retention, and 3) earnings levels. Each state negotiates with the USDOL to set state targets, and the states in turn negotiate with each of the roughly 600 local Workforce Investment Boards (WIBs) to determine local performance targets. As this practice of setting standards evolved over the past decade, states and WIBs increasingly found that negotiations were not taking into account factors that affected their performance but were beyond their control and not related to the services they provided. These factors include the conditions of the local labor market and the personal characteristics and work history of participants in their programs. Without accounting for differences in these factors across states and across WIBs, those entities with more favorable labor market conditions or more capable participants are likely to have higher outcomes, and those for which these factors are unfavorable can expect lower outcomes. Differences in these outcomes are not the result of how well service providers have met the needs of their customers, but of factors outside 234 Eberts, Bartik, and Huang their control and extraneous to the effectiveness of their service delivery . Therefore, the measures are not fulfilling their intent of measuring the value added of the workforce system, and may even distort decisions by administrators of whom to enroll in workforce programs. In response to these concerns about the measurement and setting of performance goals, the ETA has contracted with the W.E. Upjohn Institute for Employment Research to adjust national performance targets for differences (actual and forecasted) in unemployment rates. To make adjustments, the Institute estimated the relationship between individual participants’ performance outcomes and local unemployment rates. These adjustments are incorporated in President Obama’s annual budget request and the national performance targets.1 In addition, the ETA, through the help of the Upjohn Institute, is exploring procedures to adjust state and local WIA performance targets for factors that affect performance outcomes but are outside the control of state and local administrators. This procedure provides a systematic , transparent, and objective method to set WIA performance targets; it helps to level the playing field by making the targets neutral with respect to the observed characteristics of WIA participants and of the local labor market conditions in which they seek employment. It also provides a more accurate measure of the value added of WIA programs at both state and local levels by controlling for observed factors that affect outcomes but are unrelated to the services provided by the workforce development system. The purpose of this chapter is to describe the two procedures of adjusting performance targets for economic conditions and personal characteristics. The first procedure adjusts the national performance targets for changes in unemployment rate, and the second adjusts state and local performance targets for differences in local market conditions and personal characteristics. The contribution of both sets of factors is estimated using one general model that relates performance outcomes (the common measures) to unemployment rates and personal attributes. The chapter is divided into two major parts. The first part describes the general methodology and then provides estimates of these effects for each of the common measures for each of the three WIA programs. The second part demonstrates how these estimates can be used to adjust performance outcomes at the national and state levels for differences in these factors. Recent Advances in Performance Measurement 235 ACCOUNtING FOR DIFFERENCES IN LABOR MARKEt CONDItIONS AND PERSONALAttRIBUtES Adjusting for differences in labor market conditions and personal characteristics is not new for the workforce system programs. WIA’s immediate predecessor, JTPA, used statistical analysis to adjust performance targets for a list of factors which were deemed outside the control of administrators. The adjustment procedure that the ETA has adopted to adjust national performance measures and that the ETA is considering to...