In lieu of an abstract, here is a brief excerpt of the content:

15 3¥¥ Planning for Useful Evaluation Users, Uses, Questions TODD H. McKAY and JOHN McE. DAVIS THE STRATEGIES DESCRIBED IN the previous chapter are designed to help readers understand general best practices for conducting a successful evaluation project. In this section, we move to the key concepts of planning the evaluation project itself, and in particular, use-focused evaluation concepts that will help the project to be productively useful and achieve important program aims. During the early stages of an evaluation, there can be a common and premature move by novice evaluators to start planning immediately for data collection.“We need a survey!” is the common refrain shortly after the project has begun.A lesson long-learned in evaluation research and practice is that deciding how to collect information before clarifying who needs to know something from the evaluation, what those people need to know, and why they need to know it leaves an evaluation project dangerously prone to collecting information decision-makers do not want or need. Evaluation in such instances fails to fulfill its potential and purpose, resources and time are wasted, and a valuable opportunity to better understand or improve language instruction through evaluative inquiry is missed. How, then, can this state of affairs be avoided? In a use-focused approach to evaluation , three strategies are needed at this initial stage of evaluation planning: 1. Identifying and engaging specific users of the evaluation findings 2. Identifying users’ evaluation uses of the evaluation information 3. Creating evaluation questions that capture the focus (or foci) of the evaluation project A use-focused evaluation approach involves deliberately and explicitly clarifying each of these project components. Doing so helps ensure that the results of an evaluation are actually used to make important decisions and take needed programmatic actions. This chapter discusses each of these planning components in detail. 16 Todd H. McKay and John McE. Davis Evaluation Users, Stakeholders Language programs are complex. They involve many processes and components, including instructional activities, curricula, learning objectives or outcomes, assessment, teaching and learning materials, cocurricular activities, and professional development. Each of these components involves various program actors and agents, such as students, instructors , administrators, funders, program managers, alumni, and parents. In almost all educational contexts, a diverse range of individuals and entities will likely be affected by an evaluation project. Many will be impacted by the evaluation’s effects and consequences. Some may have responsibility to take action on the basis of its results; others may be held accountable for program performance and the extent to which goals or outcomes are being met. In a use-focused evaluation approach, evaluation is conducted in an intentional way to meet the needs of specific individuals. As such, part of the evaluation process involves sorting out who should or could use evaluation findings toward some programmatic or educational goal. To this end, evaluation planning requires identifying, engaging, and involving two types of program constituents: stakeholders and evaluation users. Stakeholders Stakeholders are people who take an interest in an educational program.They are the individuals or groups that will be affected—either directly or indirectly—by evaluation findings or actions taken on the basis of findings.There is a fundamentally democratic element to high-quality evaluation practice in that the individuals who have a stake in the language program should not be separate or excluded from the evaluation process, especially during the planning phases. Since program stakeholders are closely tied to the purposes of an evaluation, evaluation success is hindered when stakeholder perspectives are overlooked. Without stakeholder involvement, there is less understanding, appreciation, information sharing, legitimacy, or commitment to evaluation (Bryson, Patton, and Bowman 2011). By contrast, when there is stakeholder participation in evaluation, people develop project “ownership”—they care about the evaluation since it responds to their needs and priorities . If an evaluation is planned well, with active stakeholder engagement, interested parties will anticipate evaluation findings since results will shed light on things they care about. In this way, stakeholder involvement and ownership increase evaluation usefulness. The evaluator or project team must organize participation for stakeholders throughout the evaluation project. Students (often overlooked), teachers, faculty, instructors, teaching assistants, chairs, principals, supervisors, advisors, administrators, funders, parents , future students, employers, alumni, community leaders, and others all potentially have a stake in the outcomes of an evaluation project, and the evaluator must find ways to promote stakeholder engagement (see below). Evaluation Users A key subgroup of stakeholders in a use-focused approach to evaluation are the evaluation “users.” In...


Back To Top

This website uses cookies to ensure you get the best experience on our website. Without cookies your experience may not be seamless.