In lieu of an abstract, here is a brief excerpt of the content:

69 8¥¥ Questionnaires for Evaluation AMY I. KIM and JOHN McE. DAVIS QUESTIONNAIRES ARE A VERSATILE tool for use in program evaluation. They elicit information by having respondents choose answer options (e.g., multiple-choice questions or agreement ratings) or by supplying written or typed text (e.g., on items that ask for comments). Like focus groups and interviews,questionnaires are another way of collecting stakeholder views on the educational processes happening within their programs. Questionnaires can be used to obtain any of the types of information that people can supply about themselves or their environment, such as their perceptions, attitudes, feelings, opinions, preferences, behaviors, or perceived language abilities. Questionnaires are arguably the most commonly used evaluation tool, though they can be effective for certain evaluation-related aims and less effective for others. In this chapter, we address some key aspects of designing and administering questionnaires in order to provide useful information for language program evaluation. Evaluation Questionnaires: Advantages As noted throughout this guide, the instinctive and quick impulse to use questionnaires for program evaluation presents certain risks. Questionnaires are effective for providing certain types of information about a language program that may or may not be useful depending on stakeholders’ and users’ evaluation needs. The various ways in which questionnaires are helpful for evaluation projects can be divided into two broad categories: (1) the logistical advantages issuing from questionnaire administration, and (2) the advantages provided by questionnaire-type information. Questionnaires are useful for evaluation purposes because of a number of administration or logistical advantages. A main advantage is their efficiency; questionnaires are quick to develop, easy to administer, cost-effective, and accessible from just about anywhere via mail or email. These advantages taken together lead to a main administration benefit: information can be collected from large numbers of respondents.If a project needs 70 Amy I. Kim and John McE. Davis to capture the views of a large number of students, teachers, alumni, or parents, questionnaires will often be the best option to shed light on what a large group thinks about programmatic issues (and the best option to collect those views in a relatively short period of time). A number of information-related advantages comes from the ability of questionnaires to reach large groups of evaluation informants.Historically,the main purpose of questionnaire research was to find out what a group of individuals thinks about a specific topic, garnering those views on the basis of a smaller sample of respondents (Dillman, Smyth, and Christian 2014). Likewise, evaluation purposes will often call for knowing whether certain known program elements are “useful” or “effective,” ranking program elements for importance, indicating agreement with statements or assertions about the program, rating language abilities or attitudes about learning, and so on. As such, evaluation questionnaires can identify notable patterns of opinion in entire groups of program stakeholders. Moreover, if questionnaire administration is conducted carefully, results from questionnaires can be generalized to a population, which can be useful—or even necessary—for certain evaluations where it is important to know about majority views on important program issues. Doing so, however, requires a systematic approach to sampling (i.e., probability sampling) and high responses rates. Yet, if these things are done, questionnaires can collect large amounts of representative information that capture what a majority of stakeholders thinks about particular program elements. Another important administration benefit of questionnaires is that data collection and analysis is quick and relatively efficient. Online questionnaire design and administration software, such as Google Forms, SurveyMonkey, and Qualtrics (among others), permits users to quickly download results in various file formats. Many online questionnaire platforms also allow users to automatically generate charts, figures, and other forms of data visualization that can be used to instantly analyze results. An additional information-related advantage of questionnaires is that they provide numerical, quantitative data. Percentages or counts of yes/no or true/false responses; proportions of selected responses; and average ratings of usefulness, effectiveness, agreement, or language abilities are parsimonious, easy to interpret, and allow for statistical analysis (see chapter 9). In addition, quantitative results may satisfy particular users’ data preferences (e.g., stakeholders who have a desire for “hard,” statistical data), which may be needed for buy-in from key project stakeholders. Finally, questionnaires have the ability to reduce certain types of response bias. Since questionnaires offer a certain amount of anonymity compared to interviews and focus groups, respondents are more likely to provide honest views and opinions. In addition, since there is no interviewer...


Back To Top

This website uses cookies to ensure you get the best experience on our website. Without cookies your experience may not be seamless.