International and Language Education for a Global Future: Fifty Years of U.S. Title VI and Fulbright-Hays Programs

14. Beyond Accountability: A Balanced Approach to Assessment and Benchmarking

Developing a national capacity of proficient language users is a central purpose of Title VI. The need was urgent in 1959, and is felt even more so today. In these fifty years, however, there has been no systematic measurement of language outcomes from Title VI programs. Title VI programs do indeed produce proficient speakers of a wide variety of languages, and the means to document these outcomes are generally available. Why, then, have National Resource Centers (NRCs) been unable or unwilling to assess students’ language abilities? Conversations with colleagues reveal a variety of reasons: • No language test could possibly capture the richness and depth of language and culture, much less students’ intellectual development, in a liberal arts curriculum. • Forcing standardized bubble tests on campuses like some postsecondary version of No Child Left Behind would undermine academic freedom. • We are still waiting for a reliable test that quickly and inexpensively measures everything a student knows. • Every minute and every dollar given to testing is a minute and dollar taken from learning. These arguments are all quite reasonable at face value, but are based on an outdated paradigm of language testing that equates assessment with measurement, C H A P T E R 1 4 Beyond Accountability: A Balanced Approach to Assessment and Benchmarking Carl Falsgraf 303 usually in the form of a single, high-stress, timed event with a purpose to sort out winners from losers. I will argue in this chapter that the future of benchmarking and assessment in Title VI can and should move from the hegemony of measurement to an era of balanced assessment, in which multiple measures of linguistic and academic progress create a rich portrait of student ability consistent with individual student goals and local educational context. Performed correctly, this approach can not only keep political dogs at bay but also enhance students’ motivation, help us advocate for our programs, and guide curricular and instructional decisions. This conceptual shift provides us with a more productive and practical way to look at assessment in Title VI and in language and culture education in general. The Measurement Paradigm The measurement paradigm has been so ubiquitous that we scarcely recognize it as a simple construct we can choose to change. The central tenet of the measurement paradigm is that a fairly stable body of knowledge and abilities exists in a learner’s brain that scientific techniques of elicitation and mathematical models of analysis can objectively and quantitatively measure. The SAT, GRE, and TOEFL are common examples of this technology. The degree to which various instruments actually measure that body of knowledge and abilities is the subject of highly technical and impassioned arguments within the testing field. For the sake of argument , let us assume here that these instruments are generally reliable and valid. Even if we had instruments that measure knowledge and ability perfectly, do we not care about other educational outcomes, such as the application of knowledge, critical thinking, cultural insight, and creativity? Is our sole purpose for assessment the generation of a number summarizing a learner’s knowledge base? Certainly, a liberal or professional education should provide students with a body of knowledge and abilities, but to ignore more complex outcomes simply because they cannot be quantitatively measured in a timed test and reduced to a number sells ourselves and our students short. Assessment should be an opportunity to demonstrate excellence, not just an exercise in sorting students from “best” to “worst.” We are politically and educationally foolish not to document the most important product of an international education: a global outlook allowing students to gather, analyze, and think critically about new information in a cultural context. The search for a magic bullet, the one big test that will tell us everything we need to know, should end. I say this as the director of a U.S. Department of Education -designated Language Resource Center (LRC) with test development at the core of its mission. Although tests have their place, we must not limit ourselves to measuring the relatively small subset of outcomes amenable to elicitation by 304 C A R L F A L S G R A F standardized testing. The alternative to exclusive reliance on testing is not subjective evaluation but a contextualized portrait of student learning, a mosaic of quantitative and qualitative evidence. Balanced Assessment Systems: The Educational Imperative Following Stiggins (2008), I will refer to this orientation...