Inference and Generalizability in Applied Linguistics: Multiple Perspectives (review)

This book is a collection of chapters inspired predominantly by the 2002 meeting of the American Association for Applied Linguistics (AAAL) and its meeting topic: 'Drawing the Line: The Generalizability and Limitations of Research in Applied Linguistics.' The introductory chapter from one of its editors (Chalhoub-Deville) characterizes AAAL sessions as having 'brought together researchers working in different areas of applied linguistics to share their views and debate issues,' and this meeting of minds appears also to describe the aims of this volume. The book begins with seven chapters that provide individual perspectives, and ends with three thematic discussions of those perspectives; each deals to some extent with the questions of how applied linguistic data can or should be generalized, validated and inferred from, and how researchers can or should justify such generalizations and inferences.

As a linguist who runs experiments (though I do not work in the applied domain), I find it laudable that a disparate group of researchers chose to address such huge and slippery questions, and the result does at least kick up some intellectual dust. To take one example, the authors collectively highlight just how many different kinds of inferences we make from tests and experimental results: inferences about other tasks, other learners, other populations – as well as other domains, other theoretical constructs, and other predictions. The range of viewpoints is certainly wide, impressively or perhaps dauntingly so; some authors seem to disagree on such fundamental points as why exactly applied linguistic data are studied at all.

While I see the virtues of holding these discussions in workshops or group sessions among experts, it regrettably has not translated well into a printed collection. Many chapters are reminiscent of invited talks given to audiences known to include both experts and non-experts, in which too little is asserted to be useful to the former and too much assumed to be clear to the latter.

The volume is presumably aimed at applied linguists, mostly focusing on L2 language pedagogy research, so its wealth of jargon and offhand references to what are presumably seminal works in the field are understandable. Yet many chapters seem to do little but raise broad questions and call for paradigm shifts of a vague and unexplained nature. The chapters I found most interesting were those built around their authors' own area of research, giving concrete arguments as to how and why results from their type of studies can be generalized. One of the nicer examples from my perspective is Patricia Duff's contribution, which compares, among other things, the generalizability and external validation of quantitative and qualitative research paradigms, using several clear examples from her own work. (In fact, one recurrent theme of many chapters is the tension – or sometimes the asserted lack of tension – between quantitative and qualitative methodologies; it may well be that these chapters are more informative about that lively discussion than about other topics.)

It should be said that, taken as whole, these papers do come to a number of consensuses and conclusions – mostly in pointing out pitfalls that all linguistic research can face (on both sides, I would add, of the theoretical/applied divide). Many come in the form of caveats: for example, that in order to infer support for a theoretical model from data, one must know what the theory really is, and what it is a theory of (e.g., of task performance or of linguistic knowledge); that when averaging across heterogeneous data, one must either be very limited in one's conclusions or very rigorous in one's justifications; and so forth. The last three discussion chapters provide many further such caveats, especially from philosophical and ethnographic viewpoints, which may be of interest to researchers at the relevant interfaces.

In the end, this volume may serve best as the fodder for a literature review, or as a tool to start more concrete discussions about different methodologies and their reliability, validity, and best...