Descriptive hypothesis testing is distinct from comparative hypothesis testing: Commentary on Davis, Gillon, and Matthewson

1. Introduction

I fully agree with Davis, Gillon, and Matthewson’s target article (DG&M; 2014) on some of their key points: (i) that sophisticated first-hand work on small languages should be a priority for contemporary linguistics, (ii) that hypothesis-driven elicitation is a very important technique of descriptive (= language-specific) linguistics, (iii) that many regularities of language are difficult to discover on the basis of small or moderate-sized corpora, and (iv) that comparative linguistics is not linked to a particular (meta)theoretical approach, and that generative linguists have made very important contributions to this field.

So where is the controversy? I think that the main problem is that DG&M have not framed the divisions in our field (as highlighted in Levinson & Evans 2010) in the right way. It is not hard to see that linguists who work on linguistic diversity tend to fall into two very rough sociological groups: those who are more likely to attend conferences like the Association for Linguistic Typology and publish in journals like Linguistic Discovery, and those who are more likely to attend the West Coast Conference on Formal Linguistics (WCCFL) and publish in Natural Language and Linguistic Theory (NLLT). But what kind of intellectual difference, if any, corresponds to this grouping?

My feeling is that the intellectual difference between the two sociological groups is not well understood in our field, and that many linguists who tend to hang out in one of the groups more than in the other are perhaps not committed to a particular intellectual orientation. This may be particularly so with linguists who work in depth on a few small, little-studied languages.¹ In previous work (Haspelmath 2014), I have argued that the most interesting intellectual difference is between linguists who are committed to an aprioristic approach (working with crosslinguistic categories that are given in advance by the restrictive framework) and linguists who have no such commitment and are open to discovering completely new categories, and who are also open to diverse ways of explaining the generalizations they find. It seems to me that to a large extent, DG&M’s work (and the other work they report) falls into this latter category. However, they adopt one key idea from the aprioristic approach: that descriptive hypothesis testing is the same as comparative hypothesis testing. I discuss this assumption in §2 and argue that descriptive hypothesis testing is very different from comparative hypothesis testing.

That DG&M are critical of large-scale comparison as epitomized by the World atlas of language structures (WALS; Haspelmath et al. 2005) is not surprising, as we have received a fair amount of criticism for supposed inaccuracies from descriptive linguists. I address their criticism of WALS in §3. DG&M reject the idea that a special set of comparative concepts must be used for crosslinguistic comparison (see their n. 5, p. e185), but I would argue that their own work also appeals to such special comparative concepts (see §4). I end by making a few more comments on the relation between sociological and intellectual divisions within the field of diversity linguistics (§5).

2. Descriptive vs. comparative hypothesis testing

2.1. Descriptive hypothesis testing

DG&M rightly observe that a thorough description (or analysis) of a language system requires experimental as well as observational evidence: to determine the precise boundaries of a generalization, negative judgments are very helpful and sometimes indispensable. Hypotheses about generalizations must be checked and replaced by new hypotheses, these must be checked again, and so on.

But it is not necessary to base the hypotheses on what is known from other languages, as DG&M seem to imply. In each of their case studies, they start with a hypothesis that derives from another language, generally English. In case studies 1 and 2, they start with the idea that their Pacific Northwest languages are like or unlike all other languages, and in case studies 3–5, they start with the idea that their languages are like or unlike English. Of course, this is the simplest method, but...