Measuring Linguistic Complexity Independent of Plausibility

1. Introduction

Many factors affect the ease or difficulty of understanding a word in a sentence, including the lexical frequency of the word, the syntactic context, the discourse context, the intonation of the sentence, and the plausibility of the situation described by the sentence thus far (see Gibson & Pearlmutter 1998 and Tanenhaus & Trueswell 1995 for summaries of relevant factors and evidence). In order to investigate any one of these factors, it is therefore necessary to control for the others. One factor that has recently begun to be investigated in the psycholinguistics literature is the derivational complexity of different lexical structures. In order to investigate lexical derivational complexity, researchers have compared reading times and sentence-acceptability judgments for different kinds of noun-verb pairs. Unfortunately, these measures are necessarily contaminated by potential differences in plausibility between the real-world situations described by the particular sentences that are used to instantiate the structures. For example, consider McKoon and Macfarland’s (2000; henceforth M&M) evidence that externally caused change-of-state verbs are more complex than internally caused change-of-state verbs (see Levin & Rappaport Hovav 1995 for a lexical-semantic distinction between the two types of verbs). M&M demonstrated that participants were slower in deciding that externally caused sentences like 1a were acceptable than they were in deciding that internally caused sentences like 1b were acceptable.¹

(1)

a. External cause: The leaves wilted.

b. Internal cause: The signal faded.

From this evidence, M&M concluded that the externally caused materials were lexically more complex than the internally caused materials. But there is an alternative interpretation of the observed data pattern: that the set of items in the external-cause conditions described less plausible situations in the world on the whole than the set of items in the internal-cause condition. Such a difference is not intuitively obvious in 1, because both events are plausible. But across the items, there may have been a small difference in this dimension.² In general, the two types of explanations—lexical complexity vs. plausibility—will always be possible in comparisons between different sets of items, as in M&M’s comparison.

Indeed, a plausibility questionnaire that we conducted with twenty-nine participants on a single survey containing the two sets of items that M&M tested (intransitive and transitive) revealed similar differences between M&M’s conditions: the externally caused items were rated as less plausible than the internally caused items for each comparison (intransitive: F1(1,28) = 34.6, p < 0.001; transitive: F1(1,28) = 7.12, p < 0.02). Although people were asked to rate the real-world plausibility of the target sentences in this task, participants may have rated only whether the sentences sounded ‘good’ or not, which probably reflects both plausibility and lexical-syntactic complexity. Thus, like M&M’s acceptability-time task, the plausibility-rating task may reflect differences in lexical complexity of the items or real-world plausibility of the events described in the items, or both.

2. A new questionnaire method

In order to distinguish the two parameters involved in these judgments—ease/complexity of linguistic derivation of the expression vs. implausibility/plausibility of the situation described by that expression—we designed a new questionnaire format in which participants rated both of these factors independently. In this paradigm, participants were presented with pairs of sentences, many of which were good paraphrases of one another. Participants were asked to rate the degree to which the two sentences described the same situation, rating (i). If they rated them as similar, they were then asked to give three further ratings: (ii) the plausibility of the situation described by both, (iii) the complexity of the manner of expression of this situation for the first paraphrase, and (iv) the complexity of the manner of expression of this situation for the second paraphrase. This method gave us the potential to dissociate plausibility ratings (rating (ii)) from linguistic-complexity ratings (ratings (iii) and (iv)).

Before evaluating M&M’s items using this new method, we wanted to see if the method would be sensitive enough to detect...