From Words to Numbers: Narrative, Data, and Social Science (review)

What is the reader to make of a 475 page book in the Structural Analysis in the Social Sciences series that contains 56 pages of notes, 44 pages of references, and 334 pages of text bracketed by two poems in Italian? That ranges through, amongst others, sociology, linguistics, semiotics, history, and alchemy? That within the span of a few pages goes from abstract questions of epistemology or linguistics to technical details and practical advice on sampling and coding? First, this is not a standard methodological treatise on content analysis: when the author calls From Words to Numbers a kind of alchemy, and uses the metaphor of a journey to describe the book, you know that there is more here than a postmodernist's nightmare of positivism run amok. Second, the author is tremendously erudite and unafraid to show it (e.g., a single paragraph on page 19 quotes Levi-Strauss, Sorokin, Mandeville, T.S. Eliot, and Peter Berger). Scattered throughout the book are brief histories of content analysis, the quantitative study of political conflict, the development of text or story grammars in linguistics and psychology, pilgrimages and voyages of discovery. Third, a thread of postmodern irony and playfulness runs through the book, as he recognizes the likely reactions of various types of reader (skeptical historian, frustrated methodologist, Faust! – 131), and even (quoting Calvino) inquires if the reader needs to pee before proceeding with the rest of the chapter.

Readers familiar with Franzosi's previous methodological papers will know the basic story of From Words to Numbers. From linguisitics he adopts the concept of story grammars and combines the resulting semantic triplet of actor-action-object (and their modifiers) to collect and code narrative data on protest from newspapers, using relational database management systems to store and retrieve the data (with useful practical instructions on how to implement the method using commercial software). Like Shapiro and Markoff's work on les cahiers des doléances in Revolutionary Demands (1998), Franzosi tears content analysis coding away from the particular theoretical concerns of the researcher, thereby simplifying the coding work and leaving theoretical decisions to the researcher, not the coder. This method preserves more information than traditional methods, most importantly that concerning the relation between subject and object embedded in the narrative. Consequently network models and their graphic representation can be used to analyse the data.

In addition to outlining the methodology and its theoretical underpinnings, Franzosi engages in serious and thoughtful consideration of data quality issues for researchers on protest events who use newspapers as sources of data. The critiques are familiar: selectivity and biases of newspaper reporting depending on the type, size and other characteristics of events, their place in media issue attention cycles, and their distance from the place of publication; so are the defenses: there's no alternative, large-scale events are consistently reported, and once known, biases can be taken into account. He offers useful suggestions on how to select newspaper sources and how to check for biases using additional independent sources. The emphasis on reliability in content analysis is misplaced, he argues: the real problem is validity, and he criticizes tradtional content analysis for its naïve faith in coding instructions, neglect of problems of meaning, and adherence to rituals of quantitative social science. There is also considerable critique of standard quantitative methods in social science and history, much of which will be familiar to readers of Cicourel (Method and Measurement in Sociology, 1964) and Lieberson (Making it Count, 1985).

Having outlined the method, Franzosi keeps returning to the key issue: what are its benefits and limitations? The examples he gives from the mounds of data produced by his research on the Red Years (1919 – 1920) and the Black Years (1921 – 1922) in Italy are not enough to convince a sympathetic but skeptical reader that this tremendous labour tells us anything that existing histories do not — they might, but unlike Shapiro and Markoff, who show that...