Full-Text Databases and Historical Research: Cautionary Results from a Ten-Year Study

Full-text electronic searching of books, periodicals, and government documents is providing historians with the ability to survey unprecedented amounts of historical material for references traditionally not associated with those sources. Yet because the research conducted with these new tools is often not possible or practical using other methods, the shortcomings of the results are not always evident. This article highlights some of those shortcomings through a detailed discussion of a ten-year study that spanned the development of some of the main databases now used in nineteenth-century British history. The study was designed to uncover the number of newspaper reports discussing sex between men in three major London newspapers between 1820 and 1870, ultimately detecting over one thousand such reports. The study was designed in 1999 using the Palmer's Index to the Times on CD ROM in conjunction with archival sources, and those results were contrasted in later years to the results from the Times Digital Archive, the Palmer's Index to the Times within C19: The Nineteenth Century Index, and the British Library Newspapers 1800-1900 databases. The inability of newer tools to detect large amounts known material sparked a systematic crosschecking of the results of these databases to map the extent of the discrepancies. The article discusses some major shortcomings of these new tools, demonstrates methodologies for working around the limitations, and highlights some of the new findings related to the public discussion of sex between men obtained through this study.