The Missing Link: Assessing the Reliability of Internet Citations in History Journals

One of the bedrock values of professional historians is reliance on verifiable documentation. We have well-developed methods for storing, citing, and finding many media, including books, journals, and archival material. Recently, historians have relied on a new medium, the internet, to document the sources of their information. Scholars in the sciences, however, have raised alarms about the frequency with which internet sources have disappeared after their citation in journals. An influential article in Science found that 13 percent of internet citations in three leading journals were inactive within twenty-seven months of publication. In five leading medical journals, 4.4 percent of internet citations were inaccessible within three months of publication. In six oncology journals, 33 percent of internet citations decayed within twenty-nine months.¹

Do humanists and social scientists face the same problem? This research note marks the first published attempt to answer that question in any field of the humanities or social sciences.² We examined the reliability of worldwide web citations in two leading history journals (Journal of American History and American Historical Review) over seven years and found that 18 percent of web links cited over that period were inactive. The problem increased over time. In articles published seven years earlier, 38 percent of web citations were dead. A digital archive enabled us to locate 57 percent of the missing web pages, leaving 43 percent unavailable even to scholars who use the archive. These findings suggest that historians (and probably other humanists) face a major problem in scholarly practice: we are citing internet sources as though they were permanent, when in fact they are ephemeral.

Readers who have used the internet already know that web links die. The contribution of this research note is to quantify the extent of the problem and place it on our professional agenda. Historians of technology are well positioned to consider this problem because of our focus on the interaction between tools and social practice.

In the first section, we briefly survey the development of source citations as integral parts of historical practice. The key point here is that historians over time increasingly valued comprehensive, verifiable references for information in their texts. The footnote evolved in tandem with this desire, especially as history professionalized in the nineteenth century. Central to the success of this system was the reliance on a particular technology, paper. The second section comments on the introduction of a new technology, the internet, as a historical source. In the third section we turn to the heart of this research note, our quantitative analysis of the reliability of the internet as a source for historians. The fourth section discusses some efforts to address the problem of "link rot" by creating archives of web sites, and it describes the way we tested the completeness of the archive. We conclude with a call for professional societies and journals to create better means for ensuring the durability of internet citations.

Documentation and Paper

Before the modern era, historians felt little need to document their sources in footnotes. Political historians wrote under a rhetorical tradition that focused on lessons in virtue and vice, and they prized the conveyance of moral and political lessons that would be valid in all times and places. They alluded to authorities, but often eschewed citing chapter and verse and expected readers to trust their veracity. Some fields, notably law, developed systems for citing sources as early as the Middle Ages, but these practices apparently had little impact on historiography.³

The footnote flowered in the nineteenth century as a way to prove historical arguments. During the eighteenth century, the footnote had been an entertaining form of literary art, and its popularity in fiction helped the footnote spread to historical writings. Its importance grew in the nineteenth century as history professionalized. Archives and libraries opened their collections to more scholars, including younger historians, who now had to demonstrate their mastery of a literature rather than allude to authorities. A professional culture developed in which historians made their arguments in the text and...