Our cultural institutions host large collections of audio recordings comprising important cultural artifacts. Some of these recordings date back to the nineteenth century and up to the present day. These recordings include music but also poetry readings, field recordings, and presidential speeches and phone calls, as well as the only recordings of languages, oral traditions, and voices that we no longer remember. We have dedicated significant resources to digitizing these collections, yet, even digitized, these artifacts are only marginally accessible for listening and almost completely inaccessible for new forms of access and scholarship. In order to discover convergences in seemingly divergent theories that may guide how we build information infrastructure around our sound heritage, this article considers how early information theory, much of which was crafted within the context of developing communication and sound technologies, can provide a framework for thinking through how to build an information infrastructure that facilitates inquiry with digital audio collections in the humanities.