This article listens to the role played by sound and silence in Civil War prisons. Applying the methodology of sensory history, it argues that the experience of captivity cannot be understood without considering the aural environment of prisons, including patterns of listening, conflicting interpretations of the same sounds, and the role of sound in resistance. Although war was noisy for participants in general, prisoners had good reason to infuse the aural environment with meaning. Amid the confusion of captivity, prisoners actively listened for clues to help interpret their current circumstances and future prospects. Echoes of battle, voices of civilians, and rumors allowed prisoners to listen beyond the guards and slightly empower themselves. Sound also entwined with how prisoners perceived the passage of time, especially on days they infused with acoustic importance such as Sundays and national holidays. Focusing on listening as a methodology is important because it has the flexibility to incorporate competing interpretations. Keepers and captives attached different patterns of meaning to the same sounds and this is useful to consider in examining lived-experience. Preserving acoustic harmony was part of prison guards’ larger task of maintaining order. Conversely, effective use of noise offered prisoners a unique method of resistance because it utilized one of the few advantages of a weak but numerically superior population. The aural world was a channel of practical knowledge and captivity experience, a point of disagreement between prisoners and guards, and a method of resistance.