The “ba, ba, ba” sound universal to babies’ babbling around 7 months captures scientific attention because it provides insights into the mechanisms underlying language acquisition and vestiges of its evolutionary origins. Yet the prevailing mystery is what is the biological basis of babbling, with one hypothesis being that it is a non-linguistic motoric activity driven largely by the baby's emerging control over the mouth and jaw, and another being that it is a linguistic activity reflecting the babies’ early sensitivity to specific phonetic–syllabic patterns. Two groups of hearing babies were studied over time (ages 6, 10, and 12 months), equal in all developmental respects except for the modality of language input (mouth versus hand): three hearing babies acquiring spoken language (group 1: “speech-exposed”) and a rare group of three hearing babies acquiring sign language only, not speech (group 2: “sign-exposed”). Despite this latter group's exposure to sign, the motoric hypothesis would predict similar hand activity to that seen in speech-exposed hearing babies because language acquisition in sign-exposed babies does not involve the mouth. Using innovative quantitative Optotrak 3-D motion-tracking technology, applied here for the first time to study infant language acquisition, we obtained physical measurements similar to a speech spectrogram, but for the hands. Here we discovered that the specific rhythmic frequencies of the hands of the sign-exposed hearing babies differed depending on whether they were producing linguistic activity, which they produced at a low frequency of approximately 1 Hz, versus non-linguistic activity, which they produced at a higher frequency of approximately 2.5 Hz – the identical class of hand activity that the speech-exposed hearing babies produced nearly exclusively. Surprisingly, without benefit of the mouth, hearing sign-exposed babies alone babbled systematically on their hands. We conclude that babbling is fundamentally a linguistic activity and explain why the differentiation between linguistic and non-linguistic hand activity in a single manual modality (one distinct from the human mouth) could only have resulted if all babies are born with a sensitivity to specific rhythmic patterns at the heart of human language and the capacity to use them.