The MIT Press
abstract

This paper describes the Rhythmotron: a percussion-centered robotic orchestrion, commissioned for the CoLABS festival in Sydney in 2017. The authors describe how they electronically reimagined the mechanical components of a cylinder piano by using a variant of the XronoMorph software, and they consider the synergy between algorithmically generated rhythms in a digital environment alongside its analogous mechanical counterpart. The authors detail the idiosyncratic behaviors of linear actuators when used to power drumming robots, and they discuss the aesthetic implications of the Rhythmotron.

mechanical musical instruments

The two industrial revolutions (1733–1913) brought the invention of a number of novel musical instruments that used new technology [1]. During that time, mechanical instruments were a source of wonder and amusement to the public [2], offered inspiration to inventors [3] and left social commentators pondering possibilities of new technology [4]. Automatic player pianos such as the pianola (a pneumatic, foot-operated organ) and the cylinder piano (which used pneumatics or cylinders with pins designed to strike or blow different instruments) were popular [5]. In Europe, orchestrions were popular and often incorporated drums and cymbals.

In the 1920s, modern technology such as the electric microphone, radio and, later, tape recording caused a decline in the popularity of mechanical musical instruments, and before the end of the Great Depression in 1939, pianola production in the United States had ceased [6]. In the 1940s, Conlon Nancarrow (1912–1997) produced a number of contemporary works for the player piano (including his lesser-known works for mechanical percussion). These compositions embraced rhythmic complexity facilitated by the existing design of the player piano [7]. In 1950, Australian composer Percy Grainger combined mechanical musical instruments and modern technology to compose "Free Music Experiments" [8], collaborating with Burnett Cross on a piece for three Solovoxes (monophonic synthesizers) played by pianola roll [9].

Fig. 1. The Rhythmotron at CoLABS, Bungarribee Park, Sydney, Australia, 29 September–8 October 2017. (Photo © John Taylor and Andrew Milne) See the online supplementary files at for a video of the Rhythmotron in action. John R. Taylor (researcher), MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Locked Bag 1797, Penrith NSW 2751, Australia. Email: . ORCID: 0000-0002-4435-0657.
Click for larger view
View full resolution
Fig. 1.

The Rhythmotron at CoLABS, Bungarribee Park, Sydney, Australia, 29 September–8 October 2017. (Photo © John Taylor and Andrew Milne) See the online supplementary files at www.dynamictonality.com/rhythmotron_files for a video of the Rhythmotron in action. John R. Taylor (researcher), MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Locked Bag 1797, Penrith NSW 2751, Australia. Email: j.taylor@westernsydney.edu.au. ORCID: 0000-0002-4435-0657.

Modern digital technology again innovated mechanical musical instruments. Seattle-based artist Trimpin, a pioneer of robotic and mechanical musical instruments [10], digitized Nancarrow's hand-punched rolls into MIDI format in Studies for Player Piano [11,12], while contemporary examples include mechanical instruments such as those by Denki [13] and Tinguely [14] and robotic instruments such as those by Kapur [15] and Raes [16] among others [17]. This paper describes the conceptual, developmental and aesthetic implementation of a novel percussion instrument: a computational-mechanical hybrid instrument analogous to the orchestrion and cylinder piano. Here, we introduce the Rhythmotron—a softwarebased percussion instrument driven by microcontrollers and housed in an upright piano (Fig. 1). [End Page 67]

We generated the Rhythmotron's rhythms using a custom version of Andrew Milne's XronoMorph software (the X is pronounced as "K") [18] and linear actuator-driven beaters that struck membranophones and idiophones. Various rhythmic parameters could be modified with pushbuttons and potentiometers. The following sections describe the Rhythmotron's design as well as challenges we encountered and their solutions.

conceptualizing the rhythmotron

We were commissioned to display an innovative art installation in Bungarribee Park in Western Sydney Parklands for the 10-day CoLABS festival. The broad focus of the commissioning not-for-profit organization, Art Futures Ltd, was to fuse art with science, technology, engineering and mathematics (STEM) [19]. Given our prior interests, including rhythm, percussion, mathematics, interactive media and music performance, we decided to create a novel, interactive percussion-based instrument.

musical and aesthetic considerations

For the Rhythmotron, we had three dependent artistic goals: first, to encourage interaction and exploration of rhythm; second, to provide appealing and compelling musical rhythms; and last, to encompass a wide range of sounds and timbres that reflected the cultural diversity of the festival's locale.

Instead of routine manipulations of precomposed MIDI loops (e.g. tempo, swing ratio and instrumentation changes), we envisaged more dramatic and fluid interactions that would still produce musically appealing rhythms. We achieved this with the XronoMorph software, which generates and parameterizes two families of rhythms: well-formed rhythms and perfectly balanced rhythms. Examples of the former are common in Balkan aksak music [20], West African timelines (e.g. those used by the Yoruba and Ewe peoples) [21], Afro-Cuban traditions (e.g. the tresillo and cinquillo rhythms) [22] and contemporary rock (e.g. Radiohead [23]); examples of the latter are conventional meters in Western music, the cross-rhythms found in jazz and minimalist classical works, and complex out-of-phase cross-rhythms such as those of the Aka people [24] (although XronoMorph and the Rhythmotron are not intended to replicate specific real-world rhythms). Well-formedness is an organizational concept originally defined for musical scales [25] that can be naturally applied to rhythms as well [26]: In a well-formed rhythm, each interonset interval (IOI) comes in one of two sizes—"long" or "short"—and these are arranged as evenly as possible. Perfect balance is a recently introduced organizational principle for scales or rhythms [27,28]: If each onset in a rhythm is represented by a "weight" on a circle, that rhythm is perfectly balanced if its center of gravity is at the circle's center. Milne details these rhythmic principles, and their implications for complex polyrhythms, elsewhere [29]. Crucially, XronoMorph allows manipulation of continuous and discrete parameters in both rhythmic families, enabling users to explore diverse and musically appealing rhythms.

The Rhythmotron's resulting rhythms could be simple orunusual (with complex or no time signatures), but all are polyrhythmic: They interweave multiple individual rhythms to make a whole rhythm—the polyrhythm—that is greater than its parts. Thus, sparse independent rhythms can combine to produce a dense polyrhythm (interweaving poly-rhythms such as these are detailed in Arom [30]).

We chose seven percussion instruments that produced diverse timbres and which—in combination—were not reminiscent of any single musical culture: baya tabla, tamborim, frame drum, wood block, snare, hi-hat and mini ride cymbal. The frame drum provided a bass-rich timbre; the tabla and tamborim, midrange timbres; the remaining instruments, treble-rich timbres. Video and audio examples are available from the online supplementary files at www.dynamictonality.com/rhythmotron_files.

xronomorph: transforming and virtualizing the piano cylinder

In a cylinder piano, the cylinder typically has small raised steel pins, similar to those of a music box (see Ord-Hume [31] for examples and descriptions). During rotation of the cylinder (which is usually at a constant speed), the pins strike or cause a mechanical object to pivot and hit a sound-producing object. These raised pins are staggered across the length and diameter of the cylinder, thus producing sequences of mechanical action. One rotation of the cylinder is equivalent to a specified musical time and is dependent on the cylinder's rotational speed. An end-on view of a cylinder with these raised pins would show pins occupying numerous points of the circumference. This end-on view is closely associated with the XronoMorph software, as now explained.

The piano cylinder is a sufficient mechanism for producing music comprising a finite number of timbres and/or pitches periodically arranged in time. The cylinder's circular dimension (its circumference) determines timing (assuming the cylinder rotates at a constant speed), while the circular location (angle) of each peg precisely determines when it will make a sound. The cylinder's linear dimension (its length) determines which timbre or pitch each peg produces. The circular dimension is embedded in two-dimensional space, which implies that the combined structure must be embedded in three-dimensional space (as is obvious from looking at any cylinder).

For a two-dimensional screen-based user interface to avoid conflating pitch/timbre and time, the linear dimension of pitch/timbre must be spatially collapsed to form a circle; hence, pitch/timbre must be identified by some other means. As shown in Fig. 2, XronoMorph achieves this by visualizing each distinct rhythm within a polyrhythm by a separate polygon inscribed in a circle. The small red disk rotates clockwise around the circle; when it touches a vertex of a polygon, a sound plays, and the polygon lights up. Each polygon plays a different pitch/timbre, but every vertex of a given polygon plays the same pitch/timbre. In this way, XronoMorph functions as a two-dimensional software analogue of the piano cylinder, except that our virtual piano rotates around the stationary virtual cylinder. Using this virtual [End Page 68] analogue for algorithmic rhythm generation allowed us to bypass the traditional piano key interface in favor of novel conceptual controls.

the rhythmotron's preparation, materials and configuration

We obtained an unwanted piano, in good cosmetic condition, via an online advertisement. (We hired a piano removalist in the local Sydney area who, on delivery, found an old business card fastened to the inside of the piano. The card was from his own grandfather who had moved the piano 40 years earlier!) We prepared the piano (not in the musical sense) by removing the keyboard, the hammer rail and hammers, and a number of broken or heavily corroded strings. We then cleaned all outside surfaces, sealed all edges and applied several coats of waterproof marine varnish (the average chance of rainfall in October in Sydney being approximately 35.5%). We reserved the front section of the piano, above the keybed, for the drums and the robotic drumsticks. We positioned the seven percussion instruments evenly in the available space while ensuring each drumstick mechanism could be firmly clamped to the keybed or soundboard of the piano. We placed addressable LED lights around the inside of the front casing. Synchronized to flash with the frame drum, the lights added a complementary visual effect. An Apple Mac mini ran the XronoMorph soft ware, which sent control messages through Max [32], via serial bus, to Arduino microcontrollers [33]. With limited space in the piano's front, we positioned the remaining electronics between the soundboard braces, as shown in Fig. 3.

Fig. 2. The XronoMorph interface showing the perfectly balanced Pygmy mò.nzòla rhythm, as transcribed by Arom [], 2016. (© Andrew Milne)
Click for larger view
View full resolution
Fig. 2.

The XronoMorph interface showing the perfectly balanced Pygmy mò.nzòla rhythm, as transcribed by Arom [41], 2016. (© Andrew Milne)

Fig. 3. The rear of the Rhythmotron, 2017. (Photo © John Taylor and Andrew Milne)
Click for larger view
View full resolution
Fig. 3.

The rear of the Rhythmotron, 2017. (Photo © John Taylor and Andrew Milne)

We replaced the piano's back frame with clear acrylic screwed to the soundboard braces. The acrylic protected the electronics from the elements and permitted us to visually inspect the equipment, which, in combination with wireless computer control, meant that we could modify the soft ware to diagnose unforeseen problems. As a 360-degree experience, visitors could watch a video played on a rear-mounted flat screen monitor (Fig. 3, top right) that described the science (S) and technology (T) involved, observe the engineering (E), listen and control the art (A) and visually learn about the mathematics (M) behind XronoMorph's rhythms.

the robotic motor system: design and build

The robotic drums needed to be durable to perform 100,000+ repeated actions. Aware of some of the limitations of robotic drums [34], we opted to use variable electric linear car door lock actuators. These provided push-and-pull force (> 4 kg) to the drumstick butt. A bolt through the shaft of each drumstick acted as fulcrum and secured the stick to its housing. The fulcrum position varied along the shaft of each drumstick depending on the stick-to-drum interaction and the stick's position within the piano. The drum mechanism is illustrated in Fig. 4.

The actuators were Arduino-controlled using motor shields (two actuators per microcontroller) and DC 12V power. Actuators were assigned an ID produced by XronoMorph that, when received by the microcontroller, activated the relevant motor's sequence (pushing the butt of the drumstick upward, thus pivoting and lowering the tip of the drumstick toward the surface of the drum).

The drum mechanism presented a number of challenges. First, we encountered weak velocity, caused by power loss through the system; Raes documented similar issues for his <Rotomoton> [35]. We found 12V enough to ensure a good velocity and therefore a good strike on the drum. Second, because we used membranophones and idiophones, and because of the configuration inside the piano body, we needed optimum stick-to-drum contact, considering the different amounts of stick "bounce."

The actuator "strike time" was governed by the motor's [End Page 69] operation time: Insufficient times resulted in poor or unreliable contact; excessive times resulted in drum overstrikes, which dampens membranophones and causes double-strikes on idiophones. Each case degraded the timbre. The time at the actuator's open position (hold time) needed controlling, because excessive hold times caused stick bounce similar to an overstrike. Furthermore, we needed to specify the actuator retraction time (retract time): Insufficient retract times positively shift ed the subsequent strike start point by the difference between the actual retract time and the retract time required to fully close the actuator, resulting in an overstrike. Given the unique calibration of each motor, the third challenge was to precisely synchronize all the drums. We overcame this by adding up to 30 ms of latency to the motor activation signals of the faster-actioned motors.

Fig. 4. The Rhythmotron's drum mechanism. (© John Taylor and Andrew Milne)
Click for larger view
View full resolution
Fig. 4.

The Rhythmotron's drum mechanism. (© John Taylor and Andrew Milne)

The final challenge was the trigger speed of the actuator, where activation had a minimum 250 ms time period for reset to its starting position to perform another strike (strike time + hold time + retract time), although we can attribute this partly to the voltage used. In the soft ware, we filtered out triggers less than 250 ms apart. This was unproblematic because, with XronoMorph's interweaved polyrhythms, even when no single rhythm has fast repetitions, the resulting polyrhythm can have short durations between successive onsets (see [36]). Furthermore, this repetition speed constraint produced appealing nonlinearities in the Rhythmotron's Tempo control (detailed in the next section).

the rhythmotron's control interface

For users to control various rhythmic aspects of the Rhythmotron, we created a console on the front of the piano to replace the keys (Fig. 5). We needed to remove all the keys to maintain sufficient waterproofing. Consequently, we used an acrylic panel with 10,000-ohm potentiometers, pushbutton switches and toggle switches. Underneath the panel, LED lights added ambience, particularly at dusk. To preserve the actuators' life spans, we idled the Rhythmotron if no control had been adjusted for 30 seconds; the Rhythmotron would then only restart when a button was pressed.

As soft ware designed for musicians, XronoMorph has comprehensible and predictable controls, and the rhythms are clearly visualized with inscribed polygons (Fig. 2). For the Rhythmotron hardware controls, we used ambiguous nomenclature and oft en mapped them to multiple functions in XronoMorph; furthermore, visualizations of the rhythms were not visible from the console. Together, these encouraged a more naive rhythmic exploration.

Fig. 5. The Rhythmotron control interface, 2017. (© John Taylor and Andrew Milne.)
Click for larger view
View full resolution
Fig. 5.

The Rhythmotron control interface, 2017. (© John Taylor and Andrew Milne.)

Four buttons (Simple, Medium, Complex, Weird) cycled through preset seed rhythms where manipulations were introduced by other controls. Simple presets had familiar time signatures (e.g. 4/4, 9/8, 12/8); Weird presets had high prime-numbered time signatures such as 23/8 and 43/8, sphenic numbers (positive integers, products of exactly three distinct prime numbers) such as 30/8 or 42/8, or irrational grooves with no conventional time signature [37]. [End Page 70]

A Mode button toggled between well-formed and perfectly balanced rhythms and affected the other controls' functionalities. In well-formed mode, a hierarchy of successively faster rhythms was generated; some controls changed the numbers of long beats and short beats in the slowest rhythm; others changed the long-to-short beat length ratios (see [38] for a full exposition). Perfectly balanced mode used six independent perfectly balanced rhythms, with controls changing their rhythmic density and their rotations (temporal phase) (again, see Milne, Bulger and Herff [39,40] for a full explanation).

Other controls modified global rhythmic aspects: random event filtering; switching the mappings to the drums; and turning individual drums on/off. The Tempo control knob had notable nonlinear behavior because of the 250 ms strike constraint on each drumstick; as the tempo increased, single-level events might drop out, but the overall polyrhythm still sounded faster.

conclusions and further work

Here, we introduced the Rhythmotron and highlighted some singular aspects of its construction. During the CoLABS festival, the Rhythmotron received significant attention from children and adults alike. We were both available throughout the festival to explain the Rhythmotron's inner workings and observe interactions with it. Some visitors (particularly children) furiously engaged with the controls, enthused by the immediate and synergistic changes made by presses of buttons and turns of knobs; some enjoyed dancing to the various rhythms; while others were more considered, simply observing the audiovisual spectacle of complex rhythms and synchronized LED lights. Most were exhilarated and curious about the technology and the drum patterns; ideally, this stimulated interest in the musical and artistic potential of STEAM. It seems that, even now, mechanical instruments can be a source of wonder and amusement to the public and offer inspiration to inventors and musicians. [End Page 71]

John R. Taylor, researcher
MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Locked Bag 1797, Penrith NSW 2751, Australia. Email: a.milne@westernsydney.edu.au. ORCID: 0000-0002-4688-8004.
Andrew J. Milne, researcher
MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Locked Bag 1797, Penrith NSW 2751, Australia. Email: a.milne@westernsydney.edu.au. ORCID: 0000-0002-4688-8004.
John R. Taylor

john R. taylor is an Australian composer and new media artist and a research fellow in music cognition and computation. His research and creative practice are primarily concerned with the perception and production of percussive timbres. Further information may be found at www.johnrtaylor.net.

Andrew J. Milne

andrew J. milne is a musician, music software designer and senior research fellow in music cognition and computation. In his research capacity, he develops computational models of music perception and uses these models to shed light on human cognition, to drive creative musical outputs and to inform the development of educational tools for mathematics and music. His music software can be downloaded from www.dynamictonality.com.

Acknowledgments

The authors wish to thank Art Futures Pty Ltd, for commissioning the Rhythmotron to appear in CoLABS, Bungarribee Park, Sydney, 29 September–8 October 2017.

References and Notes

1. Encyclopaedia Britannica, "Industrial Revolution" (20 July 1998): www.britannica.com/event/Industrial-Revolution (accessed 23 May 2018).

2. Merlin's Mechanical Museum, Morning and Evening Amusements at Merlin's Mechanical Exhibition, Princes Street, Hanover Square, (London: J.P. Coghlan, 1794).

3. "Mechanical Musical Instrument," Musical Opinion and Music Trade Review 5, No. 55 (April 1882) p. 280: retrieved from https://search-proquest-com.ezproxy.uws.edu.au/docview/7177568?accountid=36155.

4. J.F. Runciman, "Mechanical Musical Instruments," The Saturday Review of Politics, Literature, Science and Art 97, No. 2,520, 199–200 (13 February 1904).

5. C.B. Fowler, "The Museum of Music: A History of Mechanical Instruments," Music Educators Journal 54, No. 2, 45–49 (1967).

6. J. Van Riper, "The Reproducing Piano: A Portrait of the Artist," unpublished PhD thesis (2012).

7. K. Gann, The Music of Conlon Nancarrow (Music in the Twentieth

8. A. Hugill, "Percy Grainger: A Pioneer of Electronic Music," in S. Robinson and K. Dreyfus, eds., Grainger the Modernist (Abingdon, U.K.: Routledge, 2015) pp. 231–253.

9. E. Wrobel and R. Florrimell, Percy Grainger's Paradoxical Quest for "World Music": Free Music and Free Music Machines (Melbourne: University of Melbourne, 1994): www.grainger.unimelb.edu.au/__data/assets/pdf_file/0011/2036666/Free_Music_exhibition.pdf (accessed 23 May 2018).

10. A. Kapur, "A History of Robotic Musical Instruments," 2005 International Computer Music Conference Proceedings (Barcelona, 5–9 September 2005) pp. 21–28.

11. Kapur [10].

12. J. Murphy and Trimpin, "Transcoding Nancarrow at the Dawn of the Age of MIDI: The Preservation and Use of Conlon Nancarrow's Player Piano Studies," Leonardo Music Journal 27 (2017) pp. 32–35.

13. M. Denki, "Nonsense Machines," NTT Intercommunication Center [ICC] (3 November–26 December 2004) www.ntticc.or.jp/en/feature/2004/NONSENCEMACHINES/Works/tsukuba.html (accessed 27 September 2018).

14. J. Tinguely, "Music Machines/Machine Music," Museum Tinguely (19 October 2016–22 January 2017): www.tinguely.ch/en/ausstellungen/ausstellungen/2016/Musikmaschinen-Maschinenmusik.html (accessed 27 September 2018).

15. Kapur [10].

16. G.-W. Raes, <Rotomoton> percussion robot, version 2.0 (2007): www.logosfoundation.org/instrum_gwr/rotomoton.html (accessed 28 September 2018).

17. See also M. Bretan and G. Weinberg, "A Survey of Robotic Musicianship," Communications of the ACM 59, No. 5, 100–109 (2016).

18. A. Milne et al., "XronoMorph: Algorithmic Generation of Perfectly Balanced and Well-Formed Rhythms," Proceedings of the 2016 International Conference on New Interfaces for Musical Expression (NIME '16) (Brisbane, 2016) pp. 388–393.

19. Western Sydney Parklands, "Art & Science will collide at Bungaribee Park in Western Sydney Parklands these October school holidays" (1 August 2017): www.westernsydneyparklands.com.au/about-us/parklands-news/art-and-science-will-collide-at-bungaribee-park-in-western-sydney-parklands-these-october-school-holidays (accessed 21 July 2019).

20. N. Fracile, "The 'Aksak Rhythm,' a Distinctive Feature of the Balkan Folklore," Studia Musicologica Academiae Scientiarum Hungaricae 44, No. 1, 191–204 (2003).

21. G. Toussaint, "Classification and Phylogenetic Analysis of African Century) (Cambridge, U.K.: Cambridge Univ. Press, 2006) p. 45.

22. J. Stewart, "Articulating the African Diaspora through Rhythm: Diatonic Patterns, Nested Looping Structures, and the Music of Steve Coleman," Intermédialités No. 16 (2010) pp. 167–184.

23. B. Osborn, "Kid Algebra: Radiohead's Euclidean and Maximally Even Rhythms," Perspectives of New Music 52, No. 1, 81–105 (2014).

24. S. Arom, African Polyphony and Polyrhythm: Musical Structure and Methodology (Cambridge, U.K.: Cambridge Univ. Press, 1991).

25. N. Carey and D. Clampitt, "Aspects of Well-Formed Scales," Music Theory Spectrum 11, No. 2, 187–206 (1989).

26. A. Milne and R.T. Dean, "Computational Creation and Morphing of Multilevel Rhythms by Control of Evenness," Computer Music Journal 40, No. 1, 35–53 (2016).

27. A. Milne et al., "Perfect Balance: A Novel Principle for the Construction of Musical Scales and Meters," Mathematics and Computation in Music 5th International Conference, Vol. 9110 of Lecture Notes in Artificial Intelligence (Berlin: Springer, 2015) pp. 97–108.

28. A. Milne, D. Bulger and S.A. Herff, "Exploring the Space of Perfectly Balanced Rhythms and Scales," Journal of Mathematics and Music 11, No. 2–3, 101-133 (2017).

29. A. Milne, "XronoMorph: Investigating Paths through Rhythmic Space," in S. Holland et al., eds., New Directions in Music and Human-Computer Interaction (Switzerland: Springer, 2019).

30. Arom [24].

31. A.W.J.G. Ord-Hume, Player-Piano: The History of the Mechanical

32. Cycling '74, "Max Software Tools for Media," www.cycling74.com/products/max (accessed 5 July 2018).

33. Arduino: www.arduino.cc (accessed 5 July 2018). andRew J. Milne is a musician, music software designer and senior research fellow in music cognition and computation. In his research capacity, he develops computational models of music perception and uses these models to shed light on human cognition, to drive creative musical outputs and to inform the development of educational tools for mathematics and music. His music software can be downloaded from www.dynamictonality.com.

34. J. Murphy, A. Kapur and D. Carnegie, "Better Drumming through Calibration: Techniques for Pre-Performance Robotic Percussion Optimization," Proceedings of the 2013 International Conference on New Interfaces for Musical Expression (NIME '12) (University of Michigan, Ann Arbor, 21–23 May 2012).

35. Raes [16].

36. Arom [24].

37. Murphy, Kapur and Carnegie [34].

38. Murphy, Kapur and Carnegie [34].

39. Milne, Bulger and Herff [28].

40. Milne [29].

41. Arom [24].

Share