Music Learning through Video Games and Apps: Guitar Hero, Rock Band, Amplitude, Frequency, and Rocksmith, and Bandfuse , and Bit.Trip Complete, and Audiosurf, and Beat Hazard, and Biophilia (review)

In recent decades, interactive video games have opened up new paths for music composition, performance, pedagogy, and appreciation, with a number of American studios, designers, programmers, and producers contributing to the creativity and entrepreneurship that push the subgenres of this field forward. These releases come from the full gamut of the game industry, from larger studios like the Cambridge, Massachusetts–based Harmonix Music Systems to smaller indie developers like Gaijin Games in Santa Cruz. Given their interactive nature, these games offer unparalleled means of immersing their audience and imparting knowledge and skills that the user might not have even set out to learn. The weekly Web series Extra Credits, written by game designer James Portnow and narrated by Pixar animator Daniel Floyd, discusses this untapped pedagogical potential of contemporary video games. In “Tangential Learning” (http://extra-credits.net/episodes), they make a case for the educational value of these games, proposing that players will self-educate when a topic is introduced in a context that they already enjoy. Though music is not directly mentioned in this video, it is not difficult to extrapolate its more general points about pedagogical benefits to musical understanding. Indeed, there are already many video games demonstrating such tendencies; they cover a wide range of genres with the integration of music and gameplay taking various forms.

The category of video games that make use of pre-existing music is best represented by the work of Harmonix. Well known for the first two Guitar Hero games (2005–6) and the later Rock Band (2007) franchise, this studio’s career extends further back with games like Frequency (2001) and Amplitude (2003), which adopted a similar setup: the player navigates an abstracted representation of the music, usually a vertically scrolling path paced to the music along with descending note-heads representing the melody line to be intercepted. Successfully hitting each note translates into more of the song being played back. However, achieving a perfect play session with an unblemished playback requires more than quick reaction time; it demands a degree of timing and muscle memory that would be a familiar challenge for any beginning musician. Many of these games come with plastic controllers to mimic the feel of playing an instrument—at first just the guitar, but later expanding to a full band set of bass, drums, and piano—thus inching players closer to the “real deal” while straining the now-simplistic definition of “toy.” In Playing Along: Digital Games, YouTube, and Virtual Performance (2012), Kiri Miller examines games like Guitar Hero and Rock Band as creative extensions to instrumental learning rather than as the instrument replacements many detractors have framed these mimetic controllers to be. More recently, games like Rocksmith (2011) or the upcoming BandFuse (2013) integrate a real electric guitar, mapping the previous games’ formula onto the input hardware of an actual instrument. With this integration of musical and gaming hardware, there’s a greater possibility of adapting the structure and instant feedback of the games to the process of learning to play an instrument.

One subset of games that use pre-existing music repurposes the user’s own music library to dynamically structure the gameplay. Indie offerings such as Dylan Fitterer’s Audiosurf (2008) and Steve Hunt’s Beat Hazard (2009) tap into the user’s digital libraries, applying an algorithmic analysis of the sound file to structure the flow of action in-game. Audiosurf has a similar visual layout as Guitar Hero, but adapts its stage design to the music’s rising and falling tempo, density, and dynamics. Beat Hazard sets out with similar goals in the format of top-down shooters (as in the classic Asteroid from 1979); here the varying intensity of the music accounts for concentrations of...