Bridging Science, Art, and the History of Visualization:A Dialogue between Scott Curtis and Robert Lue
Scientific visualization has taken many forms through the centuries, but since the digital age, and as computer-generated imagery (CGI) becomes relatively easier to access, produce, and distribute, animation has been an especially popular choice for scientists eager to communicate theories and findings to their colleagues, students, and the public. Animation has always been a viable choice for scientific instruction and popularization—recall, for example, Einstein’s Theory of Relativity (1923) and Evolution (1923), both by Fleischer Studios—but the relationship between animation and science has also been an anxious one. In addition to potentially infecting scientific endeavor with the taint of popular culture, animation threatened to mislead the untrained eye by giving an abstracted yet simplified understanding of the processes depicted. Indeed, the latter concern has also plagued the use of drawn figures in twentieth-century scientific illustration, as we shall see. So, accuracy and sobriety have always been high priorities for scientists involved in scientific animation. As CGI increasingly adapts the conventions of animation to scientific illustration, the histories and tensions between them entwine in interesting ways. Harvard’s BioVisions films provide excellent examples through which to explore these [End Page 193] histories and tensions, so cell biologist Robert Lue and film historian Scott Curtis set out to discuss the implications of these films for scientific visualization, past and present.
The BioVisions initiative at Harvard University was founded by Robert Lue and Alain Viel, who are both faculty members in the Department of Molecular and Cellular Biology. It is an educational program that seeks to bring different visualization strategies to the teaching and broader communication of biological processes and concepts. Research in biology often depends on the development of new ways of visualizing important processes and molecules. Indeed, the very act of observing and recording data lies at the foundation of all the natural sciences. The same holds true for the teaching and communication of scientific ideas; to see is to begin to understand. The continuing quest for new and more powerful ways to communicate ideas in biology is the focus of BioVisions.
The potential of multimedia in the area of biology education has yet to be fulfilled. Indeed, multimedia as a means of imparting biological information is years behind its use in other areas such as entertainment and interactive games. BioVisions is meant to close this gap by combining the highest-quality multimedia development with rigorous scientific models of how biological processes occur. In addition, this new generation of science visualizations is not meant to simply be simulations or mirrors held up to reality; rather, the visualizations are designed with a specific pedagogical goal in mind. This means that each decision made on how to represent a given biological process also includes consideration of how best to visually communicate particular aspects of that process. In this regard, these practices integrate visual pedagogy with the narrative and emotive elements of filmmaking.
In 2007 the first BioVisions visualization, The Inner Life of the Cell, was released and promptly went viral on YouTube. It is an eight-minute animation that explores the structural organization of cells as well as protein translation and secretion. The Inner Life of the Cell series remains one of the most viewed science animations in history. For example, on average the first animation is viewed over 200,000 times per month on YouTube. Based on a survey of 1,000 randomly selected public high schools across America, 83 percent regularly use the animation in a biology course.
The Inner Life of the Cell has received numerous national and international accolades, including a Telly award, honoring video and film production. As a further indication of the crossover between science visualization and art, the animation has been [End Page 194] shown at a number of museums across the world, including the Museum of Modern Art in New York, the Marian Koshland Science Museum of the National Academy of Science in Washington, D.C., and the Palais de la Decouverte in Paris, France. It has even been described as bringing artistic notions of landscape representation to hitherto unseen environments.
While the initial creation of BioVisions began with the primary goal of educating and motivating science students in college, the remarkable response from students of all ages as well as the general public reveals a widespread need for compelling visualizations of biology. It is also clear that the power of the visualizations has as much to do with their artistry as it does with the scientific veracity and pedagogy surrounding the content. This critical integration of art and science, together with pedagogy, has important historical precedents and should assume a more prominent position in the ongoing dialogue on visual communication. This conversation between Scott Curtis and Robert Lue, which began at a conference meeting, is but one reflection of this rich dialogue. [End Page 195]
Where did the idea for the BioVisions series originate, and what are its educational goals?
The true starting point for BioVisions was the convergence of a personal interest in the visual arts with the critical role played by observation in the scientific method of cell biology. Our understanding of the cell really began with our ability to see it in the microscope, and today major breakthroughs in cell biology most often involve some kind of data visualization. This led me to focus on how one might harness the emotive and motivational power of multimedia that one would typically associate with the arts and combine it with the educational or research outcomes linked to the visual models used in science. When I first proposed the creation of a program to address this possibility more than fifteen years ago, the reception from my colleagues in science was largely chilly. One notable exception was Alain Viel, a fellow cell biologist who joined me in recognizing the enormous potential that breakthroughs in animation, as evidenced by films such as Jurassic Park (Spielberg, 1993), had for the visualization of otherwise unseen molecular processes in cells.
Alain and I quickly discovered that getting one’s hands on the software and talent that produced the moving dinosaurs in Jurassic Park was both very expensive and nigh impossible. It was not until animation software became more widely available outside of Hollywood that we began to gather animation and scientific talent to create our first animations. After experimenting with programs such as Lightwave and Maya to create my first simple animations, I realized in the process that the actual programming was just the beginning and that the conceptualization, direction, and visualization based on otherwise seemingly abstract scientific data were far more important. In addition, the issue of defining the primary message and pedagogical arc of each visualization may be the most important of all.
This led to the development of an alternative model that more closely resembles that of film production. I refocused my efforts on interpreting the science, building the visual model, and directing the action and visual style of each piece. If we use the feature film analogy, it would be the closest to a combination of director, writer, and art director.
BioVisions’ educational goals are both pedagogical and affective. From a pedagogical perspective, the visualizations are meant to support the learning of fundamental facts and concepts while simultaneously placing the students in the physical and temporal contexts of the cell. This is meant to support a deeper intuition [End Page 196] for how cellular processes work, how they are interdependent, and how they change dynamically over time. Developing a deep intuition for cellular processes should enable more rigorous interpretation of scientific data nuanced by an understanding of the cellular context and its contingencies. I also expect that this will drive more creative experimental design and hypothesis building.
The affective goals of BioVisions are to trigger wonder and emotional engagement with aspects of the natural world that are beyond the usual limits of what we can directly see. In the same way that Jacques Cousteau’s undersea films brought the wonder of marine life to millions, including my childhood self, my hope is that BioVisions animations will reveal the wonder and complexity of cellular life. To this end, my direction, scripting, and storyboarding of each animation are meant to speed the pulse and trigger that sensation of excitement that drives the will to learn more while assisting with retention and comprehension.
This brings up multiple issues we could explore throughout this conversation, including the role of Hollywood feature filmmaking conventions in scientific visualization and educational film; the importance of emotion, wonder, and spectacle in educational film; the attitude toward animation in scientific circles; the difference between educational and scientific or professional visualization; the privileged place of imaging in cell biology; the relationship between particular software packages and the representations they produce—the “look” of one software versus another—or the relationship between a visualization technology and the theories of cell structure, function, or behavior that they presume; the role of spatial orientation or tactile learning (as we find in Linus Pauling’s molecular models, for example) in education or scientific method; and animation’s place within a larger educational or scientific context that includes illustrations, figures, models, lectures, etc.
But let’s start by exploring the “chilly” attitude of scientists toward animation. I’ve found that this is not unusual in the history of scientific animation. There has often been wariness about the association of animation with entertainment, cartoons, and children (or the childish). Yet scientists used it just as often as photographic (still or moving) images. Even today, when there is an explosion of the educational use of animation in biology classrooms, I wonder if the resistance is still felt. Or is it the case that animation is fine, even applauded, for educational applications but approached very cautiously or resisted for professional contexts? That is, to what extent does animation figure in the “major breakthroughs in cell biology” that involve visualization? Has the [End Page 197] attitude toward animation changed since you first expressed interest? If there is still resistance, is the source of that reluctance in the associations of animation with the childish or hesitation about the limits of what and how animation represents or something else (or a combination)?
In the past many biologists cherished visual data as an indispensable part of scientific discovery supported by experimentation. But it was just that—data—something that was to be interpreted and integrated into ever-changing models for how biological processes work. In this regard, it was not about representation so much as an experimental outcome or reflection of natural phenomena. This mind-set in which visual information is primarily data led to the concern that visualizations that seek to represent biological phenomena might be misleading, since they might be confused with data or reality. However, as the need to communicate scientific concepts and complex models becomes ever more important, increasing numbers of biologists have realized that visual information is also about representation and communication. In addition, concerns about the degree to which abstractions of reality can lead to misconceptions have somewhat abated as virtual models become increasingly prevalent in many research fields as well as popular entertainment. This has driven an inevitable expansion of what we mean by representation, coupled with a growing sense of how different representations can be both deeply informative as well as motivational.
Today, when you attend a scientific meeting it is not uncommon to see sophisticated visualizations used to represent current models for how a particular process might work. Even particular scientific methods are subject to an increasingly wide range of representations designed to help novice audiences understand the critical features of an experimental approach. This is a further reflection of the extent to which animation and video are expected elements of biology textbooks such that the students of science are also growing up with an expectation for more sophisticated representations that go beyond color diagrams and data figures.
Yes, this description of scientific attitudes toward animation and more popular forms of representation corresponds with narratives about scientific illustration in the history and sociology of science and even parallels the approach to visual representations within those disciplines. Within both the scientific and historical communities there has been a notable shift in emphasis from taking the representation as somewhat “invisible” in relation to the [End Page 198] data to recognizing the value of visual aids as both a heuristic and historical tool. Yet traditionally there has been a certain ambivalence toward the power of the image; sociologists Gilbert and Mulkay interviewed many scientists about their illustrations and framed their wariness as a dilemma along lines similar to yours: “realistic conventions must be used in a way which is not inconsistent with the fictional aspects of the pictures and which does not mislead non-specialists into taking the visual product too literally.”1 That is, all scientists recognize the provisional quality of their representations—after all, like you say, they are merely illustrations of data and theories, which can and do change—but in making a representation, such as a diagram, they must balance the provisional, theoretical, and abstract qualities of the image (which depict what they don’t know, but would like to suggest) with realist techniques designed to persuade the reader that this representation is also tied to empirical data. So this is the dilemma: how to depict the unknown as unknown while also persuading the reader that this version of the unknown is better than the other.
To illustrate that dilemma further, recall Paul Ehrlich’s images of antibody formation from around 1900 (figure 2). Cambrosio and his colleagues have argued persuasively that the innovation in cell biology that these drawings represented was considerable, but the controversy around them was not because of the theory they depicted; instead, other theorists objected to how the images depicted them.2 Look closely at the “lock-and-key” mechanism of the cell receptors in this image. That way of understanding cell receptors is completely analogical, symbolic, or heuristic; that is not what we would see under a microscope, if it were even possible. [End Page 199] And Ehrlich would have agreed. But combined with the realistic techniques of shading, three-dimensionality, perspective, and even the serial images, which suggest a process taking place in a graspable time, these techniques imply, as one biologist at the time objected, a “material stability” to a process that is unstable, unknown, and perhaps unrepresentable. In fact, when visitors to Ehrlich’s laboratory in Frankfurt asked to see these cell receptors under a microscope (they really did!), they unwittingly highlighted the significance of this dilemma in scientific representation.
So it’s very intriguing to hear from you that this dilemma has “somewhat abated as virtual models become increasingly prevalent in many research fields as well as popular entertainment.” Is this due to the increasing use of animation in the laboratory, the classroom, and popular visual culture? Does it suggest that the provisional, theoretical aspect of visual representations in science—once a domain of the expert—is perhaps now better understood at the lay level, too, as animation becomes more widespread in those arenas? That is, do animation and modeling themselves prepare students and laypeople for overtly theoretical representations? It might be the case that the utopian, fantastical character of animation actually implies some aspect of the unknown and its obviously theoretical character. Animation as a prototheoretical tool!
Yet even abated, the dilemma still exists, right? How do you negotiate it in your animations? Do you and your team make a distinction in the animations between the overtly theoretical and the straight representation of empirical data? Or are your animations designed to accomplish something else besides the traditional negotiation of this dilemma in scientific visualization?
It is true that the issue still exists of how to balance theory with empirical representation, but I view it as less a dilemma and more of an opportunity to communicate in ways that build intuition. In other words, as our understanding of the natural world and living organisms reaches down to the microscopic and chemical level, the bridge between empirical data and “accurate” visual representation becomes ever more tenuous and ultimately more abstract. How do we accurately represent something like a hydrogen bond? It operates below the wavelength of visible light and as such cannot be seen. It is also a physical and chemical phenomenon that has dramatically perceptible outcomes, such as the behavior of water, but in and of itself how can it be visually represented in an empirical way? In the increasing absence of accurate ways to visually represent structure and phenomena, the emphasis has gradually shifted to understanding and from simple picturing. In order to [End Page 200] drive deep understanding, one must therefore use existing visual frameworks that function as analogies and metaphors for reality. I focus on this approach in my visualizations, in part because it opens up new ways to achieve multifaceted understanding while also enabling a level of synthesis that not only improves the comprehension of individual ideas but also brings them together in ways that communicates broader concepts. This is the opportunity that animation and visualization presents today.
In the case of hydrogen bonds, each individual bond is a combination of electrostatic forces between atoms of particular [End Page 201] characteristics that in turn orient the participating atoms in space relative to each other. Each atom is typically represented as a visual abstraction, as are the forces of electrostatic attraction and repulsion. Textbooks and scientific articles portray them using a particular visual language based on spheres and dotted lines meant to both locate each atom and establish the spatial relationships determined by the hydrogen bonds (figure 3). But are we using this visual language effectively, and can we communicate more about hydrogen bonds and their critical roles by going to three dimensions and placing them in a higher-order visual context? Current animation allows us to go up a level, from two to three dimensions, and portray hydrogen bonds in the context of an enzyme-active site, thereby showing the role they play in coordinating the three-dimensional arrangement of a substrate based on interactions with the enzyme. The typical visual shorthand of the dotted line becomes even more informative at this level by expressing the three-dimensional geometry of the web of bonds such that the inaccuracy of the individual representation is superseded by the expression of the higher-order “image” (figure 4). In this regard, the ability to render in three dimensions and to move through constructed environments breathes new life into two-dimensional visual tropes that otherwise would seem ever more misleading. This layering of meaning as we move from two to three axes in space is what also moves us closer to the multilayered effects we associate with film. This begs the question—are we entering an era of science animation that brings the visualization of the molecular world closer to the multilayered representations we associate with science films that explored the visible or macroscopic world? Are we moving away from the relatively bloodless expressions of molecular biology to a new kind of expression that finally connects these phenomena with the excitement and visceral qualities of a nature film?
You point to an issue in scientific visualization concerning the role of conventions in representing both visible and invisible phenomena. Science educators often express concern about visual literacy, which comes down to whether viewers understand the visual conventions used by experts.3 That is, to “use existing visual frameworks that function as analogies and metaphors for reality” is already to assume that the viewer understands the visual conventions that are being used metaphorically or that she can see the analogical relationships between the visual element and the invisible phenomenon. The conventions for rendering dimensionality, for example, are quite common and recognizable. In both of your examples we see the use of perspective to give the impression of the [End Page 202] objects existing in space, while the use of shading implies that light is coming from a consistent direction (from our left in figure 3, from above in figure 4), which gives the impression of weight and dimensionality. Here the conventions draw upon our own experience with objects in the world, so we recognize the analogical relationship between the representation and the phenomenon of dimensionality. There are a number of implications to draw from this. First, both the textbook illustration and the animation use the same conventions to render dimensionality, so it is not necessarily 2-D versus 3-D that is at issue here. The textbook illustration (see figure 3) is also a “3-D model” of a molecule, so going from the two-dimensionality of the textbook page to the three-dimensionality of the animation (which is also not really three-dimensional) is not exactly what distinguishes these examples. Instead, the difference, based on your explanation, is that the textbook figure illustrates the structure of a molecule, while the animation illustrates the function of the molecule in a “real-life” situation. The inaccuracy of the first example is not that it doesn’t show the dimensionality of the molecule (it does) but that the dotted lines depict only a limited understanding of the way hydrostatic bonds work. The dotted lines indicate not only attraction between elements (a convention of chemical illustration) but also—in the animation—how that attraction functions in order to accomplish something. And this understanding is what the animation adds to the lesson, correct?
So computer animation adds to our visual toolbox not the ability to render in three dimensions (we had that before) but the ability to “move through constructed environments,” as you say. That is, computer animation adds the element of temporality (the fourth dimension?) and experience. By showing how hydrostatic bonds function in time, we get a better, more accurate understanding of the way they work. But time and space cannot be easily separated in this kind of representation, so we also see how these bonds function in relation to each other in space, and this gives us an experiential dimension to our understanding. We “move through” the space in time, and this “moves” us in that we can understand the relationship experientially, just as we understand how we move through space in the real, nonrepresentational world of our daily lives. And this is indeed a powerful, memorable, even effective form of learning, as educational filmmakers have argued since the early twentieth century.
Which brings us to the second, perhaps more pressing implication: are we trading one kind of inaccuracy for another? To what extent is our experience of the macroworld transferable to the microworld of molecules? Our conventions for rendering [End Page 203] dimensionality, weight, and movement through space give us the impression that we understand the structure and function of these molecules, but is this a “deceptive clarity,” as some science educators call it?4 This is a variation on the dilemma I alluded to earlier: the combination of realistic conventions and theoretical unknowns can put us in the position of giving material stability to phenomenon that are not material or stable. How then do we render the microworld? I guess that is the ultimate, maybe unsolvable question, given that we need metaphors (visual or otherwise) to understand, well, anything. On one hand, I would say that the BioVisions film Mitochondria is a great example of “moving away from the relatively bloodless expressions of molecular biology to a new kind of expression that finally connects these phenomena with the excitement and visceral qualities of a nature film.” On the other hand, to what extent are these actual qualities of the phenomena under study? We can see that the conventions of macro, live-action nature films inform the computer animations of molecular biology. But to what extent are these representational possibilities informing or even driving our theoretical understanding of the phenomena? Are the animations the result of received knowledge, or are they also contributing to it in ways that are as yet unacknowledged?
Ultimately it comes back to what we mean by our “understanding” of biological phenomena. On the one hand, we can [End Page 204] understand the processes shown in the mitochondria animation in quantitative terms based on physics and chemistry. This provides mathematical insight into the microscopic reality of how energy is harnessed in the cell. On the other hand, our understanding of mitochondria also requires a synthesis of the physical and chemical realities at the microscopic level with organizational and functional hierarchies that move us into the macroscopic realm. Like a Russian matryoshka doll in which figures are nested one inside the next, our understanding of any biological process requires us to unpack a hierarchy that goes from the atomic to what we can see with the naked eye.
This integration of our understanding across different scales and kinds of organization is one aspect of so-called systems biology, and the visualizations that we have been discussing are fast becoming important tools for interrogating the validity of increasingly complex models in biology. Returning to the example of the mitochondria animation—not only does it reveal the chemistry of electron transfer through protein clusters, it models the density and arrangement of these clusters in space and how this could facilitate the process. In this regard, these multilayered visualizations are not simple representations of received knowledge but rather become generative of new insights.
We know that the ink drawings of cells based on what was observed through early microscopes lead to breakthroughs, such as the cell theory. Today, a different kind of mediated biological image can have an essential role in addressing critical gaps in current biological research such as the failure to effectively bridge multiple levels of analysis. The biological image is therefore as relevant, moving and generative as ever!
Indeed! This is a good explanation of the power of moving images—or scientific visualization in general—for theory building. In this regard, we could discuss the function of computer animations, such as The Inner Life of the Cell, as scientific models, which are absolutely vital for knowledge production, especially of the microscopic and submicroscopic world. But we have run out of time, so we will need to leave that discussion until our next encounter!
Scott Curtis is associate professor in the Department of Radio/Television/Film at Northwestern University, director of the Communication Program at Northwestern University in Qatar, and president of Domitor, the international society for the study of early cinema. He is the author of The Shape of Spectatorship: Art, Science, and Early Cinema in Germany (Columbia University Press, 2015) and has published extensively on the history of animation and on scientific and medical uses of motion picture technology.
Robert Lue is a professor of molecular and cellular biology at Harvard University and the Richard L. Menschel Faculty Director of the Derek Bok Center for Teaching and Learning, where he is responsible for fostering innovative teaching in Harvard’s Faculty of Arts and Sciences. Lue earned his PhD in biology from Harvard and since 1988 has taught undergraduate courses acclaimed for their innovative and interdisciplinary approach. In 2012, Lue’s extensive work on using technology to enhance learning took a new direction when he became faculty director of HarvardX, Harvard’s university-wide online education initiative that includes the edX partnership with MIT. He now helps to shape Harvard’s engagement in online learning to reinforce its commitment to teaching excellence and expand its reach and impact globally. Lue has coauthored several undergraduate biology textbooks and has developed award-winning multimedia on topics including Understanding HIV and AIDS (1999), Biochemistry: Interactive Learning (2000), The Inner Life of the Cell (2006), and Powering the Cell: Mitochondria (2010). His media publications have been praised for their scientific accuracy, educational utility, and vibrant 3-D portrayals of the cellular world and have received innovation awards from organizations as varied as Apple Computer, the Nature Publishing Group, and Seed magazine.
1. G. Nigel Gilbert and Michael Mulkay, Opening Pandora’s Box: A Sociological Analysis of Scientists’ Discourse (Cambridge: Cambridge University Press, 1984), 160. [End Page 205]
2. Alberto Cambrosio, Daniel Jacobi, and Peter Keating, “Ehrlich’s ‘Beautiful Pictures’ and the Controversial Beginnings of Immunological Imagery,” Isis 84, no. 4 (December 1993): 662–99.
3. See, for example, Konrad J. Schönborn and Trevor R. Anderson, “The Importance of Visual Literacy in the Education of Biochemists,” Biochemistry and Molecular Biology Education 34, no. 2 (March 2006): 94–102; Maura C. Flannery, “Thinking in Pictures,” American Biology Teacher 68, no. 5 (May 2006): 299–303.
4. Roy Tasker and Rebecca Dalton, “Research into Practice: Visualisation of the Molecular World using Animations,” Chemistry Education Research and Practice 7, no. 2 (2006): 141–59; Marcia C. Linn et al., “Can Desirable Difficulties Overcome Deceptive Clarity in Scientific Visualizations?,” in Successful Remembering and Successful Forgetting: A Festchrift in Honor of Robert A. Bjork, edited by Aaron S. Benjamin, 239–62 (New York: Psychology Press, 2011). [End Page 206]