Handling Digital Brains: A Laboratory Study of Multimodal Semiotic Interaction in the Age of Computers

5. The Semiotic Mind in the fMRI Laboratory

5 The Semiotic Mind in the fMRI Laboratory Reconceptualizing fMRI brain visuals as fields for interaction has consequences for the understanding of thinking in the laboratory. The fMRI researchers’ skill to see certain concrete and spatially represented biological phenomena is not confined to an individual brain but is a gradual achievement that involves a fine coordination between digital screens, researchers, and their hands, and an array of graphical inscriptions. Far from being only an intellectual operation, the newcomer learns to see by looking at brain visuals but also by touching the digital screen and coordinating it with charts, maps, sketches, and other paper-based visuospatial inscriptions . The details of these actions indicate that this is true not only for newcomers but also for advanced practitioners: The involvement of their semiotic and working hands with computer screens and graphical inscriptions is an important aspect of how experienced practitioners think in the laboratory. Even though the practitioners that I observed use input devices such as a mouse and keyboard to direct changes displayed on computer screens, they frequently place their hands directly on the screens. In fact, computers in fMRI laboratories regularly show the prints of fingers and hands that have touched them. In this chapter, I take advantage of these apparently redundant actions to discuss their roles in meaning-making. When practitioners actually touch the surface of the screen as part of their laboratory work, they often use their hands to point (e.g., Kita, 2003) and to coordinate the screen with other graphical inscriptions. These handlings organize the visual field by momentarily inscribing forms and enacting new relationships among its elements. In this sense, they are features of the semiotic mind. 94 Chapter 5 Studying the Organization of Visual Brain Areas Here we join an apprenticeship between a postdoctoral student, the oldtimer in the laboratory, Octavia (O), and the newcomer, Nick (N), a firstyear graduate student in neuroscience spending a semester in the laboratory, and already encountered in chapter 3. Nick is a driven, fast learner who is not shy about asking clarification questions. Octavia is the laboratory’s former Ph.D. student, highly regarded by the laboratory’s director and her other colleagues. As someone who has already published important results in the field, Octavia is busy, while cheerful and kind. The atmosphere during the 4 hours of apprenticeship session is friendly but fast-paced and intellectually demanding. While the practitioners focus their efforts to understand how an external scene affects its rendering in the human brain, I want to think about seeing as an embodied process of understanding that is relative to a positioning of the perceiver in the environment of practice. This chapter follows moments of laboratory work where the analyzed data pertain to specific experimental questions. Whereas the initial stages in data analysis (such as the identification of motion artifacts, described in the previous chapter) are common procedure across fMRI laboratories, later moments in data analysis bring to light differences in the work of each laboratory.1 Thus, to master data analysis procedures, it is indispensable for practitioners to go through an apprenticeship, embedding themselves in the procedures of the relevant social group. The laboratory described in this chapter studies visual areas by applying the method of retinotopic mapping. As the general topography and location of the early visual areas relative to one another are believed to be, by and large, consistent across individuals, the researchers use retinotopic mapping to identify the configuration of the visual areas on the fMRI rendering of the subject’s cortex. Once the organization of the early visual areas is identified, that information is used to analyze data from the main experiment. If during the main experiment the researchers investigate how the early visual areas respond to the specific experimental task, they first need to map out the location and borders of these areas.2 As already mentioned in chapter 2 (when discussing the published fMRI brain figure), the idea behind the concept of retinotopy is that there is an orderly mapping between locations in each retinotopically organized brain The Semiotic Mind in the fMRI Laboratory 95 area and the locations in the visual field. In other words, the retinotopically organized visual areas are considered to be point-to-point copies of the topography of the retina, where what is of interest are the topographic correspondences generated through a translation of information...