Active Users: Project Development and Digital Humanities Pedagogy
Several recent pieces on the digital humanities (DH) and pedagogy have included the observation that, despite the rapid recent growth of DH in many academic fields and institutions, pedagogical issues are consistently overshadowed by issues related to research in DH scholarship. For example, in his piece in the recent collection Debates in the Digital Humanities, Luke Waltzer discusses how “much current work in the digital humanities . . . values research and scholarship far more than teaching, learning, and curriculum development” (Waltzer 338). Stephen Brier writes in his contribution to this collection that “teaching and learning are something of an afterthought for many DHers” (Brier 390–91). In his introduction to the collection Digital Humanities Pedagogy, Brett D. Hirsch makes the case that “research remains the principal vehicle for professional nobility and mobility . . . in the digital humanities” (5–6). Hirsch and Brier prove these points by tracking the occurrences of the words “pedagogy” and “research” (and their synonyms and variants) in recent publications in the field, noting that in popular reference collections, monographs, and journals on the digital humanities, pedagogical concerns are often what Hirsch calls “bracketed,” or relegated “to the status of afterthought” (Hirsch 5). Brier and Waltzer also turn to the NEH’s Office of the Digital Humanities for evidence of this bracketing. They note that “very few” projects funded by the Office’s Digital Humanities Start-Up Grants “have focused specifically on the undergraduate student as humanities doer,” and they show how these projects consistently fail to include words like “teaching,” “learning,” “classroom,” and “pedagogy” in their abstracts (Waltzer 341, Brier 392). This, for Brier, emphasizes the idea that pedagogy and teaching “are not yet primary in terms of digital humanists’ own conceptions of their work” (Brier 392).
This article picks up on this last point by focusing on the Research-oriented Social Environment (RoSE), a project funded by an NEH Digital Humanities Start-Up Grant from 2011–2012 at the University of California, Santa Barbara (RoSE).1 At first glance, the various materials documenting RoSE appear to follow the patterns identified above regarding the bracketing of pedagogical concerns. For example, an article published at an early stage of the project doesn’t use the words “pedagogy” or “teaching” once (Chuck et al.). Additionally, the abstract the RoSE team submitted to the [End Page 211] NEH does not contain the words “pedagogy” or “teaching,” and it only contains one instance of “students.”2
As this material suggests, we did not necessarily see pedagogical concerns as primary when we began the project. RoSE grew out of the Transliteracies Project, an existing research group on online reading practices, and, as such, we focused in the beginning of the grant period on epistemological concerns related to social computing, networked knowledge, and bibliographic data. However, in the following discussion, we reflect on how this focus shifted throughout the year. We report on the involvement of undergraduate students in the development process and on the importance of this involvement for refining our own understanding of RoSE. Thanks to the feedback we received from the students involved in the project, we began to see RoSE by the end of our development year as a tool for discovery and learning—as a system that, in many ways, has pedagogy at its core. In this way, the value of incorporating undergraduate students into the project development process has been quite obvious for us. In reporting on this value, however, we also hope to draw some connections between project development and digital humanities pedagogy. In other words, we hope to suggest how project development itself can be understood as part of a digital pedagogy.
RoSE and the Classroom Use Scenario: Defining an Audience
RoSE is a system for exploring the humanities that encourages users to seek out relationships between authors, works, and commentators—both living and dead— as part of a social network of knowledge. Developed out of a desire to leverage the evolutionary and dynamic qualities of social networks for a bibliographic tool, it presents bibliographical information as an interconnected network of evolving relationships between, for example, an author’s influences, friends, collaborators, and readers. Like a kind of bibliographic Facebook, RoSE allows users to add items to the network, create and visualize their own collections, and produce “storyboards” to visually organize their findings into meaningful arguments or narratives. In addition to any content added directly by users, RoSE also acts as an interface to tens of thousands of records machine-harvested from other databases like Project Gutenberg, YAGO, and SNAC, including information on people, documents, and keywords.
Since RoSE was funded by a start-up grant, the goal of our grant period was never to deliver a final version. Rather, the development team, which consisted of faculty members and graduate students from UCSB’s English Department and Media Arts and Technology Program, was interested in an ongoing and iterative development process that was as much about discovering and shaping the conceptual goals and aims of the system as it was about crafting a functional prototype. We began the project with the [End Page 212] goal of creating a knowledge system that combined the affordances of a bibliographic tool with the ease and comfort of a social network. However, in the course of development, we became increasingly aware that we needed to specify who the primary audience for the system was, but we didn’t necessarily know how to think about this audience. Were they individual humanities researchers? Were they instructors and students? Were they simply interested members of the general public? How we conceived of the users for the system determined how we thought about the system’s features and what users might expect from these features. As we solicited early feedback from colleagues, we determined that the primary goals of the system were not only to create a resource for information access and research, but also to create a stage or vehicle for the exploration and discovery of knowledge—for a kind of pedagogy. It was our pursuit of this goal that led us to the conclusion that one of the primary audience groups for RoSE was undergraduate students in a humanities classroom.
In order to better understand how this audience might use RoSE to facilitate their learning and research and to determine how their feedback might impact development, we designed a classroom-use scenario.3 The scenario was meant to be conducted on a very small scale, so we chose students from an undergraduate class co-taught by Rita Raley and Dana Solomon in Spring 2012 as our study’s participants. The course was entitled “Distracted Reading,” and it explored reading practices with special attention to different media environments, so including RoSE as one of those environments worked well in connection to the theme of the class. RoSE was introduced in the context of a unit on new modes of digital textual analysis. The instructors crafted an assignment that required students to create a collection on the themes of the class and storyboard it in RoSE, using whichever parts of the system’s storyboard feature they deemed useful. The storyboard allows users to visually arrange items they have gathered into collections in meaningful ways. They can group and connect these items together, adding colors, shapes, and annotations in order to tell their story or make their argument (Fig. 1). We demonstrated RoSE and its capabilities in class and worked with the students over two additional class meetings to create their storyboards. Overall, we received feedback from students about RoSE in three different ways. First, as the students worked to create their storyboards, one member of our research team interviewed students about their immediate thoughts on RoSE, recorded their responses, and drew up a narrative report. Second, after creating their storyboards, the students were asked to draft a short reflective statement on RoSE, focusing particularly on the utility and value of the system for humanities pedagogy and scholarship. Finally, we asked volunteers to fill out a questionnaire we had created focusing on both conceptual and general usability issues.4 [End Page 213]
Click for larger view
View full resolution
Student Responses and Implications: RoSE and/as Pedagogy
The use scenario offered undergraduates the opportunity to directly affect the development of a web application. Excited by their ability to affect change on the system’s design and goals, the student participants offered a great deal of feedback during the use scenario and afterwards in their final reflections. They granted the development team a great deal of insight into the usability of the system, pointing to problem areas and adding suggestions for improvement. One student, for example, noted that “[a]s far as improvements, I think the biggest addition would be short biographies for authors and summaries for documents similar to the way Wikipedia is set up” while another emphasized that links to actual sources would be ideal when available (Liu et al. 44). This student also added, “I would like to see the use of images in RoSE greatly increased. In this time of picture [End Page 214] blogs and Facebook, images are a crucial commodity and attraction. If RoSE could capitalize on this, it would be a far more enjoyable experience” (Liu et al. 44). This type of feedback was useful to us in our final months of project development, helping us to focus on immediate areas of concern and to build a wish list for future development.
Many students also moved beyond usability issues to envision future development goals and challenges for the project. For example, one student commented:
With the sheer amount of data that has the potential to be entered into the system, there is an equally large potential for the entry of incorrect or outdated information. . . . The alternative would be that RoSE users simply confirm this data by their own methods, but that feels like an essential element of research is being omitted from what should be a complete experience from RoSE (by which I mean that RoSE should ultimately become a website that you go to at the start of your research, and use all the way through to completion, having to use as few outside resources as possible).
Another suggested the “biggest limitation associated with RoSE will be whether or not there is someone overseeing the validity of the information that is being added to the databases. It would be easy for someone to add false information to RoSE without some kind of moderation on the users [sic] actions” (Liu et al. 47). Such comments encompassed conceptual issues concerning who has the authority to produce knowledge in online environments like RoSE, and what this authority means, questions circling around the margins of the Distracted Reading class itself. As both developers and teachers, it was exciting to see the degree to which students took advantage of the use scenario as an opportunity to meditate on the themes of the class in which the use scenario was taking place. Comments like these demonstrate the ways in which RoSE can be used as a pedagogical tool. Exploring RoSE as an example of new modes of online reading and textual analysis, for example, helped students to apply concepts from their class in a different context.
Due to the nature of the assignment students were given, much of their feedback centered on the storyboard feature. This feedback focused on both usability and conceptual issues. For example, one student suggested that “the storyboards [become] more telescopic and interactive,” offering different ways this goal might be accomplished (Liu et al. 44). Another student thought “the storyboard could be improved with some small changes such as more color options, being able to adjust the size of the canvas and of the nodes, being able to edit and abbreviate the text that appears with the nodes, and of course being able to save [one’s work]” (Liu et al. 44). A third student “found that perhaps the ability to expand the size of your storyboard would also be more helpful for times when those dealing with [End Page 215] a very complex grouping of relationships need it” (Liu et al. 45). This feedback helped us to focus development efforts on the storyboard feature. We followed many of the suggestions the students gave us, adding to the functionality and flexibility of this feature by adding more color, shape, and connector options, and by adding the ability to save storyboards and load previously saved storyboards. More importantly, perhaps, the use scenario also signaled the students’ identification of the storyboarding feature as an important component of RoSE. Two-thirds of the students who responded to the questionnaire rated the storyboard as RoSE’s most important feature. As this feedback shows, many students saw this feature as one of the most unique components of the system, something that clearly separated RoSE from other social networks and bibliographic tools.
The strength of the positive student feedback about the storyboard feature was surprising to us, but it did help us solidify our own understanding of the storyboard’s general significance for the system as a whole. When we reflect on this shift in the project’s white paper, we write, “We were surprised how clearly the purpose and shape of our project came into focus . . . once we decided that the end-user should get out of our system the ability to ‘build’ a visual canvas ‘interpreting’ an argument” (Liu et al. 14). This purpose and shape came to center on the ability to produce a visual narrative of research that the user has done in the system. Unlike RoSE’s more familiar social network visualizations, the storyboard feature does not produce high-impact visualizations at the click of a button; instead, it provides the opportunity for the user to interact and experiment with the system, constructing his or her own interpretations of the research he or she has done in the system.
This emphasis on constructing, on building, is the pedagogical core of the RoSE system. As mentioned in the project white paper, the storyboard feature “engage[s] student thinking in a particular way that revolves around the active production of or interaction with information/knowledge” and it “work[s] on a meta-level to recover the mental process of organizing information into narrative, argument, ‘story,’ or knowledge” (Liu et al. 14). The storyboard feature can be understood as “pedagogical,” then, not because it necessarily teaches students something they didn’t know before about Shakespeare, for example, but rather because it asks students to create and construct this knowledge about Shakespeare from what the system gives them. In this way, the storyboard feature shows how RoSE is not “just” a pedagogical tool; it is also a mode of pedagogy. It asks students to contribute to communities of knowledge—perhaps by adding a character or commentator to Shakespeare’s network, perhaps by creating a new collection of publishing houses that printed Shakespeare’s works—and then it asks them to process their own participation in these communities. The storyboard feature, in many ways, shifted our understanding of RoSE from an environment where one goes to “do research” to one where [End Page 216] student researchers go to discover and make sense of knowledge. RoSE is not just a “research” environment; the storyboard feature shows us how RoSE is also a discovery and learning environment.
“Screwing Around” with RoSE
How, then, does this involvement of undergraduate students in the project development process relate to pedagogy in the digital humanities? More broadly, how can RoSE itself be understood as exemplary of digital pedagogy? There has been a resurgence of interest surrounding the term “digital pedagogy” of late, and there are, of course, many different understandings of what it can mean.5 By way of conclusion, we want to focus here on one aspect of the term that surfaces again and again: the value of experimentation, of playfulness, of what Stephen Ramsay refers to as simply “screwing around.” In his piece on screwing around, Ramsay wonders if it is possible to see “screwing around, broadly understood . . . as a research methodology” (7). In other words, he emphasizes understanding as the kind of “serendipitous journey” browsing the internet makes possible—if you like one thing you might also like another thing, on and on—as a kind of research (7). Katherine D. Harris has applied this concept to her teaching and implemented ways of “screwing around” in class by having her students blog collaboratively about each other’s ideas and the texts they are reading, hoping to emphasize the kinds of unexpected insights collaborative work can generate. Similarly, Jesse Stommel states that digital pedagogy is “less about knowing and more a rampant process of unlearning, play, and rediscovery,” and Cathy N. Davidson likewise writes about digital pedagogy as “a different way of knowing the world” that courts “failure” and “unlearning” (Stommel; Davidson, “Collaborative Learning”). As these examples indicate, digital pedagogy often involves an openness to experimentation, collaboration, and even failure—a willingness simply to try different things and see what happens.
RoSE encourages users to experiment with different ways of arranging, interpreting, and, ultimately, knowing. Each part of RoSE allows for a (re) discovery of knowledge that aligns it with digital pedagogy and its notions of exploration and experimentation. The visualization function is designed to highlight emergent connections and relationships between actors in the system that may have otherwise remained undiscovered. In addition to the visualization options that allow for the visual discovery of new relationships or objects, the storyboard feature begins with a blank canvas and offers a flexible medium for the organization, representation, and discovery of knowledge. Even a more minor feature, like RoSE’s history-tracking tool, which produces a breadcrumb-style record of a user’s movements through the system, is designed to encourage free exploration; by removing the need for the user to manually document every new discovery he or she makes, this feature increases the user’s exploratory momentum. [End Page 217] Though the system allows for a more rigid approach to locating and cataloging specific resources, it affords users multiple points of entry into its information stores and lends itself particularly well to “freeform” research.
The flexibility of the system is due in large part to the flexibility of the development process we used when working on RoSE. This process was iterative and the insights and goals of the development team were constantly reinforced, challenged, or balanced by external feedback and practical experimentation. In terms of internal play, several meetings were spent with team members exploring the system and calling out issues, suggestions, or just trying to break things in order to give us a good idea of where to redouble development efforts. The iterative development process encouraged the incorporation of undergraduate students into the project’s development cycle because it specifically asked them to try and break the system, identify its shortcomings and highpoints, and otherwise play around and let us know what they thought. In other words, it was a low risk, high reward process. Giving all developers—including undergraduate students—a degree of responsibility and power to impact the project while also constructing a context of play and exploration generated unexpected insights and enthusiasm.
Indeed, this is perhaps the most valuable insight we gained from involving undergraduate students in the development process. We learned, quite simply, that the involvement of students in this iterative development process was integral to the project itself. The undergraduate students who participated in our use scenario were not just our guinea pigs; rather, they became our research partners. Like all members of our team, they played around with RoSE, reflected on how the system might best serve users’ needs, and suggested changes. This student involvement undoubtedly improved RoSE’s features and functionality, and we believe that many DH project teams would benefit from more direct involvement from undergraduate students in the development process. Beyond simply improving RoSE, however, involving undergraduate students in our development process also demonstrated how iterative project development itself is a pedagogical technique. Asking students to use, break, and comment on a project currently in development—and then, ideally, repeating this cycle—transforms how they think about the project itself and about their roles as researchers, students, and developers. This process is capable of producing concrete and actionable feedback on a project, while at the same time encouraging the kind of exploration and unlearning at the heart of digital pedagogy.
Our experience with RoSE has certainly proven to us the pedagogical value of screwing around, but it’s also shown us how valuable experimentation and play can be for research as well. In fact, RoSE has shown us that the differences between “research” and “pedagogy” are often smaller than we think.6 When we wrote at the end of the last section that “RoSE is not [End Page 218] just a ‘research’ environment” but also “a discovery and learning environment,” perhaps we should have written that RoSE is an environment where research is discovery and learning and vice versa, where research and teaching both involve experimentation and play. While it’s true that issues related to research have often overshadowed those related to pedagogy in the digital humanities, perhaps what we should strive for are tools, scholarship, and practices that take both research and pedagogy into account. Perhaps we need to pay more attention not only to pedagogy, but also to how pedagogy and research can inform and influence one another. Perhaps we need to browse, experiment, play, and explore just a little bit more.
Lindsay Thomas is currently completing her dissertation, “Security Media: Speculative Fictions and Technologies of Preparedness,” at the University of California, Santa Barbara. This project examines the digital media and technologies used by national security agencies to project possible future catastrophes alongside popular science fiction films and novels, arguing that these various media train us to accept catastrophe as part of everyday life. In addition to the RoSE Project, she has worked for UCSB’s Scanner Praxis Project and as the lead research assistant for 4Humanities. Her writing has appeared in Media Fields Journal, O-Zone, and Surveillance and Society. She will start as Assistant Professor of English at Clemson University in the fall of 2014
Dana Solomon is a PhD candidate in English at the University of California, Santa Barbara, working on the use of information visualization in the digital humanities.
1. For more on RoSE and its features, see the RoSE Documentation site.
2. The project abstract can be found via the Digital Humanities Start-Up Grant page of the National Endowment for the Humanities site.
3. We originally conceived of three use scenario studies, each involving a subset of our target audience: an individual academic researcher, a group of participants at a conference, and the above-mentioned undergraduate class. As our thinking on the primary audience for RoSE developed, however, we came to focus most of our energy on the classroom use scenario. Though we completed all three studies, the undergraduate class study was the most extensively developed and the most productive in terms of actionable feedback.
4. See the project white paper for the narrative report, samples of student feedback, and the questionnaire (Appendixes E, G, and H, Liu et al.). The white paper can be found via the Start-Up Grant page of the NEH site and is also published on RoSE’s documentation site.
5. The scholarship on digital pedagogy is large and varied. Hirsch, for example, emphasizes that pedagogy has started to resurface as an important area of concern in the digital humanities, citing a number of recent conferences, panels, and publications focused on digital pedagogy (and this special edition is certainly another example of this shift; see Hirsch, pages 6–7). For more on this renewed interest in pedagogy in the digital humanities, see “Digital Humanities Made Me a Better Pedagogue: a Crowdsourced Article.” For a great review and reflection on much of this scholarship, see Jesse Stommel’s “Decoding Digital Pedagogy, pt. 2: (Un) Mapping the Terrain.” See Cathy N. Davidson’s “Collaborative Learning for the Digital Age,” Paul Fyfe’s “Digital Pedagogy Unbound,” and Sean Michael Morris’s “Decoding Digital Pedagogy, pt. 1: Beyond the LMS” for many more examples of what the term can mean.
6. See Davidson’s “Research is Teaching” for more on this subject.