Discovering New Stories in What AI Doesn't Know: An Interview with Audiovisual Performance Artist Debashis Sinha

"Shanti and Debashis Have a Conversation," as generated by text-to-image tool, CLIP (see https://colab.research.google.com/drive/1FoHdqoqKntliaQKnMoNs3yn5EALqWtvP?usp=sharing).

[End Page E-15]

Artists working with machine tools are forging new creative ground as they pose essential questions about consciousness, ethics, and aesthetics. Some are also pushing conversations about social identities through the lens of AI. Toronto-based, Indian Canadian artist Debashis Sinha, whose sound works cross the worlds of theatre, dance, and multimedia installation, began using neural networks in 2019 to further his exploration of the sonic significance of storytelling and cultural heritage.¹ He situates his work within a social justice paradigm and the need to amplify marginal voices in debates about technology. In this interview with Shanti Pillai, who is an assistant professor of theatre at Williams College, he discusses his lifelong curiosity about percussion, his passion for collaborating, and the excitement of discovering new stories in the digital world.

Shanti Pillai:

Creation begins with sound in the Hindu philosophical tradition. How do you see the significance of sound?

Debashis Shina:

I tell my students in the Theatre Production and Design program at York University that our secret power is the fact that we work with sound. All humans have been swimming in sound since before we even took our first breath, so our relationship to sound is encoded in our DNA, in our bones and flesh, whether we perceive it or not. It can't help but be—for lack of a better word—spiritual.

SP:

Now that we've established the existential basis, let's start with the elephant-headed god Ganesh, who in the Hindu tradition is invoked at the outset of any endeavor. It so happens that one of the series of short videos you're working on presently is titled The Elephant Headed God Ganesh Sleeps in the Bamboo Grove (https://vimeo.com/513623121). Can you tell us about this twenty-first-century incarnation of the beloved deity?

DS:

To be honest, I am not sure why I chose this. I'm working with a neural network called CLIP; it's a library that was released by Open AI and ported to Google Colab and GitHub.² It's a text-to-image neural network. This phrase, "The elephant headed god sleeps in the bamboo grove," just came to me and I typed it in. The videos that make up the piece are outputs from that sentence. In my machine learning (ML) research, I'm interested in exposing neural networks to sounds and images outside their worldview. Networks are trained on internet data, and that leaves out a huge part of human experience. This network might know what Ganesh is, or an elephant, or bamboo, but it doesn't understand putting them all together. As soon as I made the first video, I saw a story. It struck me because so much of my understanding of what it means to be Bengali or Hindu came through fables my mother and grandmother told me. I'm a father now and I want to share that love of stories in transmitting my heritage.

SP:

Is the sound in the video also generated in the neural network? [End Page E-16]

DS:

I didn't do that in this case, but I have done that in other works, such as rig_veda000_03 (https://soundcloud.com/debsinha/therigveda-sinha-ocdf/s-DvFM5A5Yj1h?in=debsinha/sets/the-rig-veda/s-m7Iduw0Fb0h), where I exposed networks to field recordings I collected in Kolkata (among other processes). Here, I use some recordings that I collected and some from sound libraries.

SP:

You began your artistic journey as a percussionist. What was your initial training?

DS:

When I was growing up in Winnipeg, we had a very small Indian community. We didn't have the cultural resources that second- and third-generation immigrant kids have now. I was piecemealing together an understanding of my culture. When I was five, I saw on television Mr. Dressup...