By Shanti Pillai

Artists working with machine tools are forging new creative ground as they pose essential questions about consciousness, ethics, and aesthetics. Some are also pushing conversations about social identities through the lens of AI. Toronto-based, Indian Canadian artist Debashis Sinha, whose sound works cross the worlds of theatre, dance, and multimedia installation, began using neural networks in 2019 to further his exploration of the sonic significance of storytelling and cultural heritage.1 He situates his work within a social justice paradigm and the need to amplify marginal voices in debates about technology. In this interview with Shanti Pillai, who is an assistant professor of theatre at Williams College, he discusses his lifelong curiosity about percussion, his passion for collaborating, and the excitement of discovering new stories in the digital world.

Shanti Pillai (SP): Creation begins with sound in the Hindu philosophical tradition. How do you see the significance of sound?

Debashis Shina (DS): I tell my students in the Theatre Production and Design program at York University that our secret power is the fact that we work with sound. All humans have been swimming in sound since before we even took our first breath, so our relationship to sound is encoded in our DNA, in our bones and flesh, whether we perceive it or not. It can't help but be—for lack of a better word—spiritual.

SP: Now that we've established the existential basis, let's start with the elephant-headed god Ganesh, who in the Hindu tradition is invoked at the outset of any endeavor. It so happens that one of the series of short videos you're working on presently is titled The Elephant Headed God Ganesh Sleeps in the Bamboo Grove. Can you tell us about this twenty-first-century incarnation of the beloved deity?

DS: To be honest, I am not sure why I chose this. I'm working with a neural network called CLIP; it's a library that was released by Open AI and ported to Google Colab and GitHub.2 It's a text-to-image neural network. This phrase, "The elephant headed god sleeps in the bamboo grove," just came to me and I typed it in. The videos that make up the piece are outputs from that sentence. In my machine learning (ML) research, I'm interested in exposing neural networks to sounds and images outside their worldview. Networks are trained on internet data, and that leaves out a huge part of human experience. This network might know what Ganesh is, or an elephant, or bamboo, but it doesn't understand putting them all together. As soon as I made the first video, I saw a story. It struck me because so much of my understanding of what it means to be Bengali or Hindu came through fables my mother and grandmother told me. I'm a father now and I want to share that love of stories in transmitting my heritage.

SP: Is the sound in the video also generated in the neural network? 

DS: I didn't do that in this case, but I have done that in other works, such as rig_veda000_03 (, where I exposed networks to field recordings I collected in Kolkata (among other processes). Here, I use some recordings that I collected and some from sound libraries.

SP: You began your artistic journey as a percussionist. What was your initial training?

DS: When I was growing up in Winnipeg, we had a very small Indian community. We didn't have the cultural resources that second- and third-generation immigrant kids have now. I was piecemealing together an understanding of my culture. When I was five, I saw on television Mr. Dressup go to a music store and meet a drummer named Steve (Mr. Dressup was a children's show on our public broadcaster, CBC). I was enthralled and started lessons at a music store with bitter teachers who really wanted to be rock stars, not teachers! I stopped quickly and after a few years I taught myself by playing along with records. In my teens, I got more experimental. What happens if I set up the drums backwards? What happens if I roll the bass drum across our basement floor? I also learned a little bit of tabla, the set of two-hand drums played in Hindustani classical music. Much later in life, I received funding from the arts councils here in Canada to study with drummers in different parts of the world. That curiosity about percussion instruments still drives me.

SP: It's interesting to think about how the cultural orientation of your household shaped your autodidactic process on Western instruments.

DS: Yes, at home and at religious festivals we listened to music all the time. My father had wanted to become a musician, but had to give it up to take care of his family. And my mom had grown up in Kolkata dancing in Uday Shankar's troupe—they did a lot of cross-cultural experimentation. She had a troupe in Winnipeg that my sister and I performed in. She would do things like invite an Indigenous drumming group and a ballet dancer and just put them together to see what could happen. So I appreciate the depth of Indian tradition, because I learned that it can give us the opportunity to build new things, like people have been doing for thousands of years.

SP: In Canada, you work with theatre companies such as Soulpepper Theatre and at the Shaw and Stratford festivals. In addition, you work in film and television. How has collaborating in these contexts nourished your own artistic voice?

DS: Every time I do a play, somebody says something and it leads me down a completely different path. In theatre, I've made music and sound that I never would have if left to my own devices or if I was working with musicians. When I speak to a designer, actor, or director, we don't share a language; we meet in this shared middle ground of the unknown. That generates new ideas for me to follow.

SP: You also have a long working relationship with Peggy Baker Dance Projects.

DS: When I came to Toronto, I made a living accompanying dance classes. Teachers talked about opening up space between your vertebrae or imagining something on top of your arm. Creating images with the body told me a lot about music. I am fond of telling Peggy that I consider her my teacher in musical composition!

SP: When did you begin using ML tools?

DS: In 2019, the person who runs the label that I released my last record on, Peter Kirn, wrote on his blog about MUTEK—Japan's upcoming AI music lab in Tokyo. I was ignorant at that time about what was happening with AI and artists and I was curious. The lab was an incredible ten days. It sparked my interest in creating a mediated performance environment where we could use real-time ML models to interpret hand gestures or to create a large library of content with which to improvise. Some of that has come to pass with some of the live stream concerts I've done and with the record that's coming out. But it immediately became clear to me that these tools made content I would never have made by myself. The question of what stories I could uncover that are encoded in that latent space became very exciting.

SP: You have said that your work participates in the radical and ethical AI communities. Could you explain the core interests of those communities and the ways that your work fits into their ongoing reflections?

DS: Those conversations seek to imagine ways of using AI tools that are aligned with social justice and with amplifying marginalized voices. Last year at NeurIPS, there was a workshop called "Radical AI" that brought together folks interested in these issues. They placed one of my works in their online gallery and another in the conference online gallery. In reading the comments, it was clear that these scientists, engineers, and activists have an openness to discovering new stories that haven't been told by neural networks yet. It's great for me as an artist to feel like there's space for me in that conversation.

SP: You have explained that in your process you "collaborate" with a machine, but what does it really mean to collaborate with a computer?

DS: Collaboration is a conversation where you're exchanging ideas so that new ones arise. I'm offering something to these networks that they can't handle and I see how they respond. There's a process called a style transfer, where you feed the model some source material—say a free jazz improv—and it thinks, "That is what sound is." But then I feed it a recording of an arathi (a moment in a Hindu ritual in which a flame is offered to a deity accompanied by singing or the ringing of bells) and so it spits out something else based on its understanding of the initial sound content. I see what comes back and I respond to that and make something, or continue with the process.

SP: Some people assert that AI will replace many forms of labor. Isn't there the possibility that the human artist may be rendered obsolete?

DS: That is never going to happen. There are still people there in the labor chain. For example, today we have AI-powered robots in factories that clean up spills. The robot sees a spill. It pipes the video back to somebody in, say, the Philippines who has to sit in front of fifty screens and identify the spill and then deploy the machine with a click. There's so much hidden, exploited labor, much of it on the backs of people in other parts of the world. And in terms of art, no way. There are now websites where you can type in a bunch of words and then the URL will spit out some music. That's a kind of art … I guess. But in terms of the things that we humans do—that involve critical conversations around content and processes and creating—that is never going to stop. And as AI gets more sophisticated, that will open up more questions for us to investigate as artists.

SP: I want to turn to one of your works right now. I was wondering if you could walk me through and the resplendent. What did it mean to train an ML algorithm on an English translation of the Rigveda?

DS: In Tokyo, they sent us home one night and said, "Tomorrow come back with 10 megabytes of text." I thought about the many English translations of the Vedas. I was pretty sure something strange was going to happen; the recombinant neural network (RNN) takes the text you give it, breaks it apart to understand how it's structured in terms of what is a word, what is the space, what is a comma, and then puts it back together again. When it's a huge amount of text, it takes a long time for it to train itself. If you interrupt the training before it's done, you'll get an "almost." It's like reconfiguring the archive to discover a new story. I interrupted the machine after about 30 epochs, or 30 rounds of training. When I read the output of generated text, it blew my mind. It was nonsense, but it had a structure. It made me think of the thought processes of divine beings. They're not bound to the same physical rules as us, so why should their consciousness and language be the same? This felt like a great storytelling conceit. The RNN-generated text is like language at the outer edge of our consciousness. There's a vocalist here in Toronto who read the text for me as you hear in the recording.

SP: Why not use the original Sanskrit, a language in which carefully ordered sounds are meant to shift our experience of consciousness?

DS: Yes, in Sanskrit we summon the hum of the universe. Perhaps it's a missed opportunity. I don't think there are any AIs that can translate Sanskrit, and even if they could, it may not be readable by someone who knows the language. It's an interesting idea, though, because of the sonic ramifications of Sanskrit. I guess this current work is mediated by own cultural gaps.

SP: Last but hardly least, we're having this interview during the pandemic. What possible futures do you see for live performance?

DS: One of the things we've discovered is that the hunger for stories is never going away. And a digital infrastructure can allow us to deliver storytelling that can be moving when well-crafted. I suspect that digital offerings will continue and may also become part of the live experience. I also think we can move towards a theatrical world that's more inclusive. Sure, there will be excitement about returning to theatres with expensive tickets. But hopefully the lessons we've learned from this pandemic are not just about the delivery of stories, but also about who we involve in their making and with whom we share them. 



1. Neural networks are sets of algorithms that enable computers to recognize patterns in input data as part of processes of classification or problem-solving.

2. "Ported" refers to the process of reformatting code to create compatibility with various digital architectures. In this instance, specific adjustments were made to the code to allow it to run in Google Colab, a notebook environment that operates entirely in the cloud.