XRMust_OffRails_Still06b.jpg
November 16th, 2022 | by Agnese Pietrobon

"We need to offer artists a place to express themselves" - Ari Tarr (OFFRAIL / SEIKUKEN)

Not only Ari Tarr's latest works offer actors and improvisers a space where they can express themselves. They also open a path to better understand the audience and the psychology behind VR, human interactions and human behaviour.

We discussed with him the in-progress SEIKUKEN and the engaging OFFRAIL, an immersive theatre performance that debuted a few days ago at Fivars 2022 and is now at Raindance Immersive. From his words, we discovered something new about a world, that of theatrical improvisation, about which there is still much to learn.

Cover: OFFRAIL, photo credits Carlos Austin

A new year at Raindance Immersive

ARI TARR - Marie (a/n Mária Rakušanová, curator of Raindance Immersive) is so amazing. I love how she put this festival together and for us to join an event like this is magical… a magical exposure! After all, VR theatre pays the bills as little as IRL theatre does! (laughs)

Anyway, Adventure Lab attended Raindance once in 2020, with Dr. Crumb's School for Disobedient Pets (a/n the experience won Best Multiplayer Experience and Outstanding Achievement in Design) and we had a great time. The experience put me in touch with some truly amazing virtual community members and it was then that I realized that the Metaverse is already here and better understood the impact of VRChat in building a VR culture and society, made possible by the incredible community building resources it offers.

At the time, I was working with a Canadian company called Flipside to develop shows that would run on a Unity-based platform, and the challenges we were facing were the same ones The Under Presents encountered: we are forging a new genre, at the intersection of immersive film and theater, broadcast and game design. The people who work on it built their VR experience mainly during these shows and their multiple performances. That’s how they created a unique talent. No one else has so much experience in running these kinds of immersive shows. You can imagine it all day long, but if you haven’t done it, you simply don’t have the skills! So, what I really wanted was a place where we could continue using those skills! And that’s why I kept working in this field.

XRMust_OffRails_Still02.jpg
OFFRAIL, photo credits Carlos Austin

About the shows at Raindance Immersive: SEIKUKEN and OFFRAIL

A. T. - I have two projects, this year, at Raindance: SEIKUKEN and OFFRAIL.

OFFRAIL's influence is AI art-generating apps. It is coming for all artists and the question is: when will it come for performers too? With this work I wanted to explore the concept before it even happens, so OFFRAIL became an exploration of the very near future of when we will be obsolete.

I also wanted to make it a container for all of us artists, in a way, because so far every single project I have been involved in has been driven by engineers and game designers. A big part of my job is to facilitate communication between the artists and the teams of engineers working on the projects, who speak a very difficult language and often need a translator!

Generalizing a bit, of course, I have always noticed the way engineers tend to underestimate what we artists can do. Artists know how to keep something going so that, even when the story gets a little off track, they find this digression a special addition to the work. Also, if there is a technical “crack” in the work, as an artist you can make a joke about it, turn it into a non-issue, even a narrative element and that at the minimal cost of an hour for an actors’ time. But if you pay an engineer to solve the same problem, the cost certainly becomes different and maybe in the end the issue is still there. So, at the base of this project there’s also the need to offer artists a place to express themselves.

The other motivation behind the creation of OFFRAIL was the enthusiasm I felt in working with high-demand, high-level theater improvisers and training them to use VR as a tool to express the skills they already possess. Expert hosts, like the ones we have, offer the chance to experience more and with better interactivity. My goal was to allow them to create an interactive, basically improvised story, in which to find clever ways to guide the audience through it so that the audience itself could have more power than ever before. The Under Presents certainly demonstrated the strength of the community they were able to build, but they offered silent shows. Meanwhile, at the time, I was doing hyper-focused, customized shows that spoke to the audience. What I did with OFFRAIL was to take the lessons learned from both perspectives and try to combine them in the same piece, with the final goal to empower the performers.

XRMust_OffRails_Still0.jpg
OFFRAIL, photo credits Carlos Austin

My other project at Raindance Immersive, SEIKUKEN, is more serious. It is not a comedy, but a work that ties in with my background in physical theatre, including Japanese theatre and in particular mask theatre. The work is based on some studies from the Stanford VR lab that have obsessed me. One of these was Jeremy Balinson's study on so-called Zoom fatigue, which - reflecting on 2D and 3D environments - explains why you always feel exhausted after a Zoom call.

For me, the purpose of SEIKUKEN - which is a kind of mask and puppet play - is to allow people to see that there is another side to communication. Scientific experiments show that 7% of how we communicate is the words we choose, the text. 38% is the tone of voice we use. The remaining 55% is what is said with body language. This means that even when speaking in a normal phone call, we miss half the conversation, and this is what has always happened with the media.

VR brings back the other half. The physical presence adds so much and what I'm trying to explore with this work is how you interact with someone in that 3D space and how this technology allows you to do interactive things in a virtual environment that no other technology gives us the ability to do today.

While working on this project, I was also reminded of director Tadashi Suzuki's method for training actors in improvisation, which derives from traditional Japanese theatre such as Noh and Kabuki and is based on the concept of the invisible body. According to him, there are three invisible elements that we experience but do not take into account in everyday life and only notice by their absence: the oxygen we take in, energy production, and balance. And they are all relevant in VR. Maintaining balance control while resisting the wonderful mind games that VR can plays is very important in VR actor training. For example one of the neurological effects of VR produces vection, a vestibular illusion that tricks the inner ear into believing you are moving when you are not (and responsible for the painful “VR fail” memes)

SEIKUKEN is intended for a very small audience, but for everyone else there will be a small snippet of this performance at the Raindance awards ceremony that I will post. This project is also part of my VR theater curriculum: I taught at NYU Tisch and will be teaching next year at York University, where I use the amazing Bauhaus style avatar designs by Joost Eggermont to teach mask and mime training in VR for my students. Thanks to my Production manager Nithia, I will also be offering a test round of physical theater movement classes for students in VRChat this Spring.

XRMust_OffRails_Still08.jpg
OFFRAIL, photo credits Carlos Austin

Playing with comedy and with drama in VR

A. T. - Dasha (a/n Dasha Kittredge), who I worked with as a teacher at NYU Tisch, was in THE UNDER PRESENT and has her own immersive design company in LA; she was also a puppeteer in LA and is a fantastic comedic improviser. One thing she said that I very much agree with is that VR at this stage is better suited to comedy, particularly if you use Quest.

There are certainly other possibilities coming up, however: I'm currently working on an Unreal Engine project that requires a full body and facial mocap to create a hyper-realistic avatar. That's where I think the drama can come in: when you can show the subtlety of your performance on your face.

But there is also another reason to work with comedy today. We live in dark times. Think about what Alan Moore (a/n a comics artist and writer) said: happy times are when you create dark things as a warning. In dark times, it is difficult to follow that direction and you have to find lighter things to discuss. But… it is certainly harder to be taken seriously when doing comedy.

The role of VR in psychology and as a therapeutic tool

A. T. - The applications of XR technology for entertainment, education and business are really exciting applications that are creating new industries that are a new exciting frontier. However, beyond the applications, it is the future convergence of these technologies (XR, AI, haptics). There is a lot of research about how XR tech makes the brain store memories very differently, and most of the coverage we see today is about how XR can be used to cause real kinds of PTSD with virtual assault happening in Horizon worlds, etc, that can cause real lasting trauma, and of course that is terrible. But in my opinion it shows how much research still needs to be done, because if XR can harm us in this new way, it begs the question, is it able to heal us in new ways as well?

I worked in medical VR for a while, and there I met people like Dr. Albert “Skip” Rizzo, Director of the University of Southern California Institute for Creative Technologies Medical VR Lab. He's working on using VR to treat PTSD. In the UK, there's also a psychology program called UXF, Unity Experimental Framework, to assist the development of virtual reality human behavior experiments that tries to overcome the replication crisis in science. One of the fields that, in my opinion, is most interesting to explore is that of immersive theatre.

XRMust_OffRails_Still04.jpg
OFFRAIL, photo credits Carlos Austin

I've always thought that when you get enough people involved and give them permission to act and have agency, they start to distribute themselves statistically, and eventually you can start to understand common behaviors.

One of the things that happens is that you indirectly give them permission to take off the social mask they are wearing and in this way the experience ends up looking a lot like what you do in therapy. Looking, for example, at the work of my mother, a therapist specialized in neuro-linguistic programming, one can see strong parallels with how VR works: VR allows you to use your body in space and even use some of the more subtle therapy techniques that don’t involve speaking at all. VR requires psychology and a great deal of empathy, especially in this type of installation where you need to understand very quickly who the person is, what they are going to do, how to help them participate. VR allows you to experiment and play with your identity in a controlled environment. I know from my experience in VR theater and creating VRchat worlds for live therapy sessions and speaking with Skip Rizzo, who cures PTSD with VR, the potential of these technologies to assist humans in healing trauma. By creating interactive psycho-physical immersive worlds to explore (think a 3d version of C.G. Jung's “Red Book”) and train themselves to be healthier calmer beings outweighs the costs of the current corporate/advertising dystopia.

So, ultimately, I believe that VR will take sandplay therapy, dramatherapy, art therapy and effectively combine them with technology and artificial intelligence to co-create interactive myths with qualified therapists that will help heal patients better than previously possible with the current state of telehealth, or even IRL therapy.

I love to make people laugh and there are great apps and shows out there that do that fantastically. At the same time, I can't resist the future possibilities of using VR for actual therapy: it's a more powerful tool than many people give it credit for, and I think it has the potential to heal much more and to have a much bigger worldwide impact than we expect.

On the challenges of VR and on VR communities

A. T. - What is happening in VR and especially in platforms like VRChat is very interesting from an anthropological perspective. Let's take the example of furries: at first these communities creeped me out, but then I started reading them under the lens of sci-fi anthropology and I changed my mind. Of course, they pose a problem when proposing VRChat to corporate investors who don't know much about the field, because they end up being all they see. At the same time, when creating an experience, you need to look at both sides: those who come from outside and are unfamiliar with VR (but perhaps have the financial means to bring your work to life) and the people who are part of that world and know it all too well and expect certain things from your pieces.

The interesting aspect of platforms like VRChat, however, is the presence of strong improv communities that want to learn the art from professionals. For example, our production manager comes from VRChat's improv community, but has never done improv outside of VR Chat. They work as a production manager for these career comedic improvisers who work as professionals in the industry and there is a really strong combination between these two communities, the virtual and the IRL one.

XRMust_OffRails_Still03.jpg
OFFRAIL, photo credits Carlos Austin

A. T. - And then there’s the audience.

I've collaborated on many immersive installations that include interactive performance elements, whether it’s psychedelic tech/art for music festivals or brand ambassadorships…. and as immersive designers we never know what the work is until we’ve actually seen how an audience will interact with it. Human psychology is so unpredictable that the audience makes choices that you could never expect, sometimes hilariously! But after many performances, patterns of audience emerge and the actors develop adaptable scripts for every possible situation… It's almost like running a psychology experiment, without collecting data. When done properly, the actors, audience and crew become a feedback loop that lets the artists use the tech to create a collaborative story that can adapt to any new situation and balance a satisfying order with gleeful chaos. Sometimes the most amazing interactions happen when the story starts following a specific new direction no one ever thought of, and then… it never happens again, making that moment a unique interaction. These magical moments have always existed in live immersive theatre, but thanks to XR media, they can be captured, shared, amplified and scaled all over the world instantly.

IRL theatre and virtual theatre: the problem of acceptance

A. T. - The problem of acceptance by the 'serious' theatrical and academic circles exists and amounts to about... 85% rejection. There are, however, some interesting exceptions, such as that of the great Robert Barry Fleming, the Artistic Director of Actor’s Theater of Louisville, a director with whom I worked with for an updated version of A Christmas Carol, where ghosts were created using Unreal Engine.

Generally, however, the attitude of those working in academia or classical theatre is to see these new approaches as a threat to a more established style. Mostly, I think, they do not want to change the way things have been done so far, because that is what they are used to. I understand that, to be honest: VR is not easy to do right now. You need a lot of patience and a dose of obsession... personally, I think I have some form of ADD that has pushed me forward even when I should have stopped! (laughs) Sometimes it was definitely stressful, because we are still in the start-up business and my father calls it 'jumping from ice flow to ice flow' and sometimes that next piece of ice is pretty far in the distance. But I'm still extremely grateful for it all and I'm really committed to how physical expression combined with spatial computing will evolve our society over the next few years..

XRMust_OffRails_Still07.jpg
OFFRAIL, photo credits Carlos Austin

Incoming challenges for immersive virtual theatre

A. T. - There are so many things that complicate the development of an immersive piece. I presented prototypes of OFFRAIL to investors in 2016, knowing that eventually we would have to build a backend where people could buy tickets and participate in the experience. The Under Presents worked very well from this angle, getting all the right people into the right instance at the right time, and working with Adventure Lab to help design their system made me understand how much work is still needed in a fractured XR landscape where friction is still the greatest impediment to mass adoption. In fact, I would say it is one of the biggest obstacles to widespread adoption in this industry. The only thing that has allowed us to make such progress is using social VR rather than building a custom VR client for performances as was necessary in the past because the strong networking architecture makes performances as dependable as possible (every show still has inevitable network errors! And our actors are still using Quest 1s from previous VR shows…if anyone in the industry can send them a loaner Quest 2 please reach out!)

But what has really made the difference as far as our ability to challenge the conventional limits of this new medium, is the generosity of the VRchat creator and pre-fab community, which has allowed me to bring my 11-point tracked live body and vive face tracking in a live performance with humanoid avatars, which I think is pretty groundbreaking. Finally I need to thank our crew of improvisers who are fitting in these VR rehearsals and shows around their IRL acting and teaching work in New York, L.A., and the UK. And I want to thank Festivals like Raindance Immersive and FIVARS that give the small amount of professional XR performing artists a chance to work as an ensemble and share our work with ticket holders and communities that are hungry for new styles of this incredible and evolving immersive medium.

This time we don't have the benefit of massive VC funding, but it has become apparent that not only have the industries of entertainment, corporate events and education begun to seriously look at real world uses for the “metaverse,” but also that there is a thriving community of passionate creators and innovators creating the future of how communities will develop on the future of the 3D internet. After working with wave after wave of abandoned metaverses, right now I am starting to believe the future of the underground is happening now in VRChat. I’m seeing real communities thrive that bring in our full ability to communicate as empathic humans in real time rather than polarizing mediums hacking our brains with reduced information in a tweet. My choice was to focus on the more purely acting aspect, but I had to expand my production crew and luckily Maria helped me, but also Mandy (a/n Mandy Canales, cast member of OFFRAIL), who is part of the cast, and who contacted the improvisation community and found people who donated their time to expand the production crew as volunteers... I mean, it was great to see! I feel like I’m still in my twenties, when I first started performing: limitations require artists to use their creativity to overcome those obstacles, and the work is generally better and more relevant as a result of that struggle.

XRMust_OffRails_Still05.jpg

Growing up in a successful performing family, I learned that even with the illusion of fame, you don’t go into live performance to make money, you do it because you love it so much you can’t bear to do anything else. But what I have always seen is that even though the vast majority of performers do not do it as a full time career, these performances have built some of the strongest communities I’ve ever seen. This is something that is also happening with virtual theatre, except now XR allows you to create that community with anyone around the world. People come forward to help you with your work simply because they want to see it happen! It makes you believe in the future of the internet and virtual communities and the possibility of keeping these technologies in the hands of artists first, instead of just to profit the big companies. VR is changing people's lives in new ways, and it allows us to create something we really want: stronger communities built around new technology that allows us to scale back our abuse of fossil fuels... which is what we need in this world today.

.


XRMust will be back to talk about OFFRAIL in the coming days. In the meantime, you can visit Ari Tarr's Twitter profile to keep up to date with all the news about these shows and the world of improvisation.

Share article

Related posts

XRMust_Mutek2022_04.jpg
September 1, 2022

Editorial⎪Coming back from Mutek Montreal Forum 2022

From August 23 to 26, 2022, hundreds of artists and thinkers gathered at UQAM's Cœur des Sciences for the eighth edition of the MUTEK Forum. Once again this year, the Forum offered a series of activi…

XRMust_WtR_GIIOII_CoACT.jpg
May 22, 2022

Immersive studio Giioii is adapting theater XR performance WELCOME TO RESPITE for Korea

Digital Broadway emerges as pioneers in the VR theater world team up to bring a show that originated in English to a Korean speaking audience.

XRMust_Targo_NotreDame.jpg
July 14, 2020

“We have to make content that is going to speak to everyone” - Victor Agulhon (TARGO)

One of the main problems in relation to immersive productions is the audience: how can we make more people interested in this medium?