"Technology must facilitate the story" - Anna Abrahams, Avinash Changa (ANGELS OF AMSTERDAM)

We've often talked about innovation in our previous interviews, and when it comes to storytelling, one of the best innovations you can have is a technology that is so cutting edge that most users don't even realize it's there when they're experiencing a narrative piece.

This type of innovation can be found in Angels of Amsterdam, a co-production by WeMakeVR and Stichting Rongwrong that premiered at the 78. Venice Film Festival and is currently touring the world for a festival run that looks very promising.

In Angels of Amsterdam the user is plunged into a 17th century Amsterdam inn (recreated based on the original bar in Amsterdam, which has changed very little since then) where he meets four women who will each share their own tragic story.

Through 3D scanning, volumetric images and custom techniques, Angels of Amsterdam brings a specific time period to life with an absolute realism that is often only found in 360 experiences where no interaction is allowed. This realism is further emphasized by the way in which the user connects to specific characters in the story, which feels natural and mirrors the way we connect in “real life”.

To activate a performance, in fact, all you have to do is look into the eyes of one of these young women for a moment, whose necessary length is marked by a white dot on her head: this simple exchange of glances between the viewer and the woman strengthens the connection between the two and is the engine that kicks off her performance.

From this point of view, Angels of Amsterdam is a work that could have been simply an ode to technology but finds its greatest strength in the way it manages to conceal it and let the characters, the historical period, the music and the atmosphere of that strange world do the talking and tell you how little some things have changed since then.

On top of that, the immense research and historical investigation that co-director Anna Abrahams has carried out to reconstruct the stories of these four girls - who really existed! - literally brought joy to my curious soul. And, I admit: as a woman myself, the question at the heart of Angels of Amsterdam - "what would history be like if told by those who were too unimportant to speak?" - is sincerely a question I continue to yearn for an answer to. And certainly, this is a work that tries to make the most of the possibilities offered by the medium to give it.

We spoke to directors Avinash Changa and Anna Abrahams about the origins of Angels of Amsterdam and how it fits into an immersive storytelling perspective that sees advanced technology as a tool to bring storytelling to the forefront,. Here's what they told us.

AGNESE - Anna, Avinash, congratulations on your work! I experienced it at VeniceVR Expanded 2021 and I was really impressed by what you have created. The technology is amazing, of course, but at the same time it is so advanced that, when you are just an "ordinary" member of the audience you don't even notice it. It simply disappears and lets the story take over.

AVINASH CHANGA - Thank you, that's exactly what we wanted! From my point of view, if you work well with technology, then the technological side becomes invisible. But if people get too distracted by technical novelties and tricks, then something has gone wrong. The point is: technology must facilitate the story and not the other way around. I have to say the audience response has been positive on many levels and the story behind Angels of Amsterdam is definitely a huge part of that!

A. - For me it was a story about connecting with other human beings: the way you facilitated this connection was simple and made it seem very natural.

ANNA ABRAHAMS – There's a lot of interaction in Angels of Amsterdam, but not the complicated interaction that makes you think 'What should I do? Am I doing this right?". Here you are the one activating the performances, but somehow you don't even notice it because the way you connect with the characters is exactly the way you would connect with people in real life: you look at someone and that someone looks at you and something happens.

A. C. – When we were developing Angels of Amsterdam, we thought back to what happens to a lot of people who try VR for the first time: the fatigue that sets in when you put on a VR headset is a pretty big factor, and some people get up to 10-15 minutes, and then they take the headset off because they're a bit tired.

In Angels of Amsterdam the pattern of interaction is quite clear: you choose to interact with someone and you do so simply by making eye contact. Because of this agency, and how intuitive it is, people engage in the piece, and are inclined to stay longer. So almost everyone - and this is something I heard at the Venice Film Festival - stays for the full duration, 30-35 minutes.

A. - Let's jump back a bit: how did you meet? How did this adventure together begin?

A. A. - The idea of Angels of Amsterdam has been in my head for a long time. At first, as a director, I wanted to make a short film out of it, but then I thought about VR and started talking to people about it.

What really mattered to me was to be respectful in my approach to this subject and to carefully present what these ladies went through during their lives. Some people didn't understand this, but then Michel (AN: Michel Reilhac, curator of the Venice Film Festival - you can read XRMust's interview with him here) put me in touch with Avinash and Avinash immediately understood what I wanted to do: to tell stories that may be historical, but whose dynamics we can still identify in our society today. And that's why it's important to share these kinds of stories... so that people learn something from history and recognize that some situations haven't changed: if you're a young woman, things can be difficult even if you live in the 21st century!

A. C. - Issues such as inequality between men and women, discrimination against immigrants, poverty: they are all still very relevant. We live in a time when things like that should no longer exist. For me it is surprising and shocking that they still do. This is one of the reasons why Angels of Amsterdam had to be made.

A. A. - When we talk about history, we always talk about kings, emperors and leaders. But there are many unheard stories, like the ones we talk about in Angels of Amsterdam: small stories of ordinary people who lived somewhere in Amsterdam and never became famous. We know very little about them, we only have a few documents, but we can still tell them.

A. C. - There's also the whole idea of the male gaze, which is prevalent in many stories and pretty much all Hollywood cinema. It's interesting that a creator like Anna joins the VR field and brings the other perspective. Because even in VR, we see a lot of male creators, and we see a lot of the same stories. So, there's a void waiting to be filled and a need for this kind of work.

A. A. – It started when I found a book called “Amsterdamsch Hoerdom” (Amsterdam Whoredom). It's a rough guide to Amsterdam written in the 17th century by one man for other men: how to find the best girls, how to find the best cafés, where to start drinking, how not to get into a fight.

It was a very funny book, but it was also an important historical source that I wanted to approach from another perspective: if you don't look at things through the eyes of men, but through the eyes of women, like the ones mentioned in the book, what would history have been like then? How did those women end up in that particular café? Why and what were they looking for?

So I went to the city archive and started researching different women from 17th century Amsterdam. I read many historical books and followed their notes to the archives. There I found police reports about Elsje's crime and her confession (which was literally used in our piece); a doctor's note about Maritgen, the girl who became a boy to be a sailor; an official complaint made to the police about Juliana's escape from the man who 'owned' her. As for the last girl, she is mentioned in Amsterdam Whoredom since she owned a brothel: there is an archive piece about one of the times she was arrested by the police where she states that her name is Pussy Sweet – I just loved to see a 17th century woman thinking of her sexuality as something sweet. They sent her outside the city walls twice, but she found a way around the wall and got back in: a strong woman, to me, and I liked her.

A. – This is incredible. Are these documents available?

A. A. They’re in old Dutch and handwritten so it’s quite difficult to understand them in their original for, but we have made a translation available on our website, so people can access them. We have also made an artist’s publication where you’ll find artist proof prints of the four main characters, their stories and facsimiles of all the archive documents we used (AN: you can find out more about it here and about the publication here).

A. C. – When we showed Angels of Amsterdam at the Eye Filmmuseum, Anna and I set up a table where facsimiles of the documents were available. While waiting to access the experience, some people were looking at these documents and a couple of them even commented that reading them made the story even more real, bigger. These artefacts made the experience stronger and were a way of reaffirming that those we had just seen were real people and real stories, even if they were stylized.

A. – If I went to that bar in Amsterdam, where Angels is set, would it still look like that?

A. A. - Exactly the same... Only now it has a refrigerator and a coffee machine! (laughs)

A. C. - It was built in 1642 and has not changed in terms of architecture since then. Anna found the place. We went in and did a LiDAR scan (a really high-resolution scan of the whole place, accurate to the millimetre); then we moved from that model to change things and restore the tavern to its original state: we took out the electric lighting and put in candles, changed the paintings and other details, and left everything else intact.

At the end of the experience, when the scene fades away, the café is brought back to the present day. Some people don't even notice it, but now there is a coffee machine. And you turn around and see the electric light of the Exit sign. Little details that make you realize you're back in the 21st century.

A. A. - There's also the sound: at the end you start to hear the cars outside again, and the train. Everything is quite subtle, but we wanted you to feel like you're going back to your own time.

A. C.- One thing we wanted to do, but which wasn't possible in Venice for logistical reasons, was to show the piece by putting a real table in front of you so that you could connect what you could see in virtual reality to the physical sensation of having a real wooden bar under your hands. We did this at the Nederlands Film Festival and it was beautiful! When something physical in the real world aligns with something virtual, something magical happens. It helps transport you to that time. That's why we're thinking of taking this experience to the real bar where we filmed it and having the consumers try it out while they're there.

A. - What were the biggest challenges in the creation of Angels of Amsterdam, technologically speaking?

A. C. - For the story to work, it was absolutely crucial that the users felt a real connection with the characters in the bar. Technology couldn't get in the way.

One of the best ways to visually recreate humans until not so long ago was 360 video. You put someone in a high-end 360 stereoscopic video experience, they look around and at first they're very excited because they feel like they're really there. But at some point they're always trying to move, they're always trying to get closer, and that doesn't work in a 360 video. As a creator you promise to immerse the viewer in a piece to make them feel part of the story; when the user tries to move but can’t, it's like you've just broken that promise and it can even take the user out of the narrative.

For us, it was crucial to find a way to bring real people in the experience but keep it interactive. What you normally do is scan the person and then work with motion capture, but for me in this procedure what you lose is humanity: there are no real facial expressions for the characters anymore and they all look like characters from a game who move realistically but don't look real.

To solve this problem, we came up with a mix of technologies and developed our own workflow and pipeline to capture the characters in the high-quality volumetry you saw in the experience: we used very advanced 3D scanners combined with high-resolution cameras and special lenses, and on top of Depthkit-software we developed a production-pipeline to capture these performances in a very realistic way. And then, we reproduced all this inside the bar, with the corresponding lights, to make you feel as if you were really there. From a technological point of view, it was a great challenge.

A. - You mentioned interaction: how did you work on it?

A. C. - A lot of VR pieces work with controllers: you point at things, you click, you have that kind of interaction pattern. We needed to do something different. The main thing was to find the right balance: you're inside a bar and you're looking around as you would in real life, and suddenly you make eye contact with someone. How long should it take for you to connect with this person? And how can we communicate to the user that she just activated something? With Angels of Amsterdam there was a lot of experimentation, a lot of user testing and also testing with people who don't know about VR and are new to the medium. I think we found a good compromise between what works and keeping the interface very subtle, the only visual interface you see is a little white dot​​, and that turns out to be enough for most users.

Originally we also experimented with a version where you didn't even need the dot: you looked at someone long enough and that person was "activated". It was a very smooth transition, but it had a problem: users didn't realize that they had chosen something and interacted with the piece, so they ended up thinking that the story ran on rails and everything was activated automatically.

The white dot is a less visually sophisticated choice, but it allows the user to realize that they have activated something, they have chosen something... And that makes them more engaged in the piece.

A. C. - One thing that helps a lot when you want to immerse people into a world - but it was also a technical challenge for us - is sound design. The sound design of Angels of Amsterdam has a high level of spatialization. You can clearly hear where all the different sounds are placed and where they naturally come from. The wrong sound design can really distract people from the story. Making sure that everything comes out correctly, that it makes sense, that it is well blended helps the user to accept the world as a real one.

A. - Music also plays an important role. Can you tell me something about that?

A. A. - We worked with a genius artist called Harald Austbø: he plays the cello player in the experience and wrote the music for the different performances (AN: the song that accompanies the story of Maritgen, the girl who pretends to be a boy, is inspired by old Dutch sailors songs).

The musicians and the girls all come from Amsterdam's alternative art scene. The musicians, in particular, are all improvisers, even though this specific music was composed. They are used to working together and have been doing so for a long time, so they really complement each other.

A. C. - If you watch the cellist at the right time, he will also give you a private performance in addition to the song he plays for the girls. Not everyone sees the same experience: all users see the four women, because they are the central element of the story, but depending on where you look you might trigger different interactions: for example, if you catch the bartender's eye a second time she might try to explain the paintings and tell you which girls are available or unavailable.

A. - The sounds, the music, the paintings... It's the 17th century all around! Look at the clothes the girls are wearing - which, by the way, are really beautiful!

A. A. - I made all the clothes myself in the summer with my mother helping me! (laughs) I bought old clothes second-hand and then changed everything. I really wanted the girls to wear old fabrics and not things that looked brand new.

A. - All these details show the deep care you have put into this work and which I, as an audience member, found essential to immerse myself in the experience.

A. C. - This whole work has been a labour of love. We've had a lot of support from the Creative Industries Fund, The Netherlands FilmFund and the Amsterdam Fund for the Arts, but if you want to do something that really explores this medium you have to be dedicated to it. You have to be willing to invest a lot of your personal time, blood, sweat and tears - in a very literal way - to go that far, and I think that's been a really crucial part for me in collaborating with Anna.

A lot of creators who approach us have an idea that they want us to make and then they disappear for months. But if you really want to explore this medium and do it with works like Angels of Amsterdam, you have to be equal partners, both willing to put in the extra effort to make it happen.

Anna was one of the few and rare people who, from the beginning, had that level of dedication: when that happens, WeMakeVR will go above and beyond what we have to do to get the best result.

A - Avinash, I still remember one of your talks at the Biennale College years ago. It was about VR and storytelling and you were so enthusiastic about it that it was impossible not to end up fascinated by this medium as well. I have to ask you: what is your history with VR and why do you love it so much?

A. C. – Everything I've done in my life up to this point comes together in VR. As a kid, I always imagined what it would be like to step through that screen and become part of those worlds I loved so much. This fascination has stayed with me all my life. I have also always been interested in the concept of VR and what I knew about it from movies and books. When I was a teenager, I even tried to create a prototype headset to explore it!

In 2012 my VFX studio received an early prototype of the first Oculus headset to explore what we could do with it. And that's when we started creating very simple demos and video games and then invented one of the first professional VR cameras that we got a patent on. That was how we started and from there things just snowballed: suddenly it was possible for me to imagine something in my mind and put myself in that world and also share it with others. Literally a dream come true.

That is why I am not passionate about specific works, but about the medium in general. People in this field are not just creating interesting pieces of art or entertainment. I think we are all sitting on technology and technological evolution that will fundamentally change our world: how will we look at education? How will we look at mutual understanding? How will we look at art? How will we look across cultural boundaries? There is an infinite realm of possibilities!

A. – In fact the works you produce with WeMakeVR are very different: Angels of Amsterdam is a completely different piece from, say, Jason Moore’s The Metamovie Presents: Alien Rescue.

A. C. - All these different types of works are part of the exploration of this new language of VR. We have more than a century of film history, but with VR we are still exploring all these new rules - or lack of rules. We don't have a vocabulary yet and we need to develop it.

Every project we do, whether it's a game, an opera, a work of art, Angels of Amsterdam itself, helps expand that vocabulary and helps us become better creators. But it's also about sharing that knowledge! That's why, specifically, I'm spending a lot of time tutoring for La Biennale or speaking at conferences. So we can help build the industry. This is my passion and my reason for being here.

A. C. - When we talk about storytelling many people still think of linear media where the user is sitting back. Someone tells you a story and you just listen to it. Whereas in VR the dynamic changes. We make an experience, we create worlds, and our audience is a guest in those worlds, and we empower them, we give them the choice to look where they want to look.

Over the last six years, we've seen a number of filmmakers venture into the 360 video medium, and try to cling to the classic rules, 'This is my shot, this is where I want you to look, this is my pace'. But now we're learning that those rules don't work: a lot of directors have decided that this medium is not for them, and instead other people who don't necessarily come from the film industry but from theatre or art or other very different fields are finding their new voices in this medium.

I think everything we know about narrative experiences is about to change. We're now seeing a wave of new students fresh out of school approaching this medium with a completely new blank slate: they're not held back by any convention. So, what we're doing now is innovative, but we're going to see completely new paradigm-shifting experiences in the next 5 to 10 years from creators who are approaching this medium from a completely different perspective.

A. - What do you see happening?

A. C. - There are a lot of creators who are into programming and are working on smell and taste tools to make VR a more sensory experience - and that's great, but I think it's going to be even more interesting to learn how the brain works, how it fills in the blanks in a VR experience, and what we need to do with the medium to make you feel more present without adding all these little gadgets.

We'll get to a point where we can basically recreate real life visually. And when technology is no longer a factor, it really comes back to the creator and our creative vision. Then we can really start to talk about imagination and come up with things that we have never even seen with our own eyes.

A - What path did Angels of Amsterdam take after the Venice Film Festival?

A. A. – We presented Angels of Amsterdam at the Nederlands Film Festival at the end of September and now we are submitting it to several festivals around the world and waiting to hear back from them. The challenge now is to take it out into the world, which is difficult, because you need a good computer and very good headsets and this technology is still so new that not many people have it.

A. C. - Exactly. Angels of Amsterdam is such a technologically advanced work that it's difficult for some people to experience it in the resolution and format we want them to experience it in, because not everyone has that kind of capability at home. I think this is the flip side of doing something that is really cutting edge: can everyone see it?

Anyway, I'm glad the story resonated with the people who saw it. Now we will work on making it more accessible without compromising too much on quality. After all, that's what we, as an industry, have to do: make it more accessible.

Angels of Amsterdam is a coproduction by WeMakeVR and Stichting Rongwrong, supported by Filmfonds, Stimuleringsfonds and the Amsterdam Funds for the Arts (AFK) and directed by Anna Abrahams and Avinash Changa.

Find out more about it on the official website and on its XRMust Database page.