“AR to provide context for understanding the natural environment” - Dane Christensen (KVÖLDVAKA)
Kvöldvaka is a fascinating augmented reality app for iOS that brings folklore and magic to life and helps you reconnect with nature.
If you've ever watched lightning cross the sky outside your window, or walked through a field of flowers in the middle of the mountains, or slept under the stars, you probably remember the feeling of contact with something beyond you that you felt in those moments. The landscape around you came alive and was no longer just a place you were in, but a companion - or even an adversary, depending on the situation.
A couple of years ago, I watched The Seer and the Unseen, an Icelandic documentary about Ragga, a seer who possesses "second sight, the ability to communicate directly with a parallel realm of elves".
As I watched it, I realized something: while this feeling of connection with nature is something we only remember to feel once in a while, there are places in the world - and people - who are aware of it every single moment. Especially in countries like Iceland, where the documentary is set and nature feels powerful and uncontrollable and devastating in its beauty and wilderness.
Elves and hidden people, who are such a big part of Ragga's life, are just a few of the creatures that populate Icelandic fascinating folklore. You may or may not believe in them, yet you can't help but be touched by this woman's soul and inspired by her way of looking at the world.
Her relationship with nature appeals to you and the need to recognize yourself as an integral part of the environment we are destroying. Something that has stuck with me in the years since I saw this work.
That’s why, when I read about Kvöldvaka, an app presented in the lineup of the CPH:DOX 2021 and described as a “playful interactive Augmented Reality documentary inspired by Icelandic folklore”, I knew it was a work I had to know more about.
I joined the live premiere (available at this link), where artists Dane Christensen, director of the piece and hologram designer, Carol Dalrymple, co-director and creative technologist, and Barna Szász, producer, interaction & narrative designer, presented their work.
The event was very engaging: Terry Gunnell, Professor in Folkloristics at the University of Iceland, opened it with a fascinating insight on the origins of Icelandic legends and on the Kvöldvaka, the evening wake in which all members of a household would sit together around the fire in their Hobbit-like homes to tell each other stories of things they had seen and experienced. The audience at the live premiere was asked to actively participate in some "experiments" with nature, and one sneaky member of the hidden people actually made themselves known. Seer Ragnhildur “Ragga” Jónsdóttir was also present to discuss her connection to the hidden people and the need for us humans to open our eyes and see beyond what we take for granted to recreate a connection between us and everything around us.
The event made me even more curious to know more about this app, currently available for Phone 11 or newer or iPad Pro (2018 or newer). I contacted the artists and here’s what Dane Christensen told me.
An app to connect you with the hidden folks: Kvöldvaka at CPH:DOX
AGNESE - First of all, what’s your personal story with XR?
DANE CHRISTENSEN - We are a team with several years of experience producing documentaries, installations and art projects in XR.
While I was doing my MFA at Stanford in 2015, the New York Times sent Google Cardboards to all its subscribers. This was my first time experiencing 360 documentaries and since that time, I’ve produced 360 films, XR installations and taught XR workshops around the world.
For Kvöldvaka I’m joined by two incredible colleagues, Carol Dalrymple and Barna Szász. Carol has an extensive history with XR. As an Oculus fellow, Carol has a unique ability to think outside the frame and runs the non-profit Edge of Discovery - a media arts initiative focused on discovering and amplifying voices historically silenced and minimized. Native Hungarian Barna is a fellow graduate of the Stanford Doc Film MFA program and has a keen eye for narrative design and storytelling in XR.
A. - The Icelandic folklore is one of the most fascinating. When and why did you decide to focus your app on it? And why did you choose AR, specifically, to talk about the ties that binds us humans to nature?
D. C. - I’m a dual citizen, Danish and American, and have a degree in Scandinavia Studies. I’ve known since my first experience with VR in 2015 that I wanted to produce a project about the Huldufólk folklore. There is something magical about the folk belief and the liveliness of it that has always resonated with me.
I researched the folklore while I waited for the right technology to become available. Originally this project was designed as an installation with AR headsets like the Magic Leap or Hololens, but we pivoted the project to smartphones because it fit better with the essence of the folklore. Now, people can experience Kvöldvaka outside and feel connected to their nature.
Research from the Stanford Virtual Human Interaction Lab (a/n here on XRMust) shows that augmented reality has the ability to alter your mental image of a space both during and after you’ve experienced something in AR. You literally respond differently to a space because of context provided by an AR experience. This research is at the core of Kvöldvaka.
Legend and landscape have always been connected and ancient folklore is the original form of augmented reality as it provided context for understanding the natural environment. In Kvöldvaka, you embark on an interactive journey to connect with hidden beings living in the nature around you. And we hope that you’ll come out of the experience protecting those beings where you interacted with them as well as wherever you are in nature.
A. - What kind of research did you do on Huldúfolk and Icelandic folklore? What are, in this regard, the roles of Ragnhildur “Ragga” Jónsdóttir and Terry Gunnell?
D. C. - Kvöldvaka is based on years of our own research and relies on the first-hand experiences with the Huldufólk as well as the academic study of folkloristics.
We always knew Kvöldvaka was an AR project, but how exactly to represent the Huldufolk was the longest process. This is because the Huldufólk are most often caricatured by international media as small elves with pointy hats and shoes when this isn’t accurate to the folk belief.
So, in an effort to mitigate this incorrect perception and to remain authentic to how Icelanders, such as Ragga, perceive the Huldufólk, we represent the hidden people as energy holograms in Kvöldvaka - playing into the theme of connected energies in nature.
This is where the involvement of Ragga and Terry have been crucial in the development of the project. Ragga, a Seer, has seen and communicated with the hidden people since childhood. She has described at length the beings she sees. And Terry is a leading researcher within folkloristics. He articulates the problem of caricaturization and fetishization of the Huldufólk folk belief and has consulted with us on how to authentically convey the folklore as a living and essential part of Icelandic society.
A. - For those of us who don't have an iphone and are curious to know more about Kvöldvaka even if they cannot experience it directly: how would you describe this work?
D. C. - Kvöldvaka is an interactive journey. What you’ll experience in the CPH:DOX prototype is a narrative where you interact with a member of the Huldufólk and as you prove yourself worthy, through a series of exercises in which you observe your environment, this member of the Huldufólk reveals themselves to you and shares an important message.
The augmented reality aspect isn’t just what you’ll see on your phone but what you sense as you see, hear, touch and even smell the nature around you. In the full version of Kvöldvaka, the experience will be different each time you open the app and immersive you in the hidden dimension of the Huldufólk.
Technical characteristics of Kvöldvaka
A. - Can you tell me more about how you worked on this app from a technical standpoint? And what are the most significant breakthroughs and challenges each one of you faced?
D. C. - We could easily spend hours elaborating on this question.
Aside from the Huldufólk folk belief, the use of augmented reality is what makes this experience work.
However, the challenge wasn’t just the process of technically making it all work - which was massive - but how to design a user experience for a medium that most people have little to no experience with. You can’t assume anything with the user knowing how to interact in AR.
Like swiping up on an iPhone to return to the home screen, there are implicit modes of interaction that we don’t have to think about. But with AR, nearly no interaction standards have been established. So we had to consider every detail in the experience so the user never feels “FOMO” (fear of missing out), or not knowing what to do next, and balance that with the choices and interaction the user has along the journey.
Audio is crucial to how we perceive our spaces and it is both a tool for delivering instructions to the user as well as driving the narrative. We had to think about how the Huldufólk communicates and how to implement it with the AR tools available on mobile. We wanted to deliver a sense that your space is alive and depending on your movements, you hear different things.
Now from a technical standpoint, there are many layers of technology working together to power Kvöldvaka. And if you’re familiar with XR production or game design, even the smallest of interaction design can be a merciless technical challenge. Simply put, Kvöldvaka marries mobile AR interaction and volumetric capture (holograms) to produce the experience.
We are pilot partners with Depthkit Studio and use volumetric capture to visualize the Huldufólk as holograms. Kvöldvaka is one of the first story-driven LiDAR-enabled smartphone-AR experiences, the Huldufólk appear as if they are pressing up against the boundary between our physical world and their hidden dimension. You’ll immediately sense a living essence in their magical, playful behavior.
In conjunction with real-time spatial processing on mobile, the Huldufólk holograms respond to your environment and react to your movements in real time. We thought it was important that we use the technology in this manner so the Huldufólk feel embedded in your nature - like how they are represented in the folklore.
AR as a medium to tell stories and share meanings
A. - It's still rare to see stories told using exclusively AR. Why is this the case, in your opinion, and what about the future?
D. C. - AR is still an emerging medium. Both in the sense of what it is, but also how we create with it. I think we will see much more exploration with the medium as these tools become more available.
The medium lends itself well to altering or adding context and understanding of spaces. In the same way statutes and monuments cast a certain meaning to a location, AR can add context to any space. That’s incredibly powerful. What is built physically in front of us reinforces a certain perspective and narrative. With AR, we will be able to see multiple perspectives and understandings of the same spaces we thought we knew.
A. - Kvöldvaka is not just a fun app to play with. It's also one that is thematically relevant to our times and shows how everything in the world is connected. Can you comment on that?
D. C. - I think ancient folklore is the original form of augmented reality. Ancient stories were, and still are, a way to connect with the landscape and cultivate a relationship with nature.
Industrialization and scientific innovation have distanced us from nature - and the magical realism associated with the environment. For example, I can name any five brands just by their logos, but can’t name the leaves from five different trees.
Before our modern era, folklore kept a sense of magic alive. Which is why using modern technology, like mobile AR, to reconnect with nature is really great. Kvöldvaka flips the devices that usually distract us into tools to reconnect with ancient folklore and rediscover the magic in nature. I believe that the path forward in an era of climate crisis is both a scientific and spiritual one.