January 17th, 2023 | by Mathieu Gayet

“I believe very much in the notion of story-living in interactive design, even for AR” - Barna Szász (IF THESE STREETS COULD TALK)

This is one of the projects that occupied the line-ups of the co-production forums in 2022. NewImages, GIFF, goEast, IDFA... IF THESE STREETS COULD TALK is an augmented reality experience that proposes to go back to the past of our cities, and history, to better honour a duty of remembrance on the Second World War. A heritage work that fits perfectly with the vocation of new uses: not to forget. Back on this project in the making with its creator, Barna Szász.

Barna Szász is a Budapest-born filmmaker & XR storyteller. He moved to the U.S. in 2017 on a Fulbright and graduated from Stanford University’s MFA Documentary Film program in 2019. He has been teaching XR at Stanford since 2019. (about)

Developing the XR discourse in non-fiction

Barna Szász - I come from a documentary background, in the traditional sense, but I’ve always tried experimenting with new forms and formats. Some of my films were featured by PBS, the Guardian... I did my master’s at Stanford University, in the heart of Silicon Valley, so it was geographically hard for me to escape virtual reality and its immersive uses! I took all the courses related to XR, and luckily they quickly needed someone to teach these subjects to the upcoming classes. So I was able to improve my knowledge of the sector very quickly, read the dedicated books, and experience the existing works... Since then, I have focused on the sector, and there is no turning back! The XR, as a creator, raises brand new questions, and in quantity.

B. Sz. - If I look back at the experiences that have marked me, I obviously cite NOTES ON BLINDNESS: INTO DARKNESS by Arnaud Colinart, Amaury La Burthe, Peter Middleton and James Spinney. But a fundamental project for me remains THE HERO VR (link), a multi-sensory immersive installation that questions the format, the physicality of the exchange, as much as the content (notably the ethics in VR). I could also mention THE FIGHT FOR FALLUJA by Ben C. Solomon for the New York Times, CLOUDS OVER SIDRA by Gabo Arora and Chris Milk, and in augmented reality BREATHE by Diego Galafassi (link) or DEAR VISITOR by Stanford alums (link).


B. Sz. - In my opinion, 360 video – though it was a necessary intermediary format – is not the future of XR. Instead, the ability to recreate reality through technological tools and interactions, rather than copying static reality via traditional video, is one of the main features that differentiates XR from other mediums. Art creation is evolving. So is the documentary. Initially, in the documentary we directly captured reality. Now we reproduce a version of reality in virtual environments so you can “play” in it, experience it. We can now have actors (or live performers) in documentary experiences. In a way this is similar to the tradition of reenactment in classical documentary film, but now we make these reenactments and the recreated worlds interactable. It is all part of a cycle of research on the documentary genre that evolves with each project.

B. Sz. - I believe very much in the notion of story-living, rather than storytelling, like other creators such as Celine Tricart. I rely a lot on participatory experiences, where a story is not told to you (and you are witnessing it from the outside) but the story happens with you (and you are one of the main characters in the story). I think this is going to be super exciting for the documentary genre.


IF THESE STREETS COULD TALK, a project from abroad

B. Sz. - After being away from my country of origin I suddenly moved back to Budapest where I live in the Jewish quarter of the city. I was looking at the same streets where I had lived before but with renewed, refreshed eyes. And suddenly I realized that most of its history is now totally invisible. Even though it’s called the “Jewish” Quarter, you don’t see many signs of its Jewish history. I wanted to recreate a link with history in the broadest sense of the term. IF THESE STREETS COULD TALK comes from this idea of visually and narratively recreating (in augmented reality) the streets of the early 20th century. Today, there is almost nothing left except for a few clues on street corners, in certain places. I collected stories, did visual research and began my work on the invisibility of this heritage. Even coming from a family that has lived in this environment, many parts of this history are still taboo to this day –the transmission of history remains difficult, even limited. But finally, we have a new medium to address this!


B. Sz. - I was beaten up by Neo-Nazis in Budapest before, and you can see an emergence of far-right politics in several countries (especially in Europe) which also alarmed me about the importance of this discussion between today and yesterday. We need to understand our past to better understand our present. I wanted to make history visible, and having already collaborated on an AR project (KVÖLDVAKA - see our interview with Dane Christensen), I knew all the possibilities. Now we can offer a place-based experience, with specific anchors. My XR designer mind then started the process of imagining what IF THESE STREETS COULD TALK could be.

B. Sz. - I still teach at Stanford and I am following the Stanford Virtual Human Interaction Lab (VHIL)'s work on augmented reality closely. Right now, AR is – in some sense – a much more exciting medium than VR! They have proven that AR can change your perception of the surrounding space. You can enrich an existing space, with several layers of perception - and memories! This is my ambition with IF THESE STREETS COULD TALK, to be able to give a more complex meaning to the streets of my city, and provoke a live exchange. We've already done some very successful tests at Stanford, on campus. So it's not just a theory!


Designing a collective XR project

B. Sz. - In parallel to the project itself, which started from my Hungarian-Jewish roots, on a taboo subject back home, I wanted to extend it to several cities to also give more visibility to our world heritage - or European at first. And it makes sense! Every major European city has Jewish neighbourhoods, and each time they have become almost invisible in the public space. And yet they had their importance! And their relevance to our current times. I have already approached several institutions or museums that could be potential partners.

B. Sz. - Finding images of these old neighbourhoods is not a problem. Videos, that's probably harder - by the logic of age. I was inspired by MIT's research on ethical deep fakes, even if it's not central to the project. But to include certain characters or settings, it can be a way to recreate a moment in history. You can take 2D historical photos of people and create 3D versions of them so you can represent them as actual people who are walking on the streets. For the moment, we have gone around the co-production markets to find partners. Holocaust foundations may be interested, and there is a real discussion about the interactive aspect of the subject - which remains very sensitive, so for them, this could be a very compelling pilot project The chapter in Budapest should start production in 2023, before expanding our ambitions to other cities.


B. Sz. - At the same time, I remain an interactive designer on KVÖLDVAKA where I continue to explore the balance of interactions with the audience. And bring it closer to real life, to our everyday gestures. AR is a medium that is still not much explored, not much seen in festivals, even though I see so much potential in it.


In this article

Share article

Related posts

January 9, 2023

VRROOM launches its first metaverse platform dedicated to live performance

On Tuesday, January 10, 2023, at 8pm, on the occasion of the launch of the first French metavers platform dedicated to live performances, in partnership with Deezer, Warner Music, The Orchard and Wag…

December 11, 2020

Editorial⎪MYST is back, this time in virtual reality!

The year is 1993, PC games are not really trying to look better since the '80s, 3D graphics are quite limited and the point-and-click genre is dominating the CD-ROM titles on shelves. A small yet ver…

December 20, 2021

Review⎪THE INFINITE: ⁠A Giant Leap for XR

THE INFINITE is not only one of the most visually stimulating XR exhibitions in recent memory but also, and perhaps surprisingly, a remarkably moving experience.