XRMust_Sonaria_Still01.jpg
January 03rd, 2021 | by Mathieu Gayet

"With Google Spotlight Stories I decided VR was the most exciting place in the world for me" - Scot Stafford (Pollen Music Group)

The following interview includes such little-known names as The Simpsons, Gorillaz or Aardman (and more). We take a look back at the adventure of Google Spotlight Stories with Maestro Scot Stafford, a talented composer who now collaborates on many animation and immersive projects with his company Pollen Music Group.

Music, a life and passion with curiosity

Scot Stafford - I've been a musician all my life. I grew up split between studying classical music, being in rock bands and doing a lot of experimental music as well. So I became a composer, because I can create storytelling and emotions on my own. I was lucky enough to meet a lot of creative people, including one of the most talented animators at Pixar named Doug Sweetland - he ended up directing a theatrical short at Pixar called PRESTO, that was just absolutely phenomenal. And he asked me to do the music for it, which was extremely challenging at the time, because it was a wall-to-wall big orchestral score. A few years later, he called me for a project with a skunkworks team at Motorola ATAP (“Advanced Technology And Projects” division).

S. S. - It was very confusing to me because Motorola was a tech company, sort of the opposite of Pixar, but in fact they were a bunch of highly creative people (including Jan Pinkava who co-wrote RATATOUILLE) conducting a research and development division inside Motorola led by Regina E. Dugan (ex-DARPA), with a very fast prototyping process around innovative ideas. This was around 2012, and they were already looking to create a new storytelling format for mobile devices. They brought me in very early to talk about sound and music for a 360 mobile format and it blew my mind. After that I actually had trouble driving after it because I had just entered a very different spatial reality: a portal to a completely 3D immersive world all around us. Like a magic window!

S. S. - Many months later this led to the first Google Spotlight Story about a mouse, WINDY DAY (link), which had almost everything. And... I ended up working on 17 Google Spotlight stories over the next six years! Almost everything that we discovered later, we actually discovered on that first one, which was as soon as you can look anywhere, how do you tell people where to look? There's a long list of technologies that were built for the very first Story that we used after that. There was innovation in every single Story that came out, even if it was never the same director twice!

XRMust_WindyDay_Still01.jpg

S. S. - This first Story was also the realization of the elevated importance of music and sound. The director gives up so much control to the user in virtual reality. How do you (the creative team) get some of it back? You're still limited to what you can see, which is a small sort of “cone” of the world around you, BUT you can hear everything hear/behind/above/below you.

A composer’s creative role at Google Spotlight Stories

S. S. - On immersive projects I was very fortunate to be invited into conversations much, much earlier than other projects - usually sound and music come at the end of the production process. And it was then I decided this was the most exciting place in the world for me! Every three or four months, there was a new project being pitched, a new director and storyteller coming in. I got to see so many different things from live action to animation, from Mobile 360 to VR to eventually AR - so many different perspectives on storytelling from different voices. And it felt like the next one was always going to be the best. It was a very exciting time. Everything had to be re-imagined: how to work around cuts, shot composition

S. S. - With Google Spotlight Stories, it was particularly exciting to see music and sound being spoken about on the same level as story and visual (see the recording making-of for AGE OF SAIL). A lot of people did make a comparison to Pixar. And in fact many creative people working on Stories came from TOY STORY or A BUG’S LIFE. And early CG animation and VR followed similar paths, starting with tech demos and proofs of concept, maybe one scene in a movie or a commercial, it was sort of this novelty. But then we felt we could tell emotionally powerful stories in a totally new way, laying the train tracks in front of us as we go. This was a huge challenge! Almost all of them felt like they weren't going to work until the last minute. In particular PEARL, a very music driven Story, was broken and just didn’t work at all until just a couple weeks before it premiered at Tribeca. I do think it’s the kind of film where a lot of people that were dismissive of VR started to consider it seriously. And it was this piece that actually convinced THE SIMPSONS creator James Brooks to try VR! PEARL was a milestone, certainly for the team working on it but also for the emotions, creative ideas we understood we could create.

A look back at Google Spotlight Stories

S. S. - All Stories are so special to me. Because of the relationships that I built there and the number of projects, I was one of the only creative people that worked on all of them. I was given the opportunity to pitch my own Spotlight Story, and I had a very small window of time because we were wrapping up on others productions. And we were going to start working on a Gorillaz music video (!), which I was very excited about, in London with Jamie Hewlett - who I think is an absolute genius. That was a very busy time, so I came up with the idea of SONARIA which was very sound and music driven - very heart driven. The fact that we were able to do it so quickly and the people seem to love it was very special to me.

S. S. - The funniest by far is ON ICE! I just think it's absolutely hilarious. But I think the one that's sort of the cleverest to me was SPECIAL DELIVERY that we did with Aardman Studios. It was a Christmas special, which is a very dangerous thing to try to do because these things take time and are unpredictable. Having a deadline like this was one of the hardest things we've ever done. But their approach to interactivity was amazing - and what is required of music. If you're watching the main story, there are things happening behind you that aren't just standalone Easter eggs. There are Easter eggs in fact, but they are multi-chaptered Easter egg storylines. They're just such masters of the story - and being charming and funny with it! And then we had to figure out a way to have a jazz band 180 degrees behind the main story’s field of view. If you're looking at the main narrative as you turn, the score had to blend seamlessly with an entirely different jazz score behind.

A future for immersive storytelling?

S. S. - A lot of the studios that were really trying to pioneer storytelling in VR have folded. They no longer do it. I think what's happening is, on a smaller scale, similar to the dotcom boom and bust of the late 90s. There was this frantic, overblown hype. About how the Internet was going to completely revolutionize everything. And money was being thrown everywhere. The expectations were unrealistic in terms of just how quickly something that new can roll out. And all of a sudden it went from being “this is going to change everything” to being “this is worthless”. And then a few years after that, the revolution happened even bigger than they anticipated. It just took longer. With VR there's still so many misconceptions about it, but I have hope. We already have some highly creative studios in the immersive studio, from the former Oculus Story Studio to Baobab Studios and others - which I’ve also worked with. They have shown that great stories can be told in VR.

S. S. - All of these investors, large tech companies, I just don't know what they were thinking. Google and Facebook. I think by any metric of success an artist can think of, we did it. With Google Spotlight Stories we proved you can do this thing and it can keep growing, be something really amazing. But it just didn't happen in just the right way, just fast enough for them to keep funding it, which, of course, is very sad. Producing narrative experiences is very hard, yes. We don't have a viable market place for it. It's so expensive, so much labor to tell a five to 15 minute story. In a similar way, there's not really a marketplace for shorts even in traditional media.

S. S. - But narrative VR is going to be a thing. It might be five years down the road, maybe even longer, for people to really start not just adopting it and trying to make content for it, but for people to get really excited about it. It's taking more time than people were hoping.

What’s next, Pollen Music Group?

S. S. - Pollen is built on 2 divisions: a research and development one, and a creative one - which is producing sound and music for traditional and immersive works. We just finished two immersive projects with Baobab Studios. One is BABA YAGA by Mathias Chelebourg and Eric Darnell, with an amazing voice casting including Glenn Close, Kate Winslet, Jennifer Hudson and Daisy Ridley. The idea of being in a magical, familiar, yet completely mysterious and unfamiliar forest that's rich in storytelling and folklore that felt like, OK, this is a wonderful place for VR to be. It’s a very ambitious and interactive film. The second project is NAMOO (“Tree” in Korean) from the same studio, directed by Erick Oh. It will have its world premiere at Sundance (link). In both cases we were very lucky to work on completely bespoke versions of each story - an interactive immersive version, and a traditional 2d film version. We are mixing both at our new Atmos mixing studio in San Francisco, and can’t wait for all four releases!

XRMust_Namoo_poster.jpg

S. S. - This is a wonderful thing to do as a creator, to work with creative people that do understand where artistic integrity of the project is. We learned so much from our previous VR projects or Mixed Reality on Magic Leap. We had a great time working on the musical experience OUT THERE with Thibault Mathieu (see interview) at Wilkins Avenue in 2019. What they accomplished with such a limited budget was extraordinary. They prioritize music so early in the process, which is something only big studios usually do: it’s a great proof that independent studios are on the right path! At Pollen we have the feeling that we arrived somewhere with maturity on sound design and music for immersive storytelling. There's the artistic sensibility and taste and maturity that comes from the discovery, the innovation. And what was so hard to do at spelling stories is to discover and sort of invent something or create something that had never been done before. Or at least that we weren't aware of having been done before. And then to be tasteful with it right away is very hard. That was sort of the big artistic exercise for me.

S. S. - While I can’t be too detailed about this, we are working on fixing some mistakes in our original approach to spatial audio. We fell in love with the novelty of being able to hear everything everywhere, but it also created some problems, and we're trying to correct this. I hope to share more about this soon!

Share article

Related posts

XRMust_NewAesthetics_still01.jpg
May 18, 2021

“Sunny Side of the Doc has always been an event where new projects are born” - Stephane Malagnac (PiXii Festival)

For the 5th edition of PiXii Festival, dedicated to new media, Sunny Side of the Doc in La Rochelle (France) is exploring new horizons in a future-oriented hybrid 2021 format: a professional market c…

XRMust_AnOdeToMoss_Still01.jpeg
June 10, 2021

The NIFFF 2021 goes deep into XR with its Sauvage Extended line-up

The NIFFF (Neuchâtel International Fantastic Film Festival) is a must-attend event in the world of genre cinema. From July 2 to 10, the festival will host a selection of immersive experiences... that…

XRMust_Kandara_Still01.jpg
May 5, 2021

“Notre festival doit être un laboratoire international de la création immersive” - Michael Swierczynski (NewImages Festival)

Moins d’un an sépare les deux éditions 2020 et 2021 de NewImages Festival, le rendez-vous de la création digitale et mondes virtuels produit par le Forum des images à Paris depuis 2018. Après une édi…