#Épisode 3 | Explore Tumblr posts and blogs

canmom · 7 months ago

Text

l'aventure de canmom à annecy - épisode deux: XR

so I'm going to abandon all semblance of chronological order at this point.

just like last year there was an VR room operating on a morning booking system - each thing had two headsets and a signup sheet. I didn't get to try every one but I did get most of them. enough that the volunteers noticed me coming back every day x3

^ some random people immersed in the Wired

this is easily the longest annecy post so far... so please read on for a big old discussion of the unique difficulties faced by VR film as a medium, and how this year's annecy films meet them... or more often don't.

so annecy's vr films section is for essentially linear vr projects (i refuse to write "experiences") that can be watched/played in less than an hour. i don't know if that's a hard rule but that's how long all the ones here were!

let's get the technical stuff out of the way: the Quest 2/3 was by far the headset of choice. some ran natively, some were PCVR with a wire connecting to a computer, and some were 360° videos which played back on the headset with 3dof tracking. some had a degree of interactivity, up to about a 'walking simulator' level. the average runtime was between 20m and an hour.

the preamble: on the limitations of VR

the big question I have with XR movies is basically... how well does it actually use the medium? like, is it doing anything that wouldn't work better as a flatscreen game or a film?

this might seem like a high bar to clear, like why shouldn't it be in VR - but VR is uncomfortable, the headset is expensive etc etc, and that's before even the formal stuff I'm about to get into. so that's 'why not'. and also, this is a new medium, I want to see what unique features it has to offer!

I'm sure I've said this before, but despite on the face of it being more 'inmersive' than traditional flatscreen games or films, VR is actually a pretty restrictive medium! compared to flatscreen games with their many 'buttons', you are very limited in the possible interactions. your main interaction is to 'pick up' and 'hold' objects, but this is close enough to actual physical interaction to highlight how much it isn't. what it actually means is that you position your hand or controller in a trigger and press a grab button or pinch your fingers, at which point the object snaps to your hand and moves weightlessly with it.

you also can't accelerate the pov too much without causing motion sickness, etc etc.

ok, what about film? well, compared to film, the big big thing VR lacks is the frame of the camera. you can't cut, you can't frame a subject, you don't have long shots or closeups, you can't even rely on the player/viewer looking in the direction of an interesting thing.

since movement is also tricky in VR due to the motion sickness problem, you're also limited in your ability to steer the viewer to interesting sights with Valve-style 'vistas' using the level architecture. it's not impossible - Valve themselves have their familiar vistas in HL Alyx - but it's something that depends on the player being able to move through a large space, so it doesn't fit these kinds of movie-like project so well. otherwise you can draw attention to a direction using various means, like visual effects that converge on a spot, or just keeping most of the action in the same area.

what you can also do, closer to camerawork, is move the viewer's point of view, and shrink or enlarge their surroundings. the language of VR 'shots' is still far from defined, but we have a few recurring ones: standing in a normal sized room, the giant's view in a tiny city, the floating perspective looking down on a diorama, the ant's eye view inside something regular sized.

how about theatre, which also has most of these limitations? well, compared to being in person with a real human being, you're limited by the capabilities of realtime animation systems and the rendering tech available on the device. you're looking at the character in a slightly fuzzy low resolution and unless you have AAA money which noone in VR does, you're a bit limited in the 'acting' you can pull off. this may change if the apple vision pro gets popular - apple already have a 'gertie the dinosaur' style demo where a very detailed dinosaur emerges from a portal - but it's definitely out of reach of most teams working on the Quest.

so compared to all these other media, what does XR offer?

compared to film and theatre, there is the game aspect of agency: a story feels different if you are the one doing it. so most VR narrative games have characters interact with the player somehow, though this introduces the problem of how to write the player into a story without feeling like you're railroading them or that they're superfluous to the real story.

it's very easy to undercut this sense of agency by having an amount that's not zero but still too small, e.g. if it just feels like the player is touching a button that lights up then you wonder why they even bothered.

the role that most games put the (vaguely defined) player character in is 'terrifying violence doer'. this is a fairly easy role to write around, and it gives the player a lot of control of the 'how' while letting the writer control the 'what' and 'where' and 'why'. similarly if the player's role is something like 'puzzle solver'. but for a purely narrative presentation, these roles don't exist.

still, this is the idea that a lot of VR rests on: an 'immersive experience' which puts the player into the story.

the other big thing that VR has is the joy of experiencing visual effects in 3D. particles, trails, transforming geometric shapes etc are cool on a flat screen and even cooler in stereo vision where you can move your head around. another benefit is spatial audio by default - something that is gradually coming to games but provided 'for free' by vr consoles.

in the land of games, you also have incredibly precise position and direction input... as long as it's in arm's length of the player. the most successful genres of VR games (so far) use this: a lot of shooting games, and some games that let you interact with physics objects, offer 3D jigsaw puzzles, or simulate sports to provide some real exercise. it can be really good for rhythm games as beat saber demonstrates.

VR is also really for social games like VRChat - similar to MMOs but with the benefits of more complex tracking in lieu of canned animations.

but... none of these fit the form of a predictable 15-50 minute narrative sequence! they're not films! so the VR films category at annecy is a tough problem to crack.

last year, the VR project that most impressed me was one that put you in the seat of a novice spotter in a bomber in the second world war. this was a great fit for a lot of reasons. you are in a vehicle so you have no reason to move from seated; the scenario is full of loud scary sounds that can make full use of spatial audio; your 'character' is well-defined but also doesn't have much reason to speak within the scenario. this had a small amount of interaction (by pointing your head) which you could actually fail, making it a bit closer to a game, and also giving you reason to play close attention to the bombed out cityscape below you. it did a fantastic job of capturing the tension of a dangerous air mission, the pilot character interacting with you was compelling, and overall it really benefited from being in VR.

this year's films

sadly nothing I saw this year comes close to that. but still, some are interesting, so let's go through them!

My favourite this year was probably Flow - completely unrelated to the movie in the main competition, though they share the trait of being completely wordless, conveying their story simply through imagery and music.

Flow's big trick is a very cool visual effect where characters and objects are conveyed the trails left behind by little particles, causing them to appear ghostlike . At first you're just flying through a cityscape, passing various people on the street and in the subway, with the particle trails conveying the breaths of the passengers; gradually a storm brews, the trails becoming the wind that tears at everything.

I believe this was a prerendered film, with only 3dof tracking - i think i saw some compression artefacts at some point. So it's less technically impressive than if they managed to do it realtime but it does make full use of the power this gives to render loads of particles and move rapidly through different scenes. it was also an effect that benefitted from the ability to put you in the middle of it - something that would not work as well on a flat screen.

But it also benefitted a lot from being more film-like. It has an original soundtrack, and progresses without input from the player. There's no awkward 'player' character to write around, no space you stand about in. The film can simply unfold and let you appreciate it. In this case, no interaction is better than bad interaction.

My Inner Ear Quartet from Japan did not do anything particularly novel with the medium, but to my mind it had by far the most compelling story. It tells of a young, introverted boy who habitually digs in the dirt for objects that other people would consider trash. The title refers to a string quartet which he hears when he cries, imagined to be in his inner ear; also there is a pair of tiny shrimp which he saw grow in a net.

The first half is narrated by a man who turns out to be the boy grown up, now a hearing aid salesman. While the boy abandons his box of treasures, as an adult he returns to collecting and documenting abandoned objects as a kind of urban explorer.

The geometry here is stylised in a kind of rough, children's drawing way. I think this could have been pushed further with more complex shaders but it works. For the most part, you're watching as an invisible observer seated on a floating chair. At certain points, the viewpoint is taken inside the boy's ear, or into the tin of treasures, where you can grab the objects and get the boy's brief, poetic description of each one.

I liked this story because it had substance, but left enough up to interpretation to be engaging. By showing the treasures to us with the descriptions we get to understand why they might be significant to the boy. It plays well with the classic anime theme of objects as vessels for emotional significance. I think it would have worked just as well on a flat screen, but I enjoyed my time with it.

Now the rest...

The Age of the Monster had some things going for it, but honestly I think this one shouldn't have been in VR. It's basically a film about how bad we're fucking up the planet, putting us in the house of a man who works in the nuclear industry in the 70s up to a future where the cooling towers lie in ruins. The 'monster' is a giant anthro catfish, seen first as a B-movie monster and dream vision and finally as a real kaiju scale creature in the final future scene.

We're told about the economic circumstances that led to the man getting this job, and his relief at working in nuclear during the oil crisis; we're told about the infamous repressed oil industry report about how climate change is gonna be a thing; we're told about the man's fraught relationship with his radical daughter who is furious about his extractivist ways. Then we get a collapse and humans learn to take the force of nature more seriously, i forget the exact phrasing they used.

The main problem is? These are mostly things you are told, by voiceover. There is some environmental storytelling in the evolution of the house but not enough to convey much without the v/o. the film does not seem to have the confidence in its imagery to show us what it's trying to say.

I feel like the film's vision of the post-collapse future, with flocks of birds flying over a wide river and collapsed overgrown cooling towers, is a huge missed opportunity. Here's an opportunity to apply some true visual imagination of how humans might live in a climate changed future... but nah, giant catfish kaiju just kinda hanging out there.

The environmental message is generally a stance I sympathise with, but the film doesn't make a good case for it on a propaganda level. We see the cooling towers outside the window and eventually the house, flooded, but it does little to make the collapse narrative emotionally compelling - and I question a little the choice to make it nuclear focused in a film about climate change. It's probably based on an actual guy, right? Maybe someone's parent? But... despite putting us in his shoes i don't really feel like i understand him very well.

Does this seem harsh? I know full well how involved vr dev is, and even simple things can take weeks. But i also want someone to make the most of this medium. To make something as compelling as the best short films on the main screens.

Gargoyle Doyle tells another 'skipping though time' story, depicting a gargoyle on an 800 year old church from its construction through to demolition. I would compare this one perhaps to a puppet theatre - it certainly stands out in terms of character animation, with Doyle played by Jason Isaacs as a classic grumpy old British theatre guy, his foyle foil a goofy statue of monk acting as a drainpipe with a penchant for puns.

I didn't get to watch this one in full, since I got to sub in for someone who left early (thanks to the volunteer who took pity on me when it was fully booked lol). So I didn't see the full arc of this. What I saw was... definitely edutainment material, but pretty well done. The player is cast as a visitor to a future museum and nature reserve built on the site of the church. It seems like this was originally shown in a real museum in Venice, with the 'in the museum' sections portrayed in mixed reality; obvs this wouldn't work at Annecy so they have these virtual too.

The narrative as a whole seems a tad self congratulatory and pat, with Doyle learning a valuable lesson about not being a cunt to his only friends as he's resurrected in the museum, and it doesn't do a whole lot with the VR framing, but taking it as an educational puppet show, it works pretty well - the voice performances are good and the jokes, while a little predictable, work for the kid-friendly style it's going for. I'm not sure it really needed to be 40 minutes long, but I can see they wanted to go maximalist for a proof of concept like this. It is kinda limited by the rendering capabilities of the Quest, the lack of shadows in particular, and could definitely benefit from some baked lighting given the relatively static scenes, but I give it a lot of credit for the character animation and VA.

Apparently the jury liked this one too because it won the competition!

Nana Lou has been in development a few years apparently, casting the player in the role of a psychopomp spirit whose role is to ease the passing of a woman dying of a stroke. Visually, this is one of the best looking, with elaborate forest scenes and strong environment design.

What I really like about this one is its use, at times, of a diorama-like presentation where the player looks down on tiny characters in a room. This is a concept I've wanted to try in VR for a while, and it's cool to see someone do it.

I found the kind of spiritual aspect of the story a lot more underwhelming. The player is accompanied by two other spirits who explain everything that's going on and point out the significance of all the imagery. The player is informed they have an important role, but they don't have a name and can't talk back, and the only interaction is to grab floating photos to initiate flashbacks.

I wish this film had had the confidence to trust in its acting and visual storytelling. While Nana Lou's life is a bit too lacking in serious conflict to make the premise work, it would still be far more interesting and compelling with the frame story largely trimmed. You could still cast the player as a psychopomp but you don't need to have a greek chorus telling them what to click on!

The actual story concerns Lou's relationship with her daughter, who became estranged when she quit university to raise her child, instead of staying on as Lou thought she should. This caused them to spend decades estranged. Finding out this story frees up the daughter's spirit as well, and the penultimate scene has her speak to Lou and make up.

There's definitely something to work with there, but the main delivery mechanism is rather ponderous narration triggered by interacting with objects, with the dramatic scenes largely having taken place off screen. Like The Age of the Monster, it suffers a severe telling-not-showing problem.

It's a shame because there are nice touches here. When you are beside Lou's bed in the hospital, your touch leaves a glow effect which is very evocative. The acting is solid, though the script undermines it a bit.

I don't think narration is evil - evidently, Yuri and I used it in our film, it's a very efficient way to convey information - but I do think it requires a lot of thought put into style and rhythm.

Spots of Light... ok. This one tells the story of an Israeli soldier who lost his sight in the Lebanon war, and later regained it temporarily through surgery. Given my general feelings about the Israeli military (presently carrying out a genocide), I was definitely not disposed to like this one. Nevertheless, it was the only one free so I decided to give it a shot.

This is one of those films where you interview someone and then put an animation to it. So this guy tells you what it was like to be blind and then not blind to see his family briefly, and it's illustrated with various images. And (if i remember right) some parts are on tvs showing video (and if you're using vr to embed a flat screen what is even the point??). When he's blind, everything disappears except vague outlines suggested by small points of light.

Ultimately this is a film about blindness, not the war (of course, meaning this is a person who could leave the war behind - though not to make light of the cruelty of conscription). Making a film about the experience of blindness in a purely visual medium is a choice all right, and I don't feel like this film expressed anything unexpected about it - he was sad to lose his sight, glad to see his family, depressed to lose it again but ultimately at peace. Which is conveyed, of course, primarily by narration.

So yeah this one didn't do much for me!

Stay Alive My Son, now. Agh. This one was... this one was a mess.

So this one is about the Cambodian genocide, right. It's based on a memoir by a survivor of the genocide, Pin Yathay, who became separated from his wife and son while fleeing the Khmer Rouge.

The way this is presented is essentially a walking simulator that takes you through a dungeon-like environment full of skeletons. Every so often you encounter 3D films - filmed with some kind of depth camera - showing actors playing out scenes from the life of the family. There is also a frame story where you visit Pin Yathay in his modern day house, where he sees a digital reconstruction of how his son might look as an adult

This one is difficult to review because it was severely marred by technical issues with the spatial audio, which caused the sound to cut out when you turned your head the wrong way or moved it to the wrong place. It would probably be less of a rough experience if the audio worked as intended. Nevertheless, I have plenty of reservations with the way the story is told as well.

It seems the director of the VR experience (fine! I'll write experience, there isn't a better noun for this kind of thing that sits between game and film) met Pin Yathay, there's video of her speaking to him at the end, but he had pretty minimal creative input beyond providing inspiration through his memoir. So this is a Greek/US interpretation of the Cambodian genocide. The narrative it tells is basically: Yathay and his family are living a pretty idyllic life, then the Khmer Rouge happens, seemingly not for any particular reason. Yathay and his family are evacuated and then put to hard labour growing rice; eventually, their son is put to work too, so fearing for his life, they flee into the jungle.

The Cambodian genocide is - obviously! - one of the worst atrocities of the whole bloody 20th century, and the circumstances surrounding it are worth reading about (though pretty unremittingly bleak). But you won't learn much about, say, cold war geopolitical alignment, Prince Sihanouk, the absolutely horrific civil war, the different ideologies in play in the Marxist milieu that influenced Pol Pot, or the spillover from Vietnam and the massive bombing by the Americans here which helped put the Khmer Rouge in power. You definitely won't learn much about the Cambodia that existed before the war. Instead, you're mostly traversing a dungeon that could come from any horror game, shining your torch on the things you're told to in order to unlock another segment of narration from (the actor playing) Yathay. It is, in pretty literal terms, a tour of atrocities.

Unfortunately the '3D film of actors' conceit doesn't really work because... even audio issues aside, the acting is pretty unconvincing. For some reason - perhaps that subtitles are tricky in VR - the dialogue is in accented English rather than Cambodian, and it's pretty quickly evident that they just have one guy in the role of 'Khmer Rouge soldier' and the lines he's given are kinda awkward. The horror game aesthetics of the environments and the amateur actors and costumes all clash pretty badly. The 3D filming is also kind of jank, only really working if you're fairly close to the camera position, so you aren't really free to move too far even if the audio didn't crap out.

The basic feelings it's trying to explore - the horror of living through a genocide, separation from a child, guilt for abandoning him, not knowing if he's alive or dead - is definitely worth depicting, but honestly this would have been far far better expressed as a 10-20 minute film than a slow 55 minute VR walking sim. The more abstract bits toward the end with paper plane imagery and a Buddhist temple (where you have to put a block in a slot to unlock a door) also feel too jank and videogamey to really have much impact, though by that point I had been wrestling with the audio for nearly an hour so I wasn't in the most receptive mood.

But all the execution flaws aside, that leaves the question of what even is the right way to portray a genocide artistically? This approach is very abstract, reducing the events to dislocated symbols - propaganda posters, the tree against which children were dashed - which perhaps might reflect how fragmented memory becomes, but seems to be wasting the potential of VR to establish you in a place. But then, I guess rice fields are harder to render than enclosed dark rooms.

Speaking of rendering, this was PCVR, so your torch casts shadows and it has other features that would be hard on realtime. But the lack of ambient light and general harshness of the materials adds to the 'horror game' feel.

There is something here about how genocides become associated with certain images. For Cambodia, it is primarily phrases like 'killing fields' and the stacked skulls in the genocide memorials such as Choeung Ek - few people know the name of the memorial in the west, but I think everyone who's heard of the genocide has seen the big stack of skulls. I imagine this is what all the skeletons in this experience are supposed to call to mind: they're representatives of the many ways people died. The problem that this kind of environmental storytelling has long ago been made kind of camp by videogames. A photo of a stack of real skeletons still has power to disturb, but less so a low poly 3D skeleton.

Should it have tried for a realism? The idea of trying to realistically simulate the experience of living in Cambodia though the genocide is kind of ghoulish, and I'm glad they didn't take that approach. But the 'tour of images' approach falls flat. I think The Most Precious of Cargoes elsewhere in the festival makes a stronger case for how to approach a topic like a genocide in a consciously constructed way, but it also has the ability to be in dialogue with a lot of other films made about the Holocaust. There is less in English about the Cambodian genocide - the viewer can't even be assumed to know what happened.

Overall, I think it would be possible to make a much stronger film about the genocide in Cambodia. But I'm not sure what that film would look like. I did learn one thing from this story, which is that there is a reality show in Cambodia which shows survivors of the genocide being reunited with their families. Not much is made of this here, it's something of a background detail. What would it be like to grow up in the shadow of an event like that? I wish the film had been willing to portray more of modern Cambodia - and I hope at some point someone in Cambodia will have a film at this festival, in VR or not, which can talk about it all from the first person.

Is there a good way to try to answer the curiosity of people who live safely in rich countries about what it is like to go through an actual genocide... using a Meta Quest 2 VR headset that costs a few hundred quid? I don't really know, but this film could have done with being a bit more reflective, I feel. So it goes.

the others

there were three films I couldn't see - The Imaginary Friend, Oto's Planet and Emperor. If I get some other chance to try them I'll write about them too!

Overall I felt a bit disappointed with the VR this year, but also I kind of want to put my money where my mouth is and try my hand at making this kind of thing. I do have the technical knowledge at least!

If you read all this, thank you.

#l'aventure de canmom à annecy #annecy 2024

16 notes · View notes